mercurial-scm/hg-stable: mercurial/util.py annotate

annotate mercurial/util.py @ 52669:e627cc25b6f3

pyupgrade: rewrite `yield` statements in a loop to `yield from` This is the `legacy` fixer in `pyupgrade`, with the `yield` statement yielding loop commented back in. This seems to help pytype in some cases, and hurt it in others. But that can be manually fixed later. Note that it's possibly buggy in that it aggressively changed `import-checker.py` to `yield from 'fcntl', 'grp', 'pwd', 'select', 'termios': # Unix only`, which is invalid syntax. Possibly it needed help from the token fixer that I've disabled locally (because that wants to make a bunch of unrelated changes). Just change those few places to yield from a list, to avoid having to constantly revert that.

author	Matt Harbison <matt_harbison@yahoo.com>
date	Sun, 05 Jan 2025 22:26:16 -0500
parents	5cc8deb96b48
children	30510238284e

rev	line source
17515 b5b38d21fe99 spelling: specific timeless@mozdev.org parents: 17237 diff changeset	1 # util.py - Mercurial utility functions and platform specific implementations
8226 8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	2 #
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	3 # Copyright 2005 K. Thananchayan <thananck@yahoo.com>
46819 d4ba4d51f85f contributor: change mentions of mpm to olivia Rapha?l Gom?s <rgomes@octobus.net> parents: 46714 diff changeset	4 # Copyright 2005-2007 Olivia Mackall <olivia@selenic.com>
8226 8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	5 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	6 #
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	7 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 9996 diff changeset	8 # GNU General Public License version 2 or any later version.
1082 ce96e316278a Update util.py docstrings, fix walk test mpm@selenic.com parents: 1081 diff changeset	9
17515 b5b38d21fe99 spelling: specific timeless@mozdev.org parents: 17237 diff changeset	10 """Mercurial utility functions and platform specific implementations.
1082 ce96e316278a Update util.py docstrings, fix walk test mpm@selenic.com parents: 1081 diff changeset	11
8227 0a9542703300 turn some comments back into module docstrings Martin Geisler <mg@lazybytes.net> parents: 8226 diff changeset	12 This contains helper routines that are independent of the SCM core and
0a9542703300 turn some comments back into module docstrings Martin Geisler <mg@lazybytes.net> parents: 8226 diff changeset	13 hide platform-specific details from the core.
1082 ce96e316278a Update util.py docstrings, fix walk test mpm@selenic.com parents: 1081 diff changeset	14 """
419 28511fc21073 [PATCH] file seperator handling for the other 'OS' mpm@selenic.com parents: diff changeset	15
51901 f4733654f144 typing: add `from __future__ import annotations` to most files Matt Harbison <matt_harbison@yahoo.com> parents: 51888 diff changeset	16 from __future__ import annotations
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	17
33814 bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	18 import abc
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	19 import collections
33446 fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	20 import contextlib
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	21 import errno
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	22 import gc
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	23 import hashlib
48964 5aafc3c5bdec py3: use io.BytesIO directly Gregory Szorc <gregory.szorc@gmail.com> parents: 48961 diff changeset	24 import io
34554 6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	25 import itertools
44826 1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	26 import locale
34302 3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34154 diff changeset	27 import mmap
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	28 import os
49068 225659936fff util: restore the util.pickle symbol Matt Harbison <matt_harbison@yahoo.com> parents: 49037 diff changeset	29 import pickle # provides util.pickle symbol
21907 7e5dfa00e3c2 util: rename 're' to 'remod' Siddharth Agarwal <sid0@fb.com> parents: 21857 diff changeset	30 import re as remod
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	31 import shutil
30428 1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	32 import stat
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	33 import sys
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	34 import time
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	35 import traceback
51780 ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	36 import typing
31950 cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	37 import warnings
3769 96095d9ff1f8 Add encoding detection Matt Mackall <mpm@selenic.com> parents: 3767 diff changeset	38
51302 9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	39 from typing import (
51587 a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	40 Any,
51780 ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	41 BinaryIO,
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	42 Callable,
51302 9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	43 Iterable,
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	44 Iterator,
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	45 List,
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	46 Optional,
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	47 Tuple,
51924 1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51917 diff changeset	48 Type,
1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51917 diff changeset	49 TypeVar,
51302 9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	50 )
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	51
47055 d55b71393907 node: replace nullid and friends with nodeconstants class Joerg Sonnenberger <joerg@bec.de> parents: 46926 diff changeset	52 from .node import hex
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	53 from .thirdparty import attr
51788 278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51780 diff changeset	54
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51780 diff changeset	55 # Force pytype to use the non-vendored package
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51780 diff changeset	56 if typing.TYPE_CHECKING:
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51780 diff changeset	57 # noinspection PyPackageRequirements
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51780 diff changeset	58 import attr
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51780 diff changeset	59
39287 e00123f63410 util: make timedcm require the label (API) Augie Fackler <augie@google.com> parents: 39286 diff changeset	60 from hgdemandimport import tracing
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	61 from . import (
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	62 encoding,
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	63 error,
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	64 i18n,
32406 a9c71d578a1c osutil: switch to policy importer Yuya Nishihara <yuya@tcha.org> parents: 32346 diff changeset	65 policy,
28818 6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence timeless <timeless@mozdev.org> parents: 28497 diff changeset	66 pycompat,
51780 ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	67 typelib,
34468 192f7b126ed2 urllibcompat: move some adapters from pycompat to urllibcompat Augie Fackler <augie@google.com> parents: 34436 diff changeset	68 urllibcompat,
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	69 )
51974 fa7059f031a9 interfaces: introduce and use a protocol class for the `base85` module Matt Harbison <matt_harbison@yahoo.com> parents: 51967 diff changeset	70 from .interfaces import (
fa7059f031a9 interfaces: introduce and use a protocol class for the `base85` module Matt Harbison <matt_harbison@yahoo.com> parents: 51967 diff changeset	71 modules as intmod,
fa7059f031a9 interfaces: introduce and use a protocol class for the `base85` module Matt Harbison <matt_harbison@yahoo.com> parents: 51967 diff changeset	72 )
37086 f99d64e8a4e4 stringutil: move generic string helpers to new module Yuya Nishihara <yuya@tcha.org> parents: 37085 diff changeset	73 from .utils import (
42041 3e47d1ec9da5 util: extract compression code in `mercurial.utils.compression` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 41839 diff changeset	74 compression,
44060 a61287a95dc3 core: migrate uses of hashlib.sha1 to hashutil.sha1 Augie Fackler <augie@google.com> parents: 44036 diff changeset	75 hashutil,
37121 5be286db5fb5 procutil: move process/executable management functions to new module Yuya Nishihara <yuya@tcha.org> parents: 37120 diff changeset	76 procutil,
37086 f99d64e8a4e4 stringutil: move generic string helpers to new module Yuya Nishihara <yuya@tcha.org> parents: 37085 diff changeset	77 stringutil,
f99d64e8a4e4 stringutil: move generic string helpers to new module Yuya Nishihara <yuya@tcha.org> parents: 37085 diff changeset	78 )
3769 96095d9ff1f8 Add encoding detection Matt Mackall <mpm@selenic.com> parents: 3767 diff changeset	79
51302 9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	80 # keeps pyflakes happy
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	81 assert [
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	82 Iterable,
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	83 Iterator,
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	84 List,
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	85 Optional,
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	86 Tuple,
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	87 ]
46684 7711853110b9 typing: add some type annotations to mercurial/util.py Matt Harbison <matt_harbison@yahoo.com> parents: 46114 diff changeset	88
7711853110b9 typing: add some type annotations to mercurial/util.py Matt Harbison <matt_harbison@yahoo.com> parents: 46114 diff changeset	89
51974 fa7059f031a9 interfaces: introduce and use a protocol class for the `base85` module Matt Harbison <matt_harbison@yahoo.com> parents: 51967 diff changeset	90 base85: intmod.Base85 = policy.importmod('base85')
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	91 osutil = policy.importmod('osutil')
32406 a9c71d578a1c osutil: switch to policy importer Yuya Nishihara <yuya@tcha.org> parents: 32346 diff changeset	92
32245 4462a981e8df base85: proxy through util module Yuya Nishihara <yuya@tcha.org> parents: 32203 diff changeset	93 b85decode = base85.b85decode
4462a981e8df base85: proxy through util module Yuya Nishihara <yuya@tcha.org> parents: 32203 diff changeset	94 b85encode = base85.b85encode
4462a981e8df base85: proxy through util module Yuya Nishihara <yuya@tcha.org> parents: 32203 diff changeset	95
31934 12aca6770046 util: make cookielib module available Gregory Szorc <gregory.szorc@gmail.com> parents: 31878 diff changeset	96 cookielib = pycompat.cookielib
30480 00c9ac4ce816 util: rewrite pycompat imports to make pyflakes always happy Yuya Nishihara <yuya@tcha.org> parents: 30451 diff changeset	97 httplib = pycompat.httplib
37102 6ca5f825a0ca util: make safehasattr() a pycompat function Yuya Nishihara <yuya@tcha.org> parents: 37101 diff changeset	98 safehasattr = pycompat.safehasattr
30480 00c9ac4ce816 util: rewrite pycompat imports to make pyflakes always happy Yuya Nishihara <yuya@tcha.org> parents: 30451 diff changeset	99 socketserver = pycompat.socketserver
48964 5aafc3c5bdec py3: use io.BytesIO directly Gregory Szorc <gregory.szorc@gmail.com> parents: 48961 diff changeset	100 bytesio = io.BytesIO
36964 644a02f6b34f util: prefer "bytesio" to "stringio" Gregory Szorc <gregory.szorc@gmail.com> parents: 36952 diff changeset	101 # TODO deprecate stringio name, as it is a lie on Python 3.
644a02f6b34f util: prefer "bytesio" to "stringio" Gregory Szorc <gregory.szorc@gmail.com> parents: 36952 diff changeset	102 stringio = bytesio
30480 00c9ac4ce816 util: rewrite pycompat imports to make pyflakes always happy Yuya Nishihara <yuya@tcha.org> parents: 30451 diff changeset	103 xmlrpclib = pycompat.xmlrpclib
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28882 diff changeset	104
34468 192f7b126ed2 urllibcompat: move some adapters from pycompat to urllibcompat Augie Fackler <augie@google.com> parents: 34436 diff changeset	105 httpserver = urllibcompat.httpserver
192f7b126ed2 urllibcompat: move some adapters from pycompat to urllibcompat Augie Fackler <augie@google.com> parents: 34436 diff changeset	106 urlerr = urllibcompat.urlerr
192f7b126ed2 urllibcompat: move some adapters from pycompat to urllibcompat Augie Fackler <augie@google.com> parents: 34436 diff changeset	107 urlreq = urllibcompat.urlreq
192f7b126ed2 urllibcompat: move some adapters from pycompat to urllibcompat Augie Fackler <augie@google.com> parents: 34436 diff changeset	108
32131 377c74ef008d win32mbcs: avoid unintentional failure at colorization FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 31952 diff changeset	109 # workaround for win32mbcs
377c74ef008d win32mbcs: avoid unintentional failure at colorization FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 31952 diff changeset	110 _filenamebytestr = pycompat.bytestr
377c74ef008d win32mbcs: avoid unintentional failure at colorization FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 31952 diff changeset	111
34645 75979c8d4572 codemod: use pycompat.iswindows Jun Wu <quark@fb.com> parents: 34554 diff changeset	112 if pycompat.iswindows:
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	113 from . import windows as platform
14912 ec46a7da9f2c util: move windows and posix wildcard imports to begin of file Adrian Buehlmann <adrian@cadifra.com> parents: 14911 diff changeset	114 else:
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	115 from . import posix as platform
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	116
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	117 _ = i18n._
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	118
47622 bb917eea1605 windows: introduce a `util.abspath` to replace os.path.abspath Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47452 diff changeset	119 abspath = platform.abspath
29530 3239e2fdd2e2 chgserver: extract utility to bind unix domain socket to long path Yuya Nishihara <yuya@tcha.org> parents: 29455 diff changeset	120 bindunixsocket = platform.bindunixsocket
14927 2aa3e07b2f07 posix, windows: introduce cachestat Idan Kamara <idankk86@gmail.com> parents: 14926 diff changeset	121 cachestat = platform.cachestat
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	122 checkexec = platform.checkexec
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	123 checklink = platform.checklink
15011 5e44e4b3a0a3 util: move copymode into posix.py and windows.py Adrian Buehlmann <adrian@cadifra.com> parents: 15010 diff changeset	124 copymode = platform.copymode
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	125 expandglobs = platform.expandglobs
35517 c4caf530b1c7 util: add a function to show the mount point of the filesystem Matt Harbison <matt_harbison@yahoo.com> parents: 35515 diff changeset	126 getfsmountpoint = platform.getfsmountpoint
35513 beede158ea8a util: move getfstype() to the platform modules Matt Harbison <matt_harbison@yahoo.com> parents: 35464 diff changeset	127 getfstype = platform.getfstype
47038 5b3513177f2b util: avoid echoing the password to the console on Windows py3 (issue6446) Matt Harbison <matt_harbison@yahoo.com> parents: 47031 diff changeset	128 get_password = platform.get_password
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	129 groupmembers = platform.groupmembers
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	130 groupname = platform.groupname
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	131 isexec = platform.isexec
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	132 isowner = platform.isowner
32248 d74b0cff94a9 osutil: proxy through util (and platform) modules (API) Yuya Nishihara <yuya@tcha.org> parents: 32245 diff changeset	133 listdir = osutil.listdir
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	134 localpath = platform.localpath
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	135 lookupreg = platform.lookupreg
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	136 makedir = platform.makedir
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	137 nlinks = platform.nlinks
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	138 normpath = platform.normpath
15488 6eff984d8e76 dirstate: fix case-folding identity for traditional Unix Matt Mackall <mpm@selenic.com> parents: 15392 diff changeset	139 normcase = platform.normcase
24605 98744856b7d3 util: add normcase spec and fallback Siddharth Agarwal <sid0@fb.com> parents: 24439 diff changeset	140 normcasespec = platform.normcasespec
98744856b7d3 util: add normcase spec and fallback Siddharth Agarwal <sid0@fb.com> parents: 24439 diff changeset	141 normcasefallback = platform.normcasefallback
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	142 openhardlinks = platform.openhardlinks
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	143 oslink = platform.oslink
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	144 parsepatchoutput = platform.parsepatchoutput
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	145 pconvert = platform.pconvert
25420 c2ec81891502 util: add a simple poll utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25406 diff changeset	146 poll = platform.poll
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	147 posixfile = platform.posixfile
39909 5fe0b880200e py3: convert os.readlink() path to native strings on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 39793 diff changeset	148 readlink = platform.readlink
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	149 rename = platform.rename
24692 144883a8d0d4 util: add removedirs as platform depending function FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24635 diff changeset	150 removedirs = platform.removedirs
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	151 samedevice = platform.samedevice
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	152 samefile = platform.samefile
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	153 samestat = platform.samestat
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	154 setflags = platform.setflags
17560 9ee25d7b1aed util: implement a faster os.path.split for posix systems Bryan O'Sullivan <bryano@fb.com> parents: 17537 diff changeset	155 split = platform.split
18026 ddc0323db78b osutil: write a C implementation of statfiles for unix Bryan O'Sullivan <bryano@fb.com> parents: 18013 diff changeset	156 statfiles = getattr(osutil, 'statfiles', platform.statfiles)
18868 cafa447a7d3b util: add functions to check symlink/exec bits Bryan O'Sullivan <bryano@fb.com> parents: 18775 diff changeset	157 statisexec = platform.statisexec
cafa447a7d3b util: add functions to check symlink/exec bits Bryan O'Sullivan <bryano@fb.com> parents: 18775 diff changeset	158 statislink = platform.statislink
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	159 umask = platform.umask
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	160 unlink = platform.unlink
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	161 username = platform.username
14912 ec46a7da9f2c util: move windows and posix wildcard imports to begin of file Adrian Buehlmann <adrian@cadifra.com> parents: 14911 diff changeset	162
44633 d37975386798 chgserver: update the umask cache before each run Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	163
51924 1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51917 diff changeset	164 if typing.TYPE_CHECKING:
1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51917 diff changeset	165 _Tfilestat = TypeVar('_Tfilestat', bound='filestat')
1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51917 diff changeset	166
1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51917 diff changeset	167
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	168 def setumask(val: int) -> None:
47031 f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1 Kyle Lippincott <spectral@google.com> parents: 46926 diff changeset	169 '''updates the umask. used by chg server'''
44633 d37975386798 chgserver: update the umask cache before each run Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	170 if pycompat.iswindows:
d37975386798 chgserver: update the umask cache before each run Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	171 return
d37975386798 chgserver: update the umask cache before each run Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	172 os.umask(val)
d37975386798 chgserver: update the umask cache before each run Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	173 global umask
d37975386798 chgserver: update the umask cache before each run Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	174 platform.umask = umask = val & 0o777
d37975386798 chgserver: update the umask cache before each run Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	175
d37975386798 chgserver: update the umask cache before each run Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	176
42041 3e47d1ec9da5 util: extract compression code in `mercurial.utils.compression` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 41839 diff changeset	177 # small compat layer
3e47d1ec9da5 util: extract compression code in `mercurial.utils.compression` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 41839 diff changeset	178 compengines = compression.compengines
3e47d1ec9da5 util: extract compression code in `mercurial.utils.compression` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 41839 diff changeset	179 SERVERROLE = compression.SERVERROLE
3e47d1ec9da5 util: extract compression code in `mercurial.utils.compression` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 41839 diff changeset	180 CLIENTROLE = compression.CLIENTROLE
3e47d1ec9da5 util: extract compression code in `mercurial.utils.compression` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 41839 diff changeset	181
6470 ac0bcd951c2c python 2.6 compatibility: compatibility wrappers for hash functions Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 6339 diff changeset	182 # Python compatibility
3769 96095d9ff1f8 Add encoding detection Matt Mackall <mpm@selenic.com> parents: 3767 diff changeset	183
15656 4f5a78fa4917 util: clean up function ordering Matt Mackall <mpm@selenic.com> parents: 15611 diff changeset	184 _notset = object()
4f5a78fa4917 util: clean up function ordering Matt Mackall <mpm@selenic.com> parents: 15611 diff changeset	185
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	186
30745 c1b7b2285522 revlog: flag processor Remi Chaintron <remi@fb.com> parents: 30736 diff changeset	187 def bitsfrom(container):
c1b7b2285522 revlog: flag processor Remi Chaintron <remi@fb.com> parents: 30736 diff changeset	188 bits = 0
c1b7b2285522 revlog: flag processor Remi Chaintron <remi@fb.com> parents: 30736 diff changeset	189 for bit in container:
c1b7b2285522 revlog: flag processor Remi Chaintron <remi@fb.com> parents: 30736 diff changeset	190 bits \|= bit
c1b7b2285522 revlog: flag processor Remi Chaintron <remi@fb.com> parents: 30736 diff changeset	191 return bits
c1b7b2285522 revlog: flag processor Remi Chaintron <remi@fb.com> parents: 30736 diff changeset	192
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	193
31950 cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	194 # python 2.6 still have deprecation warning enabled by default. We do not want
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	195 # to display anything to standard user so detect if we are running test and
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	196 # only use python deprecation warning in this case.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	197 _dowarn = bool(encoding.environ.get(b'HGEMITWARNINGS'))
31950 cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	198 if _dowarn:
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	199 # explicitly unfilter our warning for python 2.7
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	200 #
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	201 # The option of setting PYTHONWARNINGS in the test runner was investigated.
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	202 # However, module name set through PYTHONWARNINGS was exactly matched, so
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	203 # we cannot set 'mercurial' and have it match eg: 'mercurial.scmutil'. This
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	204 # makes the whole PYTHONWARNINGS thing useless for our usecase.
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	205 warnings.filterwarnings('default', '', DeprecationWarning, 'mercurial')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	206 warnings.filterwarnings('default', '', DeprecationWarning, 'hgext')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	207 warnings.filterwarnings('default', '', DeprecationWarning, 'hgext3rd')
48988 10839394f39b util: remove superfluous ispy3 test Gregory Szorc <gregory.szorc@gmail.com> parents: 48966 diff changeset	208 if _dowarn:
36617 281f66777ff0 py3: silence "bad escape" warning emitted by re.sub() Yuya Nishihara <yuya@tcha.org> parents: 36614 diff changeset	209 # silence warning emitted by passing user string to re.sub()
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	210 warnings.filterwarnings(
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	211 'ignore', 'bad escape', DeprecationWarning, 'mercurial'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	212 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	213 warnings.filterwarnings(
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	214 'ignore', 'invalid escape sequence', DeprecationWarning, 'mercurial'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	215 )
37455 9ecb7c471cfb py3: silence warning about deprecation of imp module Yuya Nishihara <yuya@tcha.org> parents: 37382 diff changeset	216 # TODO: reinvent imp.is_frozen()
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	217 warnings.filterwarnings(
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	218 'ignore',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	219 'the imp module is deprecated',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	220 DeprecationWarning,
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	221 'mercurial',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	222 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	223
31950 cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	224
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	225 def nouideprecwarn(msg, version, stacklevel=1):
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	226 """Issue an python native deprecation warning
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	227
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	228 This is a noop outside of tests, use 'ui.deprecwarn' when possible.
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	229 """
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	230 if _dowarn:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	231 msg += (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	232 b"\n(compatibility will be dropped after Mercurial-%s,"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	233 b" update your code.)"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	234 ) % version
36166 02ed94dd9fd6 util: call warnings.warn() with a sysstr in nouideprecwarn Augie Fackler <augie@google.com> parents: 36076 diff changeset	235 warnings.warn(pycompat.sysstr(msg), DeprecationWarning, stacklevel + 1)
45009 7af5c1f5b3a0 util: flush stderr explicitly after using warnings.warn() Pulkit Goyal <7895pulkit@gmail.com> parents: 44927 diff changeset	236 # on python 3 with chg, we will need to explicitly flush the output
7af5c1f5b3a0 util: flush stderr explicitly after using warnings.warn() Pulkit Goyal <7895pulkit@gmail.com> parents: 44927 diff changeset	237 sys.stderr.flush()
31950 cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	238
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	239
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	240 DIGESTS = {
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	241 b'md5': hashlib.md5,
44060 a61287a95dc3 core: migrate uses of hashlib.sha1 to hashutil.sha1 Augie Fackler <augie@google.com> parents: 44036 diff changeset	242 b'sha1': hashutil.sha1,
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	243 b'sha512': hashlib.sha512,
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	244 }
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	245 # List of digest types from strongest to weakest
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	246 DIGESTS_BY_STRENGTH = [b'sha512', b'sha1', b'md5']
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	247
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	248 for k in DIGESTS_BY_STRENGTH:
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	249 assert k in DIGESTS
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	250
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	251
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49031 diff changeset	252 class digester:
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	253 """helper to compute digests.
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	254
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	255 This helper can be used to compute one or more digests given their name.
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	256
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	257 >>> d = digester([b'md5', b'sha1'])
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	258 >>> d.update(b'foo')
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	259 >>> [k for k in sorted(d)]
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	260 ['md5', 'sha1']
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	261 >>> d[b'md5']
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	262 'acbd18db4cc2f85cedef654fccc4a4d8'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	263 >>> d[b'sha1']
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	264 '0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	265 >>> digester.preferred([b'md5', b'sha1'])
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	266 'sha1'
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	267 """
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	268
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	269 def __init__(self, digests, s=b''):
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	270 self._hashes = {}
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	271 for k in digests:
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	272 if k not in DIGESTS:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	273 raise error.Abort(_(b'unknown digest type: %s') % k)
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	274 self._hashes[k] = DIGESTS[k]()
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	275 if s:
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	276 self.update(s)
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	277
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	278 def update(self, data):
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	279 for h in self._hashes.values():
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	280 h.update(data)
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	281
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	282 def __getitem__(self, key):
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	283 if key not in DIGESTS:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	284 raise error.Abort(_(b'unknown digest type: %s') % k)
46114 59fa3890d40a node: import symbols explicitly Joerg Sonnenberger <joerg@bec.de> parents: 45957 diff changeset	285 return hex(self._hashes[key].digest())
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	286
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	287 def __iter__(self):
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	288 return iter(self._hashes)
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	289
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	290 @staticmethod
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	291 def preferred(supported):
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	292 """returns the strongest digest type in both supported and DIGESTS."""
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	293
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	294 for k in DIGESTS_BY_STRENGTH:
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	295 if k in supported:
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	296 return k
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	297 return None
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	298
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	299
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49031 diff changeset	300 class digestchecker:
22963 56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	301 """file handle wrapper that additionally checks content against a given
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	302 size and digests.
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	303
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	304 d = digestchecker(fh, size, {'md5': '...'})
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	305
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	306 When multiple digests are given, all of them are validated.
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	307 """
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	308
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	309 def __init__(self, fh, size, digests):
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	310 self._fh = fh
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	311 self._size = size
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	312 self._got = 0
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	313 self._digests = dict(digests)
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	314 self._digester = digester(self._digests.keys())
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	315
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	316 def read(self, length=-1):
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	317 content = self._fh.read(length)
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	318 self._digester.update(content)
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	319 self._got += len(content)
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	320 return content
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	321
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	322 def validate(self):
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	323 if self._size != self._got:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	324 raise error.Abort(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	325 _(b'size mismatch: expected %d, got %d')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	326 % (self._size, self._got)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	327 )
22963 56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	328 for k, v in self._digests.items():
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	329 if v != self._digester[k]:
23076 c312ef382033 i18n: add hint to digest mismatch message Wagner Bruna <wbruna@softwareexpress.com.br> parents: 23030 diff changeset	330 # i18n: first parameter is a digest name
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	331 raise error.Abort(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	332 _(b'%s mismatch: expected %s, got %s')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	333 % (k, v, self._digester[k])
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	334 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	335
22963 56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	336
11565 7546d4a272c8 util: improved the check for the existence of the 'buffer' builtin Renato Cunha <renatoc@gmail.com> parents: 11469 diff changeset	337 try:
46684 7711853110b9 typing: add some type annotations to mercurial/util.py Matt Harbison <matt_harbison@yahoo.com> parents: 46114 diff changeset	338 buffer = buffer # pytype: disable=name-error
11565 7546d4a272c8 util: improved the check for the existence of the 'buffer' builtin Renato Cunha <renatoc@gmail.com> parents: 11469 diff changeset	339 except NameError:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	340
33549 9a2ee9591acc util: remove dead code which used to be for old python2 versions Alex Gaynor <agaynor@mozilla.com> parents: 33446 diff changeset	341 def buffer(sliceable, offset=0, length=None):
9a2ee9591acc util: remove dead code which used to be for old python2 versions Alex Gaynor <agaynor@mozilla.com> parents: 33446 diff changeset	342 if length is not None:
51888 f5c46c3518a5 util: make buffer readonly Rapha?l Gom?s <rgomes@octobus.net> parents: 51843 diff changeset	343 view = memoryview(sliceable)[offset : offset + length]
f5c46c3518a5 util: make buffer readonly Rapha?l Gom?s <rgomes@octobus.net> parents: 51843 diff changeset	344 else:
f5c46c3518a5 util: make buffer readonly Rapha?l Gom?s <rgomes@octobus.net> parents: 51843 diff changeset	345 view = memoryview(sliceable)[offset:]
f5c46c3518a5 util: make buffer readonly Rapha?l Gom?s <rgomes@octobus.net> parents: 51843 diff changeset	346 return view.toreadonly()
10756 cb681cc59a8d util: fake the builtin buffer if it's missing (jython) Ronny Pfannschmidt <Ronny.Pfannschmidt@gmx.de> parents: 10487 diff changeset	347
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	348
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	349 _chunksize = 4096
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	350
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	351
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49031 diff changeset	352 class bufferedinputpipe:
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	353 """a manually buffered input pipe
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	354
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	355 Python will not let us use buffered IO and lazy reading with 'polling' at
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	356 the same time. We cannot probe the buffer state and select will not detect
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	357 that data are ready to read if they are already buffered.
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	358
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	359 This class let us work around that by implementing its own buffering
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	360 (allowing efficient readline) while offering a way to know if the buffer is
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	361 empty from the output (allowing collaboration of the buffer with polling).
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	362
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	363 This class lives in the 'util' module because it makes use of the 'os'
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	364 module from the python stdlib.
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	365 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	366
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	367 def __new__(cls, fh):
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	368 # If we receive a fileobjectproxy, we need to use a variation of this
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	369 # class that notifies observers about activity.
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	370 if isinstance(fh, fileobjectproxy):
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	371 cls = observedbufferedinputpipe
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	372
52668 5cc8deb96b48 pyupgrade: modernize calls to superclass methods Matt Harbison <matt_harbison@yahoo.com> parents: 52665 diff changeset	373 return super().__new__(cls)
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	374
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	375 def __init__(self, input):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	376 self._input = input
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	377 self._buffer = []
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	378 self._eof = False
25672 050dc6eabc92 bufferedinputpipe: remove N^2 computation of buffer length (issue4735) Pierre-Yves David <pierre-yves.david@fb.com> parents: 25671 diff changeset	379 self._lenbuf = 0
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	380
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	381 @property
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	382 def hasbuffer(self):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	383 """True is any data is currently buffered
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	384
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	385 This will be used externally a pre-step for polling IO. If there is
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	386 already data then no polling should be set in place."""
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	387 return bool(self._buffer)
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	388
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	389 @property
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	390 def closed(self):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	391 return self._input.closed
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	392
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	393 def fileno(self):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	394 return self._input.fileno()
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	395
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	396 def close(self):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	397 return self._input.close()
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	398
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	399 def read(self, size):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	400 while (not self._eof) and (self._lenbuf < size):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	401 self._fillbuffer()
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	402 return self._frombuffer(size)
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	403
38713 27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38576 diff changeset	404 def unbufferedread(self, size):
27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38576 diff changeset	405 if not self._eof and self._lenbuf == 0:
27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38576 diff changeset	406 self._fillbuffer(max(size, _chunksize))
27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38576 diff changeset	407 return self._frombuffer(min(self._lenbuf, size))
27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38576 diff changeset	408
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	409 def readline(self, args, *kwargs):
40030 e2697acd9381 cleanup: some Yoda conditions, this patch removes Martin von Zweigbergk <martinvonz@google.com> parents: 39910 diff changeset	410 if len(self._buffer) > 1:
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	411 # this should not happen because both read and readline end with a
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	412 # _frombuffer call that collapse it.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	413 self._buffer = [b''.join(self._buffer)]
25672 050dc6eabc92 bufferedinputpipe: remove N^2 computation of buffer length (issue4735) Pierre-Yves David <pierre-yves.david@fb.com> parents: 25671 diff changeset	414 self._lenbuf = len(self._buffer[0])
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	415 lfi = -1
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	416 if self._buffer:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	417 lfi = self._buffer[-1].find(b'\n')
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	418 while (not self._eof) and lfi < 0:
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	419 self._fillbuffer()
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	420 if self._buffer:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	421 lfi = self._buffer[-1].find(b'\n')
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	422 size = lfi + 1
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	423 if lfi < 0: # end of file
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	424 size = self._lenbuf
40030 e2697acd9381 cleanup: some Yoda conditions, this patch removes Martin von Zweigbergk <martinvonz@google.com> parents: 39910 diff changeset	425 elif len(self._buffer) > 1:
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	426 # we need to take previous chunks into account
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	427 size += self._lenbuf - len(self._buffer[-1])
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	428 return self._frombuffer(size)
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	429
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	430 def _frombuffer(self, size):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	431 """return at most 'size' data from the buffer
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	432
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	433 The data are removed from the buffer."""
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	434 if size == 0 or not self._buffer:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	435 return b''
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	436 buf = self._buffer[0]
40030 e2697acd9381 cleanup: some Yoda conditions, this patch removes Martin von Zweigbergk <martinvonz@google.com> parents: 39910 diff changeset	437 if len(self._buffer) > 1:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	438 buf = b''.join(self._buffer)
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	439
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	440 data = buf[:size]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	441 buf = buf[len(data) :]
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	442 if buf:
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	443 self._buffer = [buf]
25672 050dc6eabc92 bufferedinputpipe: remove N^2 computation of buffer length (issue4735) Pierre-Yves David <pierre-yves.david@fb.com> parents: 25671 diff changeset	444 self._lenbuf = len(buf)
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	445 else:
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	446 self._buffer = []
25672 050dc6eabc92 bufferedinputpipe: remove N^2 computation of buffer length (issue4735) Pierre-Yves David <pierre-yves.david@fb.com> parents: 25671 diff changeset	447 self._lenbuf = 0
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	448 return data
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	449
38713 27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38576 diff changeset	450 def _fillbuffer(self, size=_chunksize):
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	451 """read data to the buffer"""
38713 27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38576 diff changeset	452 data = os.read(self._input.fileno(), size)
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	453 if not data:
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	454 self._eof = True
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	455 else:
25672 050dc6eabc92 bufferedinputpipe: remove N^2 computation of buffer length (issue4735) Pierre-Yves David <pierre-yves.david@fb.com> parents: 25671 diff changeset	456 self._lenbuf += len(data)
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	457 self._buffer.append(data)
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	458
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	459 return data
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	460
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	461
51742 5f37c36f36b9 revlog: use mmap by default is pre-population is available Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51740 diff changeset	462 def has_mmap_populate():
51800 b619ba39d10a mmap: populate mapping in a background thread Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51789 diff changeset	463 return hasattr(osutil, "background_mmap_populate") or hasattr(
b619ba39d10a mmap: populate mapping in a background thread Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51789 diff changeset	464 mmap, 'MAP_POPULATE'
b619ba39d10a mmap: populate mapping in a background thread Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51789 diff changeset	465 )
51742 5f37c36f36b9 revlog: use mmap by default is pre-population is available Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51740 diff changeset	466
5f37c36f36b9 revlog: use mmap by default is pre-population is available Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51740 diff changeset	467
51740 522b4d729e89 mmap: populate the mapping by default Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51649 diff changeset	468 def mmapread(fp, size=None, pre_populate=True):
51649 ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51633 diff changeset	469 """Read a file content using mmap
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51633 diff changeset	470
51967 2d51b0cf707c util: minor copy editing of the documentation for `mmapread()` Matt Harbison <matt_harbison@yahoo.com> parents: 51966 diff changeset	471 The responsibility of checking the file system is mmap safe is the
2d51b0cf707c util: minor copy editing of the documentation for `mmapread()` Matt Harbison <matt_harbison@yahoo.com> parents: 51966 diff changeset	472 responsibility of the caller (see `vfs.is_mmap_safe`).
51649 ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51633 diff changeset	473
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51633 diff changeset	474 In some case, a normal string might be returned.
51740 522b4d729e89 mmap: populate the mapping by default Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51649 diff changeset	475
522b4d729e89 mmap: populate the mapping by default Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51649 diff changeset	476 If `pre_populate` is True (the default), the mmapped data will be
522b4d729e89 mmap: populate the mapping by default Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51649 diff changeset	477 pre-populated in memory if the system support this option, this slow down
51967 2d51b0cf707c util: minor copy editing of the documentation for `mmapread()` Matt Harbison <matt_harbison@yahoo.com> parents: 51966 diff changeset	478 the initial mmapping but avoid potentially crippling page fault on later
51740 522b4d729e89 mmap: populate the mapping by default Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51649 diff changeset	479 access. If this is not the desired behavior, set `pre_populate` to False.
51649 ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51633 diff changeset	480 """
44036 8ed8dfbeabb9 mmap: add a size argument to mmapread Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44023 diff changeset	481 if size == 0:
8ed8dfbeabb9 mmap: add a size argument to mmapread Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44023 diff changeset	482 # size of 0 to mmap.mmap() means "all data"
8ed8dfbeabb9 mmap: add a size argument to mmapread Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44023 diff changeset	483 # rather than "zero bytes", so special case that.
8ed8dfbeabb9 mmap: add a size argument to mmapread Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44023 diff changeset	484 return b''
8ed8dfbeabb9 mmap: add a size argument to mmapread Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44023 diff changeset	485 elif size is None:
8ed8dfbeabb9 mmap: add a size argument to mmapread Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44023 diff changeset	486 size = 0
48033 ae79611e3115 util: avoid a name-error warning in the `mmapread` exception handler Matt Harbison <matt_harbison@yahoo.com> parents: 47622 diff changeset	487 fd = getattr(fp, 'fileno', lambda: fp)()
51966 bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51926 diff changeset	488
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51926 diff changeset	489 if pycompat.iswindows:
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51926 diff changeset	490 _mmap = lambda fd, size: mmap.mmap(fd, size, access=mmap.ACCESS_READ)
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51926 diff changeset	491 else:
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51926 diff changeset	492 flags = mmap.MAP_PRIVATE
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51926 diff changeset	493 bg_populate = hasattr(osutil, "background_mmap_populate")
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51926 diff changeset	494
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51926 diff changeset	495 if pre_populate and not bg_populate:
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51926 diff changeset	496 flags \|= getattr(mmap, 'MAP_POPULATE', 0)
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51926 diff changeset	497
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51926 diff changeset	498 def _mmap(fd, size) -> mmap.mmap:
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51926 diff changeset	499 m = mmap.mmap(fd, size, flags=flags, prot=mmap.PROT_READ)
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51926 diff changeset	500 if pre_populate and bg_populate:
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51926 diff changeset	501 osutil.background_mmap_populate(m)
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51926 diff changeset	502 return m
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51926 diff changeset	503
34302 3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34154 diff changeset	504 try:
51966 bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51926 diff changeset	505 return _mmap(fd, size)
34302 3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34154 diff changeset	506 except ValueError:
3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34154 diff changeset	507 # Empty files cannot be mmapped, but mmapread should still work. Check
3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34154 diff changeset	508 # if the file is empty, and if so, return an empty buffer.
3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34154 diff changeset	509 if os.fstat(fd).st_size == 0:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	510 return b''
34302 3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34154 diff changeset	511 raise
3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34154 diff changeset	512
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	513
52116 82e2c99c84f3 cachestat: avoid creating cachestat for http path Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51974 diff changeset	514 class uncacheable_cachestat:
82e2c99c84f3 cachestat: avoid creating cachestat for http path Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51974 diff changeset	515 stat: Optional[os.stat_result]
82e2c99c84f3 cachestat: avoid creating cachestat for http path Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51974 diff changeset	516
82e2c99c84f3 cachestat: avoid creating cachestat for http path Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51974 diff changeset	517 def __init__(self) -> None:
82e2c99c84f3 cachestat: avoid creating cachestat for http path Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51974 diff changeset	518 self.stat = None
82e2c99c84f3 cachestat: avoid creating cachestat for http path Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51974 diff changeset	519
82e2c99c84f3 cachestat: avoid creating cachestat for http path Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51974 diff changeset	520 def cacheable(self) -> bool:
82e2c99c84f3 cachestat: avoid creating cachestat for http path Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51974 diff changeset	521 return False
82e2c99c84f3 cachestat: avoid creating cachestat for http path Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51974 diff changeset	522
82e2c99c84f3 cachestat: avoid creating cachestat for http path Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51974 diff changeset	523
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49031 diff changeset	524 class fileobjectproxy:
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	525 """A proxy around file objects that tells a watcher when events occur.
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	526
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	527 This type is intended to only be used for testing purposes. Think hard
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	528 before using it in important code.
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	529 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	530
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	531 __slots__ = (
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	532 '_orig',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	533 '_observer',
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	534 )
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	535
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	536 def __init__(self, fh, observer):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	537 object.__setattr__(self, '_orig', fh)
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	538 object.__setattr__(self, '_observer', observer)
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	539
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	540 def __getattribute__(self, name):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	541 ours = {
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	542 '_observer',
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	543 # IOBase
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	544 'close',
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	545 # closed if a property
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	546 'fileno',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	547 'flush',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	548 'isatty',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	549 'readable',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	550 'readline',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	551 'readlines',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	552 'seek',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	553 'seekable',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	554 'tell',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	555 'truncate',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	556 'writable',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	557 'writelines',
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	558 # RawIOBase
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	559 'read',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	560 'readall',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	561 'readinto',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	562 'write',
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	563 # BufferedIOBase
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	564 # raw is a property
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	565 'detach',
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	566 # read defined above
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	567 'read1',
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	568 # readinto defined above
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	569 # write defined above
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	570 }
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	571
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	572 # We only observe some methods.
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	573 if name in ours:
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	574 return object.__getattribute__(self, name)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	575
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	576 return getattr(object.__getattribute__(self, '_orig'), name)
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	577
36840 6bdea0efdab5 util: forward __bool__()/__nonzero__() on fileobjectproxy Matt Harbison <matt_harbison@yahoo.com> parents: 36801 diff changeset	578 def __nonzero__(self):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	579 return bool(object.__getattribute__(self, '_orig'))
36840 6bdea0efdab5 util: forward __bool__()/__nonzero__() on fileobjectproxy Matt Harbison <matt_harbison@yahoo.com> parents: 36801 diff changeset	580
6bdea0efdab5 util: forward __bool__()/__nonzero__() on fileobjectproxy Matt Harbison <matt_harbison@yahoo.com> parents: 36801 diff changeset	581 __bool__ = __nonzero__
6bdea0efdab5 util: forward __bool__()/__nonzero__() on fileobjectproxy Matt Harbison <matt_harbison@yahoo.com> parents: 36801 diff changeset	582
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	583 def __delattr__(self, name):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	584 return delattr(object.__getattribute__(self, '_orig'), name)
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	585
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	586 def __setattr__(self, name, value):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	587 return setattr(object.__getattribute__(self, '_orig'), name, value)
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	588
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	589 def __iter__(self):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	590 return object.__getattribute__(self, '_orig').__iter__()
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	591
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	592 def _observedcall(self, name, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	593 # Call the original object.
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	594 orig = object.__getattribute__(self, '_orig')
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	595 res = getattr(orig, name)(args, *kwargs)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	596
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	597 # Call a method on the observer of the same name with arguments
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	598 # so it can react, log, etc.
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	599 observer = object.__getattribute__(self, '_observer')
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	600 fn = getattr(observer, name, None)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	601 if fn:
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	602 fn(res, args, *kwargs)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	603
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	604 return res
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	605
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	606 def close(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	607 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	608 'close', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	609 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	610
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	611 def fileno(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	612 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	613 'fileno', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	614 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	615
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	616 def flush(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	617 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	618 'flush', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	619 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	620
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	621 def isatty(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	622 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	623 'isatty', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	624 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	625
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	626 def readable(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	627 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	628 'readable', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	629 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	630
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	631 def readline(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	632 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	633 'readline', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	634 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	635
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	636 def readlines(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	637 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	638 'readlines', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	639 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	640
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	641 def seek(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	642 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	643 'seek', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	644 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	645
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	646 def seekable(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	647 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	648 'seekable', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	649 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	650
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	651 def tell(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	652 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	653 'tell', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	654 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	655
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	656 def truncate(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	657 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	658 'truncate', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	659 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	660
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	661 def writable(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	662 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	663 'writable', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	664 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	665
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	666 def writelines(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	667 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	668 'writelines', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	669 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	670
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	671 def read(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	672 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	673 'read', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	674 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	675
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	676 def readall(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	677 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	678 'readall', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	679 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	680
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	681 def readinto(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	682 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	683 'readinto', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	684 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	685
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	686 def write(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	687 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	688 'write', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	689 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	690
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	691 def detach(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	692 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	693 'detach', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	694 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	695
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	696 def read1(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	697 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	698 'read1', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	699 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	700
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	701
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	702 class observedbufferedinputpipe(bufferedinputpipe):
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	703 """A variation of bufferedinputpipe that is aware of fileobjectproxy.
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	704
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	705 ``bufferedinputpipe`` makes low-level calls to ``os.read()`` that
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	706 bypass ``fileobjectproxy``. Because of this, we need to make
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	707 ``bufferedinputpipe`` aware of these operations.
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	708
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	709 This variation of ``bufferedinputpipe`` can notify observers about
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	710 ``os.read()`` events. It also re-publishes other events, such as
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	711 ``read()`` and ``readline()``.
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	712 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	713
49909 1d1b244a91b6 util: fix the signature of observedbufferedinputpipe._fillbuffer() Matt Harbison <matt_harbison@yahoo.com> parents: 49908 diff changeset	714 def _fillbuffer(self, size=_chunksize):
52668 5cc8deb96b48 pyupgrade: modernize calls to superclass methods Matt Harbison <matt_harbison@yahoo.com> parents: 52665 diff changeset	715 res = super()._fillbuffer(size=size)
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	716
43103 c95b2f40db7c py3: stop normalizing 2nd argument of attr() to unicode Gregory Szorc <gregory.szorc@gmail.com>* parents: 43090 diff changeset	717 fn = getattr(self._input._observer, 'osread', None)
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	718 if fn:
49909 1d1b244a91b6 util: fix the signature of observedbufferedinputpipe._fillbuffer() Matt Harbison <matt_harbison@yahoo.com> parents: 49908 diff changeset	719 fn(res, size)
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	720
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	721 return res
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	722
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	723 # We use different observer methods because the operation isn't
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	724 # performed on the actual file object but on us.
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	725 def read(self, size):
52668 5cc8deb96b48 pyupgrade: modernize calls to superclass methods Matt Harbison <matt_harbison@yahoo.com> parents: 52665 diff changeset	726 res = super().read(size)
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	727
43103 c95b2f40db7c py3: stop normalizing 2nd argument of attr() to unicode Gregory Szorc <gregory.szorc@gmail.com>* parents: 43090 diff changeset	728 fn = getattr(self._input._observer, 'bufferedread', None)
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	729 if fn:
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	730 fn(res, size)
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	731
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	732 return res
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	733
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	734 def readline(self, args, *kwargs):
52668 5cc8deb96b48 pyupgrade: modernize calls to superclass methods Matt Harbison <matt_harbison@yahoo.com> parents: 52665 diff changeset	735 res = super().readline(args, *kwargs)
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	736
43103 c95b2f40db7c py3: stop normalizing 2nd argument of attr() to unicode Gregory Szorc <gregory.szorc@gmail.com>* parents: 43090 diff changeset	737 fn = getattr(self._input._observer, 'bufferedreadline', None)
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	738 if fn:
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	739 fn(res)
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	740
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	741 return res
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	742
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	743
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	744 PROXIED_SOCKET_METHODS = {
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	745 'makefile',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	746 'recv',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	747 'recvfrom',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	748 'recvfrom_into',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	749 'recv_into',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	750 'send',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	751 'sendall',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	752 'sendto',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	753 'setblocking',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	754 'settimeout',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	755 'gettimeout',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	756 'setsockopt',
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	757 }
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	758
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	759
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49031 diff changeset	760 class socketproxy:
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	761 """A proxy around a socket that tells a watcher when events occur.
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	762
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	763 This is like ``fileobjectproxy`` except for sockets.
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	764
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	765 This type is intended to only be used for testing purposes. Think hard
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	766 before using it in important code.
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	767 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	768
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	769 __slots__ = (
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	770 '_orig',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	771 '_observer',
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	772 )
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	773
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	774 def __init__(self, sock, observer):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	775 object.__setattr__(self, '_orig', sock)
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	776 object.__setattr__(self, '_observer', observer)
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	777
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	778 def __getattribute__(self, name):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	779 if name in PROXIED_SOCKET_METHODS:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	780 return object.__getattribute__(self, name)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	781
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	782 return getattr(object.__getattribute__(self, '_orig'), name)
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	783
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	784 def __delattr__(self, name):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	785 return delattr(object.__getattribute__(self, '_orig'), name)
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	786
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	787 def __setattr__(self, name, value):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	788 return setattr(object.__getattribute__(self, '_orig'), name, value)
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	789
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	790 def __nonzero__(self):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	791 return bool(object.__getattribute__(self, '_orig'))
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	792
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	793 __bool__ = __nonzero__
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	794
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	795 def _observedcall(self, name, args, *kwargs):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	796 # Call the original object.
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	797 orig = object.__getattribute__(self, '_orig')
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	798 res = getattr(orig, name)(args, *kwargs)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	799
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	800 # Call a method on the observer of the same name with arguments
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	801 # so it can react, log, etc.
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	802 observer = object.__getattribute__(self, '_observer')
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	803 fn = getattr(observer, name, None)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	804 if fn:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	805 fn(res, args, *kwargs)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	806
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	807 return res
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	808
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	809 def makefile(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	810 res = object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	811 'makefile', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	812 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	813
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	814 # The file object may be used for I/O. So we turn it into a
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	815 # proxy using our observer.
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	816 observer = object.__getattribute__(self, '_observer')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	817 return makeloggingfileobject(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	818 observer.fh,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	819 res,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	820 observer.name,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	821 reads=observer.reads,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	822 writes=observer.writes,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	823 logdata=observer.logdata,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	824 logdataapis=observer.logdataapis,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	825 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	826
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	827 def recv(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	828 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	829 'recv', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	830 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	831
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	832 def recvfrom(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	833 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	834 'recvfrom', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	835 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	836
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	837 def recvfrom_into(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	838 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	839 'recvfrom_into', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	840 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	841
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	842 def recv_into(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	843 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	844 'recv_info', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	845 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	846
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	847 def send(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	848 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	849 'send', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	850 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	851
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	852 def sendall(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	853 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	854 'sendall', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	855 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	856
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	857 def sendto(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	858 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	859 'sendto', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	860 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	861
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	862 def setblocking(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	863 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	864 'setblocking', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	865 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	866
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	867 def settimeout(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	868 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	869 'settimeout', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	870 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	871
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	872 def gettimeout(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	873 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	874 'gettimeout', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	875 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	876
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	877 def setsockopt(self, args, *kwargs):
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	878 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	879 'setsockopt', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	880 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	881
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	882
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49031 diff changeset	883 class baseproxyobserver:
43926 4222b9d5d4fb util: move common proxyobserver attributes to the base class Matt Harbison <matt_harbison@yahoo.com> parents: 43913 diff changeset	884 def __init__(self, fh, name, logdata, logdataapis):
4222b9d5d4fb util: move common proxyobserver attributes to the base class Matt Harbison <matt_harbison@yahoo.com> parents: 43913 diff changeset	885 self.fh = fh
4222b9d5d4fb util: move common proxyobserver attributes to the base class Matt Harbison <matt_harbison@yahoo.com> parents: 43913 diff changeset	886 self.name = name
4222b9d5d4fb util: move common proxyobserver attributes to the base class Matt Harbison <matt_harbison@yahoo.com> parents: 43913 diff changeset	887 self.logdata = logdata
4222b9d5d4fb util: move common proxyobserver attributes to the base class Matt Harbison <matt_harbison@yahoo.com> parents: 43913 diff changeset	888 self.logdataapis = logdataapis
4222b9d5d4fb util: move common proxyobserver attributes to the base class Matt Harbison <matt_harbison@yahoo.com> parents: 43913 diff changeset	889
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	890 def _writedata(self, data):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	891 if not self.logdata:
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	892 if self.logdataapis:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	893 self.fh.write(b'\n')
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	894 self.fh.flush()
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	895 return
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	896
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	897 # Simple case writes all data on a single line.
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	898 if b'\n' not in data:
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	899 if self.logdataapis:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	900 self.fh.write(b': %s\n' % stringutil.escapestr(data))
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	901 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	902 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	903 b'%s> %s\n' % (self.name, stringutil.escapestr(data))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	904 )
36994 d683c7367989 wireproto: explicitly flush stdio to prevent stalls on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 36964 diff changeset	905 self.fh.flush()
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	906 return
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	907
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	908 # Data with newlines is written to multiple lines.
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	909 if self.logdataapis:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	910 self.fh.write(b':\n')
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	911
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	912 lines = data.splitlines(True)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	913 for line in lines:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	914 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	915 b'%s> %s\n' % (self.name, stringutil.escapestr(line))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	916 )
36994 d683c7367989 wireproto: explicitly flush stdio to prevent stalls on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 36964 diff changeset	917 self.fh.flush()
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	918
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	919
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	920 class fileobjectobserver(baseproxyobserver):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	921 """Logs file object activity."""
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	922
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	923 def __init__(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	924 self, fh, name, reads=True, writes=True, logdata=False, logdataapis=True
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	925 ):
52668 5cc8deb96b48 pyupgrade: modernize calls to superclass methods Matt Harbison <matt_harbison@yahoo.com> parents: 52665 diff changeset	926 super().__init__(fh, name, logdata, logdataapis)
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	927 self.reads = reads
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	928 self.writes = writes
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	929
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	930 def read(self, res, size=-1):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	931 if not self.reads:
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	932 return
36614 26a6b62919e2 util: work around Python 3 returning None at EOF instead of '' Augie Fackler <augie@google.com> parents: 36613 diff changeset	933 # Python 3 can return None from reads at EOF instead of empty strings.
26a6b62919e2 util: work around Python 3 returning None at EOF instead of '' Augie Fackler <augie@google.com> parents: 36613 diff changeset	934 if res is None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	935 res = b''
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	936
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	937 if size == -1 and res == b'':
38323 565074cc9ac6 tests: suppress read(-1) -> '' calls in fileobjectobserver Augie Fackler <augie@google.com> parents: 38197 diff changeset	938 # Suppress pointless read(-1) calls that return
565074cc9ac6 tests: suppress read(-1) -> '' calls in fileobjectobserver Augie Fackler <augie@google.com> parents: 38197 diff changeset	939 # nothing. These happen _a lot_ on Python 3, and there
565074cc9ac6 tests: suppress read(-1) -> '' calls in fileobjectobserver Augie Fackler <augie@google.com> parents: 38197 diff changeset	940 # doesn't seem to be a better workaround to have matching
565074cc9ac6 tests: suppress read(-1) -> '' calls in fileobjectobserver Augie Fackler <augie@google.com> parents: 38197 diff changeset	941 # Python 2 and 3 behavior. :(
565074cc9ac6 tests: suppress read(-1) -> '' calls in fileobjectobserver Augie Fackler <augie@google.com> parents: 38197 diff changeset	942 return
565074cc9ac6 tests: suppress read(-1) -> '' calls in fileobjectobserver Augie Fackler <augie@google.com> parents: 38197 diff changeset	943
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	944 if self.logdataapis:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	945 self.fh.write(b'%s> read(%d) -> %d' % (self.name, size, len(res)))
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	946
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	947 self._writedata(res)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	948
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	949 def readline(self, res, limit=-1):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	950 if not self.reads:
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	951 return
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	952
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	953 if self.logdataapis:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	954 self.fh.write(b'%s> readline() -> %d' % (self.name, len(res)))
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	955
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	956 self._writedata(res)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	957
36659 29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36658 diff changeset	958 def readinto(self, res, dest):
29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36658 diff changeset	959 if not self.reads:
29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36658 diff changeset	960 return
29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36658 diff changeset	961
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	962 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	963 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	964 b'%s> readinto(%d) -> %r' % (self.name, len(dest), res)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	965 )
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	966
36659 29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36658 diff changeset	967 data = dest[0:res] if res is not None else b''
41406 b141b5243b37 util: cast memoryview to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 41289 diff changeset	968
b141b5243b37 util: cast memoryview to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 41289 diff changeset	969 # _writedata() uses "in" operator and is confused by memoryview because
b141b5243b37 util: cast memoryview to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 41289 diff changeset	970 # characters are ints on Python 3.
b141b5243b37 util: cast memoryview to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 41289 diff changeset	971 if isinstance(data, memoryview):
b141b5243b37 util: cast memoryview to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 41289 diff changeset	972 data = data.tobytes()
b141b5243b37 util: cast memoryview to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 41289 diff changeset	973
36659 29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36658 diff changeset	974 self._writedata(data)
29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36658 diff changeset	975
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	976 def write(self, res, data):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	977 if not self.writes:
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	978 return
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	979
36660 8395fddde46c util: report integer result from write() Gregory Szorc <gregory.szorc@gmail.com> parents: 36659 diff changeset	980 # Python 2 returns None from some write() calls. Python 3 (reasonably)
8395fddde46c util: report integer result from write() Gregory Szorc <gregory.szorc@gmail.com> parents: 36659 diff changeset	981 # returns the integer bytes written.
8395fddde46c util: report integer result from write() Gregory Szorc <gregory.szorc@gmail.com> parents: 36659 diff changeset	982 if res is None and data:
8395fddde46c util: report integer result from write() Gregory Szorc <gregory.szorc@gmail.com> parents: 36659 diff changeset	983 res = len(data)
8395fddde46c util: report integer result from write() Gregory Szorc <gregory.szorc@gmail.com> parents: 36659 diff changeset	984
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	985 if self.logdataapis:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	986 self.fh.write(b'%s> write(%d) -> %r' % (self.name, len(data), res))
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	987
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	988 self._writedata(data)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	989
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	990 def flush(self, res):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	991 if not self.writes:
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	992 return
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	993
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	994 self.fh.write(b'%s> flush() -> %r\n' % (self.name, res))
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	995
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	996 # For observedbufferedinputpipe.
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	997 def bufferedread(self, res, size):
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	998 if not self.reads:
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	999 return
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1000
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1001 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1002 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1003 b'%s> bufferedread(%d) -> %d' % (self.name, size, len(res))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1004 )
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1005
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	1006 self._writedata(res)
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	1007
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	1008 def bufferedreadline(self, res):
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1009 if not self.reads:
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1010 return
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1011
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1012 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1013 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1014 b'%s> bufferedreadline() -> %d' % (self.name, len(res))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1015 )
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1016
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	1017 self._writedata(res)
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	1018
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1019
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1020 def makeloggingfileobject(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1021 logh, fh, name, reads=True, writes=True, logdata=False, logdataapis=True
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1022 ):
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	1023 """Turn a file object into a logging file object."""
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	1024
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1025 observer = fileobjectobserver(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1026 logh,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1027 name,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1028 reads=reads,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1029 writes=writes,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1030 logdata=logdata,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1031 logdataapis=logdataapis,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1032 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	1033 return fileobjectproxy(fh, observer)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	1034
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1035
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1036 class socketobserver(baseproxyobserver):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1037 """Logs socket activity."""
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1038
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1039 def __init__(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1040 self,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1041 fh,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1042 name,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1043 reads=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1044 writes=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1045 states=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1046 logdata=False,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1047 logdataapis=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1048 ):
52668 5cc8deb96b48 pyupgrade: modernize calls to superclass methods Matt Harbison <matt_harbison@yahoo.com> parents: 52665 diff changeset	1049 super().__init__(fh, name, logdata, logdataapis)
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1050 self.reads = reads
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1051 self.writes = writes
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1052 self.states = states
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1053
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1054 def makefile(self, res, mode=None, bufsize=None):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1055 if not self.states:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1056 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1057
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1058 self.fh.write(b'%s> makefile(%r, %r)\n' % (self.name, mode, bufsize))
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1059
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1060 def recv(self, res, size, flags=0):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1061 if not self.reads:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1062 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1063
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1064 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1065 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1066 b'%s> recv(%d, %d) -> %d' % (self.name, size, flags, len(res))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1067 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1068 self._writedata(res)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1069
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1070 def recvfrom(self, res, size, flags=0):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1071 if not self.reads:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1072 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1073
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1074 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1075 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1076 b'%s> recvfrom(%d, %d) -> %d'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1077 % (self.name, size, flags, len(res[0]))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1078 )
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1079
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1080 self._writedata(res[0])
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1081
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1082 def recvfrom_into(self, res, buf, size, flags=0):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1083 if not self.reads:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1084 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1085
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1086 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1087 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1088 b'%s> recvfrom_into(%d, %d) -> %d'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1089 % (self.name, size, flags, res[0])
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1090 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1091
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1092 self._writedata(buf[0 : res[0]])
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1093
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1094 def recv_into(self, res, buf, size=0, flags=0):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1095 if not self.reads:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1096 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1097
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1098 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1099 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1100 b'%s> recv_into(%d, %d) -> %d' % (self.name, size, flags, res)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1101 )
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1102
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1103 self._writedata(buf[0:res])
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1104
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1105 def send(self, res, data, flags=0):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1106 if not self.writes:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1107 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1108
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1109 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1110 b'%s> send(%d, %d) -> %d' % (self.name, len(data), flags, len(res))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1111 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1112 self._writedata(data)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1113
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1114 def sendall(self, res, data, flags=0):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1115 if not self.writes:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1116 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1117
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1118 if self.logdataapis:
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1119 # Returns None on success. So don't bother reporting return value.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1120 self.fh.write(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1121 b'%s> sendall(%d, %d)' % (self.name, len(data), flags)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1122 )
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1123
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1124 self._writedata(data)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1125
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1126 def sendto(self, res, data, flagsoraddress, address=None):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1127 if not self.writes:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1128 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1129
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1130 if address:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1131 flags = flagsoraddress
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1132 else:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1133 flags = 0
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1134
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1135 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1136 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1137 b'%s> sendto(%d, %d, %r) -> %d'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1138 % (self.name, len(data), flags, address, res)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1139 )
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1140
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1141 self._writedata(data)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1142
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1143 def setblocking(self, res, flag):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1144 if not self.states:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1145 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1146
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1147 self.fh.write(b'%s> setblocking(%r)\n' % (self.name, flag))
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1148
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1149 def settimeout(self, res, value):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1150 if not self.states:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1151 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1152
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1153 self.fh.write(b'%s> settimeout(%r)\n' % (self.name, value))
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1154
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1155 def gettimeout(self, res):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1156 if not self.states:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1157 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1158
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1159 self.fh.write(b'%s> gettimeout() -> %f\n' % (self.name, res))
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1160
39097 ed8160e4fea0 util: fix signature of setsockopt in socket observer Augie Fackler <augie@google.com> parents: 38852 diff changeset	1161 def setsockopt(self, res, level, optname, value):
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1162 if not self.states:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1163 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1164
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1165 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1166 b'%s> setsockopt(%r, %r, %r) -> %r\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1167 % (self.name, level, optname, value, res)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1168 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1169
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1170
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1171 def makeloggingsocket(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1172 logh,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1173 fh,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1174 name,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1175 reads=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1176 writes=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1177 states=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1178 logdata=False,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1179 logdataapis=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1180 ):
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1181 """Turn a socket into a logging socket."""
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1182
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1183 observer = socketobserver(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1184 logh,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1185 name,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1186 reads=reads,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1187 writes=writes,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1188 states=states,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1189 logdata=logdata,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1190 logdataapis=logdataapis,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1191 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1192 return socketproxy(fh, observer)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1193
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1194
7632 9626819b2e3d refactor version code Matt Mackall <mpm@selenic.com> parents: 7559 diff changeset	1195 def version():
9626819b2e3d refactor version code Matt Mackall <mpm@selenic.com> parents: 7559 diff changeset	1196 """Return version information if available."""
9626819b2e3d refactor version code Matt Mackall <mpm@selenic.com> parents: 7559 diff changeset	1197 try:
51789 92845af308b4 typing: narrow the scope of some recent disabled import warnings Matt Harbison <matt_harbison@yahoo.com> parents: 51788 diff changeset	1198 from . import __version__ # pytype: disable=import-error
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1199
7632 9626819b2e3d refactor version code Matt Mackall <mpm@selenic.com> parents: 7559 diff changeset	1200 return __version__.version
9626819b2e3d refactor version code Matt Mackall <mpm@selenic.com> parents: 7559 diff changeset	1201 except ImportError:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1202 return b'unknown'
7632 9626819b2e3d refactor version code Matt Mackall <mpm@selenic.com> parents: 7559 diff changeset	1203
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1204
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1205 def versiontuple(v=None, n=4):
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1206 """Parses a Mercurial version string into an N-tuple.
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1207
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1208 The version string to be parsed is specified with the ``v`` argument.
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1209 If it isn't defined, the current Mercurial version string will be parsed.
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1210
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1211 ``n`` can be 2, 3, or 4. Here is how some version strings map to
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1212 returned values:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1213
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	1214 >>> v = b'3.6.1+190-df9b73d2d444'
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1215 >>> versiontuple(v, 2)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1216 (3, 6)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1217 >>> versiontuple(v, 3)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1218 (3, 6, 1)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1219 >>> versiontuple(v, 4)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1220 (3, 6, 1, '190-df9b73d2d444')
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1221
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	1222 >>> versiontuple(b'3.6.1+190-df9b73d2d444+20151118')
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1223 (3, 6, 1, '190-df9b73d2d444+20151118')
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1224
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	1225 >>> v = b'3.6'
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1226 >>> versiontuple(v, 2)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1227 (3, 6)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1228 >>> versiontuple(v, 3)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1229 (3, 6, None)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1230 >>> versiontuple(v, 4)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1231 (3, 6, None, None)
29613 616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1232
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	1233 >>> v = b'3.9-rc'
29613 616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1234 >>> versiontuple(v, 2)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1235 (3, 9)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1236 >>> versiontuple(v, 3)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1237 (3, 9, None)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1238 >>> versiontuple(v, 4)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1239 (3, 9, None, 'rc')
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1240
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	1241 >>> v = b'3.9-rc+2-02a8fea4289b'
29613 616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1242 >>> versiontuple(v, 2)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1243 (3, 9)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1244 >>> versiontuple(v, 3)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1245 (3, 9, None)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1246 >>> versiontuple(v, 4)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1247 (3, 9, None, 'rc+2-02a8fea4289b')
37801 5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1248
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1249 >>> versiontuple(b'4.6rc0')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1250 (4, 6, None, 'rc0')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1251 >>> versiontuple(b'4.6rc0+12-425d55e54f98')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1252 (4, 6, None, 'rc0+12-425d55e54f98')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1253 >>> versiontuple(b'.1.2.3')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1254 (None, None, None, '.1.2.3')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1255 >>> versiontuple(b'12.34..5')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1256 (12, 34, None, '..5')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1257 >>> versiontuple(b'1.2.3.4.5.6')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1258 (1, 2, 3, '.4.5.6')
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1259 """
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1260 if not v:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1261 v = version()
44023 6d3b67a837a6 cleanup: drop redundant character escapes from `[]` character sets Matt Harbison <matt_harbison@yahoo.com> parents: 44020 diff changeset	1262 m = remod.match(br'(\d+(?:\.\d+){,2})[+-]?(.*)', v)
37801 5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1263 if not m:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1264 vparts, extra = b'', v
37801 5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1265 elif m.group(2):
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1266 vparts, extra = m.groups()
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1267 else:
37801 5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1268 vparts, extra = m.group(1), None
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1269
43888 a21a6dad4b38 typing: add an assertion to util.versiontuple Matt Harbison <matt_harbison@yahoo.com> parents: 43887 diff changeset	1270 assert vparts is not None # help pytype
a21a6dad4b38 typing: add an assertion to util.versiontuple Matt Harbison <matt_harbison@yahoo.com> parents: 43887 diff changeset	1271
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1272 vints = []
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1273 for i in vparts.split(b'.'):
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1274 try:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1275 vints.append(int(i))
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1276 except ValueError:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1277 break
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1278 # (3, 6) -> (3, 6, None)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1279 while len(vints) < 3:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1280 vints.append(None)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1281
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1282 if n == 2:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1283 return (vints[0], vints[1])
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1284 if n == 3:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1285 return (vints[0], vints[1], vints[2])
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1286 if n == 4:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1287 return (vints[0], vints[1], vints[2], extra)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1288
48036 376d08ae904f util: eliminate the possibility of returning None from `versiontuple()` Matt Harbison <matt_harbison@yahoo.com> parents: 48033 diff changeset	1289 raise error.ProgrammingError(b"invalid version part request: %d" % n)
376d08ae904f util: eliminate the possibility of returning None from `versiontuple()` Matt Harbison <matt_harbison@yahoo.com> parents: 48033 diff changeset	1290
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1291
3145 e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1292 def cachefunc(func):
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1293 '''cache the result of function calls'''
3147 97420a49188d add comments in cachefunc Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 3145 diff changeset	1294 # XXX doesn't handle keywords args
28832 f5ff10f6fa6b util: use __code__ (available since py2.6) timeless <timeless@mozdev.org> parents: 28826 diff changeset	1295 if func.__code__.co_argcount == 0:
43886 eff050dbb703 util: rename a variable to avoid confusing pytype Matt Harbison <matt_harbison@yahoo.com> parents: 43885 diff changeset	1296 listcache = []
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1297
20835 0e8417131a29 util: add the code path to "cachefunc()" for the function taking no arguments FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20679 diff changeset	1298 def f():
43886 eff050dbb703 util: rename a variable to avoid confusing pytype Matt Harbison <matt_harbison@yahoo.com> parents: 43885 diff changeset	1299 if len(listcache) == 0:
eff050dbb703 util: rename a variable to avoid confusing pytype Matt Harbison <matt_harbison@yahoo.com> parents: 43885 diff changeset	1300 listcache.append(func())
eff050dbb703 util: rename a variable to avoid confusing pytype Matt Harbison <matt_harbison@yahoo.com> parents: 43885 diff changeset	1301 return listcache[0]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1302
20835 0e8417131a29 util: add the code path to "cachefunc()" for the function taking no arguments FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20679 diff changeset	1303 return f
3145 e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1304 cache = {}
28832 f5ff10f6fa6b util: use __code__ (available since py2.6) timeless <timeless@mozdev.org> parents: 28826 diff changeset	1305 if func.__code__.co_argcount == 1:
3147 97420a49188d add comments in cachefunc Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 3145 diff changeset	1306 # we gain a small amount of time because
97420a49188d add comments in cachefunc Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 3145 diff changeset	1307 # we don't need to pack/unpack the list
3145 e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1308 def f(arg):
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1309 if arg not in cache:
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1310 cache[arg] = func(arg)
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1311 return cache[arg]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1312
3145 e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1313 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1314
3145 e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1315 def f(*args):
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1316 if args not in cache:
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1317 cache[args] = func(*args)
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1318 return cache[args]
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1319
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1320 return f
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1321
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1322
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49031 diff changeset	1323 class cow:
34358 c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1324 """helper class to make copy-on-write easier
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1325
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1326 Call preparewrite before doing any writes.
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1327 """
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1328
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1329 def preparewrite(self):
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1330 """call this before writes, return self or a copied new object"""
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1331 if getattr(self, '_copied', 0):
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1332 self._copied -= 1
46901 51841b23670b typing: make minor adjustments to mercurial/util.py to pass pytype checking Matt Harbison <matt_harbison@yahoo.com> parents: 46900 diff changeset	1333 # Function cow.__init__ expects 1 arg(s), got 2 [wrong-arg-count]
51841b23670b typing: make minor adjustments to mercurial/util.py to pass pytype checking Matt Harbison <matt_harbison@yahoo.com> parents: 46900 diff changeset	1334 return self.__class__(self) # pytype: disable=wrong-arg-count
34358 c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1335 return self
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1336
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1337 def copy(self):
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1338 """always do a cheap copy"""
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1339 self._copied = getattr(self, '_copied', 0) + 1
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1340 return self
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1341
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1342
32340 bd0fd3ff9916 util: rewrite sortdict using Python 2.7's OrderedDict Martin von Zweigbergk <martinvonz@google.com> parents: 32331 diff changeset	1343 class sortdict(collections.OrderedDict):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	1344 """a simple sorted dictionary
32346 2d19664e257d util: drop unneeded override, sortdict.copy() Yuya Nishihara <yuya@tcha.org> parents: 32340 diff changeset	1345
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	1346 >>> d1 = sortdict([(b'a', 0), (b'b', 1)])
32346 2d19664e257d util: drop unneeded override, sortdict.copy() Yuya Nishihara <yuya@tcha.org> parents: 32340 diff changeset	1347 >>> d2 = d1.copy()
50749 a2df74853f8d tests: fix sortdict doctest with Python 3.12 Mads Kiilerich <mads@kiilerich.com> parents: 50398 diff changeset	1348 >>> list(d2.items())
a2df74853f8d tests: fix sortdict doctest with Python 3.12 Mads Kiilerich <mads@kiilerich.com> parents: 50398 diff changeset	1349 [('a', 0), ('b', 1)]
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	1350 >>> d2.update([(b'a', 2)])
34154 be00af4a1ac5 doctest: coerce dict.keys() to list Yuya Nishihara <yuya@tcha.org> parents: 34152 diff changeset	1351 >>> list(d2.keys()) # should still be in last-set order
32346 2d19664e257d util: drop unneeded override, sortdict.copy() Yuya Nishihara <yuya@tcha.org> parents: 32340 diff changeset	1352 ['b', 'a']
43913 68af0228fedc util: implement sortdict.insert() Martin von Zweigbergk <martinvonz@google.com> parents: 43889 diff changeset	1353 >>> d1.insert(1, b'a.5', 0.5)
50749 a2df74853f8d tests: fix sortdict doctest with Python 3.12 Mads Kiilerich <mads@kiilerich.com> parents: 50398 diff changeset	1354 >>> list(d1.items())
a2df74853f8d tests: fix sortdict doctest with Python 3.12 Mads Kiilerich <mads@kiilerich.com> parents: 50398 diff changeset	1355 [('a', 0), ('a.5', 0.5), ('b', 1)]
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	1356 """
32346 2d19664e257d util: drop unneeded override, sortdict.copy() Yuya Nishihara <yuya@tcha.org> parents: 32340 diff changeset	1357
32340 bd0fd3ff9916 util: rewrite sortdict using Python 2.7's OrderedDict Martin von Zweigbergk <martinvonz@google.com> parents: 32331 diff changeset	1358 def __setitem__(self, key, value):
21813 c2262004c2e2 config: move config.sortdict class into util Angel Ezquerra <angel.ezquerra@gmail.com> parents: 21046 diff changeset	1359 if key in self:
32340 bd0fd3ff9916 util: rewrite sortdict using Python 2.7's OrderedDict Martin von Zweigbergk <martinvonz@google.com> parents: 32331 diff changeset	1360 del self[key]
52668 5cc8deb96b48 pyupgrade: modernize calls to superclass methods Matt Harbison <matt_harbison@yahoo.com> parents: 52665 diff changeset	1361 super().__setitem__(key, value)
21813 c2262004c2e2 config: move config.sortdict class into util Angel Ezquerra <angel.ezquerra@gmail.com> parents: 21046 diff changeset	1362
33624 524b13fc711f util: fix sortdict.update() to call __setitem__() on PyPy (issue5639) Yuya Nishihara <yuya@tcha.org> parents: 33549 diff changeset	1363 if pycompat.ispypy:
524b13fc711f util: fix sortdict.update() to call __setitem__() on PyPy (issue5639) Yuya Nishihara <yuya@tcha.org> parents: 33549 diff changeset	1364 # __setitem__() isn't called as of PyPy 5.8.0
46900 64400d05db1e util: fix the signature for the pypy override of sortdict.update() Matt Harbison <matt_harbison@yahoo.com> parents: 46827 diff changeset	1365 def update(self, src, **f):
33624 524b13fc711f util: fix sortdict.update() to call __setitem__() on PyPy (issue5639) Yuya Nishihara <yuya@tcha.org> parents: 33549 diff changeset	1366 if isinstance(src, dict):
49004 f254fc73d956 global: bulk replace simple pycompat.iteritems(x) with x.items() Gregory Szorc <gregory.szorc@gmail.com> parents: 48988 diff changeset	1367 src = src.items()
33624 524b13fc711f util: fix sortdict.update() to call __setitem__() on PyPy (issue5639) Yuya Nishihara <yuya@tcha.org> parents: 33549 diff changeset	1368 for k, v in src:
524b13fc711f util: fix sortdict.update() to call __setitem__() on PyPy (issue5639) Yuya Nishihara <yuya@tcha.org> parents: 33549 diff changeset	1369 self[k] = v
46900 64400d05db1e util: fix the signature for the pypy override of sortdict.update() Matt Harbison <matt_harbison@yahoo.com> parents: 46827 diff changeset	1370 for k in f:
64400d05db1e util: fix the signature for the pypy override of sortdict.update() Matt Harbison <matt_harbison@yahoo.com> parents: 46827 diff changeset	1371 self[k] = f[k]
33624 524b13fc711f util: fix sortdict.update() to call __setitem__() on PyPy (issue5639) Yuya Nishihara <yuya@tcha.org> parents: 33549 diff changeset	1372
43913 68af0228fedc util: implement sortdict.insert() Martin von Zweigbergk <martinvonz@google.com> parents: 43889 diff changeset	1373 def insert(self, position, key, value):
51762 ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51761 diff changeset	1374 for i, (k, v) in enumerate(list(self.items())):
43913 68af0228fedc util: implement sortdict.insert() Martin von Zweigbergk <martinvonz@google.com> parents: 43889 diff changeset	1375 if i == position:
68af0228fedc util: implement sortdict.insert() Martin von Zweigbergk <martinvonz@google.com> parents: 43889 diff changeset	1376 self[key] = value
68af0228fedc util: implement sortdict.insert() Martin von Zweigbergk <martinvonz@google.com> parents: 43889 diff changeset	1377 if i >= position:
68af0228fedc util: implement sortdict.insert() Martin von Zweigbergk <martinvonz@google.com> parents: 43889 diff changeset	1378 del self[k]
68af0228fedc util: implement sortdict.insert() Martin von Zweigbergk <martinvonz@google.com> parents: 43889 diff changeset	1379 self[k] = v
68af0228fedc util: implement sortdict.insert() Martin von Zweigbergk <martinvonz@google.com> parents: 43889 diff changeset	1380
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1381
34358 c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1382 class cowdict(cow, dict):
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1383 """copy-on-write dict
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1384
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1385 Be sure to call d = d.preparewrite() before writing to d.
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1386
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1387 >>> a = cowdict()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1388 >>> a is a.preparewrite()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1389 True
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1390 >>> b = a.copy()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1391 >>> b is a
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1392 True
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1393 >>> c = b.copy()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1394 >>> c is a
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1395 True
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1396 >>> a = a.preparewrite()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1397 >>> b is a
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1398 False
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1399 >>> a is a.preparewrite()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1400 True
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1401 >>> c = c.preparewrite()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1402 >>> b is c
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1403 False
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1404 >>> b is b.preparewrite()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1405 True
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1406 """
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1407
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1408
34358 c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1409 class cowsortdict(cow, sortdict):
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1410 """copy-on-write sortdict
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1411
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1412 Be sure to call d = d.preparewrite() before writing to d.
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1413 """
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1414
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1415
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49031 diff changeset	1416 class transactional: # pytype: disable=ignored-metaclass
33814 bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1417 """Base class for making a transactional type into a context manager."""
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1418
33814 bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1419 __metaclass__ = abc.ABCMeta
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1420
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1421 @abc.abstractmethod
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1422 def close(self):
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1423 """Successfully closes the transaction."""
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1424
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1425 @abc.abstractmethod
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1426 def release(self):
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1427 """Marks the end of the transaction.
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1428
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1429 If the transaction has not been closed, it will be aborted.
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1430 """
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1431
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1432 def __enter__(self):
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1433 return self
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1434
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1435 def __exit__(self, exc_type, exc_val, exc_tb):
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1436 try:
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1437 if exc_type is None:
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1438 self.close()
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1439 finally:
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1440 self.release()
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1441
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1442
33446 fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1443 @contextlib.contextmanager
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1444 def acceptintervention(tr=None):
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1445 """A context manager that closes the transaction on InterventionRequired
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1446
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1447 If no transaction was provided, this simply runs the body and returns
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1448 """
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1449 if not tr:
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1450 yield
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1451 return
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1452 try:
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1453 yield
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1454 tr.close()
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1455 except error.InterventionRequired:
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1456 tr.close()
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1457 raise
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1458 finally:
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1459 tr.release()
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1460
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1461
33691 609606d21765 rebase: use one dirstateguard for when using rebase.singletransaction Durham Goode <durham@fb.com> parents: 33549 diff changeset	1462 @contextlib.contextmanager
45146 a0791bfd9cfa util: enhance `nullcontextmanager` to be able to return __enter__ result Manuel Jacob <me@manueljacob.de> parents: 45062 diff changeset	1463 def nullcontextmanager(enter_result=None):
a0791bfd9cfa util: enhance `nullcontextmanager` to be able to return __enter__ result Manuel Jacob <me@manueljacob.de> parents: 45062 diff changeset	1464 yield enter_result
33691 609606d21765 rebase: use one dirstateguard for when using rebase.singletransaction Durham Goode <durham@fb.com> parents: 33549 diff changeset	1465
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1466
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49031 diff changeset	1467 class _lrucachenode:
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1468 """A node in a doubly linked list.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1469
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1470 Holds a reference to nodes on either side as well as a key-value
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1471 pair for the dictionary entry.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1472 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1473
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43542 diff changeset	1474 __slots__ = ('next', 'prev', 'key', 'value', 'cost')
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1475
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1476 def __init__(self):
46901 51841b23670b typing: make minor adjustments to mercurial/util.py to pass pytype checking Matt Harbison <matt_harbison@yahoo.com> parents: 46900 diff changeset	1477 self.next = self
51841b23670b typing: make minor adjustments to mercurial/util.py to pass pytype checking Matt Harbison <matt_harbison@yahoo.com> parents: 46900 diff changeset	1478 self.prev = self
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1479
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1480 self.key = _notset
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1481 self.value = None
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1482 self.cost = 0
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1483
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1484 def markempty(self):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1485 """Mark the node as emptied."""
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1486 self.key = _notset
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1487 self.value = None
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1488 self.cost = 0
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1489
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1490
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49031 diff changeset	1491 class lrucachedict:
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1492 """Dict that caches most recent accesses and sets.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1493
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1494 The dict consists of an actual backing dict - indexed by original
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1495 key - and a doubly linked circular list defining the order of entries in
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1496 the cache.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1497
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1498 The head node is the newest entry in the cache. If the cache is full,
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1499 we recycle head.prev and make it the new head. Cache accesses result in
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1500 the node being moved to before the existing head and being marked as the
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1501 new head node.
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1502
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1503 Items in the cache can be inserted with an optional "cost" value. This is
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1504 simply an integer that is specified by the caller. The cache can be queried
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1505 for the total cost of all items presently in the cache.
39584 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1506
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1507 The cache can also define a maximum cost. If a cache insertion would
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1508 cause the total cost of the cache to go beyond the maximum cost limit,
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1509 nodes will be evicted to make room for the new code. This can be used
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1510 to e.g. set a max memory limit and associate an estimated bytes size
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1511 cost to each item in the cache. By default, no maximum cost is enforced.
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1512 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1513
39584 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1514 def __init__(self, max, maxcost=0):
18603 2251b3184e6e util: add an LRU cache dict Siddharth Agarwal <sid0@fb.com> parents: 18537 diff changeset	1515 self._cache = {}
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1516
46901 51841b23670b typing: make minor adjustments to mercurial/util.py to pass pytype checking Matt Harbison <matt_harbison@yahoo.com> parents: 46900 diff changeset	1517 self._head = _lrucachenode()
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1518 self._size = 1
39580 5d75a3c16193 util: make capacity a public attribute on lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39579 diff changeset	1519 self.capacity = max
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1520 self.totalcost = 0
39584 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1521 self.maxcost = maxcost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1522
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1523 def __len__(self):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1524 return len(self._cache)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1525
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1526 def __contains__(self, k):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1527 return k in self._cache
18603 2251b3184e6e util: add an LRU cache dict Siddharth Agarwal <sid0@fb.com> parents: 18537 diff changeset	1528
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1529 def __iter__(self):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1530 # We don't have to iterate in cache order, but why not.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1531 n = self._head
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1532 for i in range(len(self._cache)):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1533 yield n.key
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1534 n = n.next
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1535
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1536 def __getitem__(self, k):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1537 node = self._cache[k]
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1538 self._movetohead(node)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1539 return node.value
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1540
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1541 def insert(self, k, v, cost=0):
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1542 """Insert a new item in the cache with optional cost value."""
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1543 node = self._cache.get(k)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1544 # Replace existing value and mark as newest.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1545 if node is not None:
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1546 self.totalcost -= node.cost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1547 node.value = v
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1548 node.cost = cost
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1549 self.totalcost += cost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1550 self._movetohead(node)
39584 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1551
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1552 if self.maxcost:
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1553 self._enforcecostlimit()
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1554
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1555 return
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1556
39580 5d75a3c16193 util: make capacity a public attribute on lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39579 diff changeset	1557 if self._size < self.capacity:
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1558 node = self._addcapacity()
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1559 else:
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1560 # Grab the last/oldest item.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1561 node = self._head.prev
18603 2251b3184e6e util: add an LRU cache dict Siddharth Agarwal <sid0@fb.com> parents: 18537 diff changeset	1562
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1563 # At capacity. Kill the old entry.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1564 if node.key is not _notset:
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1565 self.totalcost -= node.cost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1566 del self._cache[node.key]
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1567
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1568 node.key = k
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1569 node.value = v
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1570 node.cost = cost
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1571 self.totalcost += cost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1572 self._cache[k] = node
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1573 # And mark it as newest entry. No need to adjust order since it
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1574 # is already self._head.prev.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1575 self._head = node
18603 2251b3184e6e util: add an LRU cache dict Siddharth Agarwal <sid0@fb.com> parents: 18537 diff changeset	1576
39584 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1577 if self.maxcost:
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1578 self._enforcecostlimit()
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1579
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1580 def __setitem__(self, k, v):
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1581 self.insert(k, v)
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1582
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1583 def __delitem__(self, k):
40898 7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1584 self.pop(k)
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1585
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1586 def pop(self, k, default=_notset):
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1587 try:
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1588 node = self._cache.pop(k)
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1589 except KeyError:
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1590 if default is _notset:
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1591 raise
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1592 return default
43887 b5655f337bd7 typing: add a couple of assertions to lrucachedict to help pytype Matt Harbison <matt_harbison@yahoo.com> parents: 43886 diff changeset	1593
40898 7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1594 value = node.value
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1595 self.totalcost -= node.cost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1596 node.markempty()
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1597
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1598 # Temporarily mark as newest item before re-adjusting head to make
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1599 # this node the oldest item.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1600 self._movetohead(node)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1601 self._head = node.next
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1602
40898 7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1603 return value
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1604
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1605 # Additional dict methods.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1606
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1607 def get(self, k, default=None):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1608 try:
39587 8f2c0d1b454c util: update lrucachedict order during get() Gregory Szorc <gregory.szorc@gmail.com> parents: 39586 diff changeset	1609 return self.__getitem__(k)
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1610 except KeyError:
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1611 return default
18603 2251b3184e6e util: add an LRU cache dict Siddharth Agarwal <sid0@fb.com> parents: 18537 diff changeset	1612
40897 0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1613 def peek(self, k, default=_notset):
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1614 """Get the specified item without moving it to the head
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1615
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1616 Unlike get(), this doesn't mutate the internal state. But be aware
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1617 that it doesn't mean peek() is thread safe.
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1618 """
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1619 try:
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1620 node = self._cache[k]
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1621 return node.value
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1622 except KeyError:
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1623 if default is _notset:
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1624 raise
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1625 return default
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1626
19710 887ffa22fd0d lrucachedict: implement clear() Siddharth Agarwal <sid0@fb.com> parents: 19461 diff changeset	1627 def clear(self):
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1628 n = self._head
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1629 while n.key is not _notset:
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1630 self.totalcost -= n.cost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1631 n.markempty()
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1632 n = n.next
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1633
19710 887ffa22fd0d lrucachedict: implement clear() Siddharth Agarwal <sid0@fb.com> parents: 19461 diff changeset	1634 self._cache.clear()
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1635
39584 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1636 def copy(self, capacity=None, maxcost=0):
39581 2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1637 """Create a new cache as a copy of the current one.
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1638
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1639 By default, the new cache has the same capacity as the existing one.
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1640 But, the cache capacity can be changed as part of performing the
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1641 copy.
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1642
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1643 Items in the copy have an insertion/access order matching this
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1644 instance.
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1645 """
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1646
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1647 capacity = capacity or self.capacity
39584 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1648 maxcost = maxcost or self.maxcost
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1649 result = lrucachedict(capacity, maxcost=maxcost)
39579 b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39287 diff changeset	1650
b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39287 diff changeset	1651 # We copy entries by iterating in oldest-to-newest order so the copy
b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39287 diff changeset	1652 # has the correct ordering.
b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39287 diff changeset	1653
b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39287 diff changeset	1654 # Find the first non-empty entry.
27576 6cd3044985c2 lrucachedict: add copy method Eric Sumner <ericsumner@fb.com> parents: 27391 diff changeset	1655 n = self._head.prev
39579 b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39287 diff changeset	1656 while n.key is _notset and n is not self._head:
b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39287 diff changeset	1657 n = n.prev
b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39287 diff changeset	1658
39581 2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1659 # We could potentially skip the first N items when decreasing capacity.
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1660 # But let's keep it simple unless it is a performance problem.
27576 6cd3044985c2 lrucachedict: add copy method Eric Sumner <ericsumner@fb.com> parents: 27391 diff changeset	1661 for i in range(len(self._cache)):
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1662 result.insert(n.key, n.value, cost=n.cost)
27576 6cd3044985c2 lrucachedict: add copy method Eric Sumner <ericsumner@fb.com> parents: 27391 diff changeset	1663 n = n.prev
39579 b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39287 diff changeset	1664
27576 6cd3044985c2 lrucachedict: add copy method Eric Sumner <ericsumner@fb.com> parents: 27391 diff changeset	1665 return result
6cd3044985c2 lrucachedict: add copy method Eric Sumner <ericsumner@fb.com> parents: 27391 diff changeset	1666
39582 bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1667 def popoldest(self):
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1668 """Remove the oldest item from the cache.
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1669
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1670 Returns the (key, value) describing the removed cache entry.
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1671 """
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1672 if not self._cache:
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1673 return
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1674
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1675 # Walk the linked list backwards starting at tail node until we hit
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1676 # a non-empty node.
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1677 n = self._head.prev
46901 51841b23670b typing: make minor adjustments to mercurial/util.py to pass pytype checking Matt Harbison <matt_harbison@yahoo.com> parents: 46900 diff changeset	1678
39582 bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1679 while n.key is _notset:
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1680 n = n.prev
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1681
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1682 key, value = n.key, n.value
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1683
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1684 # And remove it from the cache and mark it as empty.
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1685 del self._cache[n.key]
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1686 self.totalcost -= n.cost
39582 bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1687 n.markempty()
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1688
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1689 return key, value
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1690
51315 c8a2fdf5ca37 pytype: add the couple annotations for pytype to understands the lrunode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51311 diff changeset	1691 def _movetohead(self, node: _lrucachenode):
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1692 """Mark a node as the newest, making it the new head.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1693
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1694 When a node is accessed, it becomes the freshest entry in the LRU
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1695 list, which is denoted by self._head.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1696
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1697 Visually, let's make ``N`` the new head node (* denotes head):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1698
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1699 previous/oldest <-> head <-> next/next newest
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1700
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1701 ----<->--- A* ---<->-----
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1702 \| \|
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1703 E <-> D <-> N <-> C <-> B
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1704
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1705 To:
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1706
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1707 ----<->--- N* ---<->-----
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1708 \| \|
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1709 E <-> D <-> C <-> B <-> A
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1710
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1711 This requires the following moves:
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1712
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1713 C.next = D (node.prev.next = node.next)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1714 D.prev = C (node.next.prev = node.prev)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1715 E.next = N (head.prev.next = node)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1716 N.prev = E (node.prev = head.prev)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1717 N.next = A (node.next = head)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1718 A.prev = N (head.prev = node)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1719 """
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1720 head = self._head
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1721 # C.next = D
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1722 node.prev.next = node.next
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1723 # D.prev = C
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1724 node.next.prev = node.prev
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1725 # N.prev = E
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1726 node.prev = head.prev
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1727 # N.next = A
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1728 # It is tempting to do just "head" here, however if node is
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1729 # adjacent to head, this will do bad things.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1730 node.next = head.prev.next
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1731 # E.next = N
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1732 node.next.prev = node
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1733 # A.prev = N
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1734 node.prev.next = node
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1735
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1736 self._head = node
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1737
51315 c8a2fdf5ca37 pytype: add the couple annotations for pytype to understands the lrunode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51311 diff changeset	1738 def _addcapacity(self) -> _lrucachenode:
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1739 """Add a node to the circular linked list.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1740
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1741 The new node is inserted before the head node.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1742 """
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1743 head = self._head
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1744 node = _lrucachenode()
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1745 head.prev.next = node
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1746 node.prev = head.prev
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1747 node.next = head
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1748 head.prev = node
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1749 self._size += 1
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1750 return node
19710 887ffa22fd0d lrucachedict: implement clear() Siddharth Agarwal <sid0@fb.com> parents: 19461 diff changeset	1751
39584 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1752 def _enforcecostlimit(self):
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1753 # This should run after an insertion. It should only be called if total
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1754 # cost limits are being enforced.
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1755 # The most recently inserted node is never evicted.
39585 cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1756 if len(self) <= 1 or self.totalcost <= self.maxcost:
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1757 return
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1758
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1759 # This is logically equivalent to calling popoldest() until we
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1760 # free up enough cost. We don't do that since popoldest() needs
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1761 # to walk the linked list and doing this in a loop would be
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1762 # quadratic. So we find the first non-empty node and then
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1763 # walk nodes until we free up enough capacity.
39586 f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1764 #
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1765 # If we only removed the minimum number of nodes to free enough
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1766 # cost at insert time, chances are high that the next insert would
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1767 # also require pruning. This would effectively constitute quadratic
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1768 # behavior for insert-heavy workloads. To mitigate this, we set a
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1769 # target cost that is a percentage of the max cost. This will tend
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1770 # to free more nodes when the high water mark is reached, which
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1771 # lowers the chances of needing to prune on the subsequent insert.
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1772 targetcost = int(self.maxcost * 0.75)
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1773
39585 cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1774 n = self._head.prev
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1775 while n.key is _notset:
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1776 n = n.prev
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1777
39586 f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1778 while len(self) > 1 and self.totalcost > targetcost:
39585 cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1779 del self._cache[n.key]
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1780 self.totalcost -= n.cost
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1781 n.markempty()
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1782 n = n.prev
39584 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1783
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1784
9097 431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1785 def lrucachefunc(func):
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1786 '''cache most recent results of function calls'''
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1787 cache = {}
25113 0ca8410ea345 util: drop alias for collections.deque Martin von Zweigbergk <martinvonz@google.com> parents: 25112 diff changeset	1788 order = collections.deque()
28832 f5ff10f6fa6b util: use __code__ (available since py2.6) timeless <timeless@mozdev.org> parents: 28826 diff changeset	1789 if func.__code__.co_argcount == 1:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1790
9097 431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1791 def f(arg):
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1792 if arg not in cache:
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1793 if len(cache) > 20:
16803 107a3270a24a cleanup: use the deque type where appropriate Bryan O'Sullivan <bryano@fb.com> parents: 16769 diff changeset	1794 del cache[order.popleft()]
9097 431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1795 cache[arg] = func(arg)
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1796 else:
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1797 order.remove(arg)
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1798 order.append(arg)
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1799 return cache[arg]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1800
9097 431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1801 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1802
9097 431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1803 def f(*args):
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1804 if args not in cache:
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1805 if len(cache) > 20:
16803 107a3270a24a cleanup: use the deque type where appropriate Bryan O'Sullivan <bryano@fb.com> parents: 16769 diff changeset	1806 del cache[order.popleft()]
9097 431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1807 cache[args] = func(*args)
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1808 else:
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1809 order.remove(args)
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1810 order.append(args)
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1811 return cache[args]
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1812
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1813 return f
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1814
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1815
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49031 diff changeset	1816 class propertycache:
8207 dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1817 def __init__(self, func):
dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1818 self.func = func
dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1819 self.name = func.__name__
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1820
8207 dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1821 def __get__(self, obj, type=None):
dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1822 result = self.func(obj)
18013 98c867ac1330 clfilter: add a propertycache that must be unfiltered Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 17962 diff changeset	1823 self.cachevalue(obj, result)
8207 dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1824 return result
dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1825
18013 98c867ac1330 clfilter: add a propertycache that must be unfiltered Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 17962 diff changeset	1826 def cachevalue(self, obj, value):
19951 d51c4d85ec23 spelling: random spell checker fixes Mads Kiilerich <madski@unity3d.com> parents: 19852 diff changeset	1827 # __dict__ assignment required to bypass __setattr__ (eg: repoview)
19845 a1237a4b437d repoview: make propertycache.setcache compatible with repoview Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 19461 diff changeset	1828 obj.__dict__[self.name] = value
18013 98c867ac1330 clfilter: add a propertycache that must be unfiltered Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 17962 diff changeset	1829
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1830
35053 be6aa0cff8ea util: add util.clearcachedproperty Mark Thomas <mbthomas@fb.com> parents: 34647 diff changeset	1831 def clearcachedproperty(obj, prop):
be6aa0cff8ea util: add util.clearcachedproperty Mark Thomas <mbthomas@fb.com> parents: 34647 diff changeset	1832 '''clear a cached property value, if one has been set'''
40728 475921a3028c py3: cast attribute name to sysstr in clearcachedproperty() Yuya Nishihara <yuya@tcha.org> parents: 40251 diff changeset	1833 prop = pycompat.sysstr(prop)
35053 be6aa0cff8ea util: add util.clearcachedproperty Mark Thomas <mbthomas@fb.com> parents: 34647 diff changeset	1834 if prop in obj.__dict__:
be6aa0cff8ea util: add util.clearcachedproperty Mark Thomas <mbthomas@fb.com> parents: 34647 diff changeset	1835 del obj.__dict__[prop]
be6aa0cff8ea util: add util.clearcachedproperty Mark Thomas <mbthomas@fb.com> parents: 34647 diff changeset	1836
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1837
7396 526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1838 def increasingchunks(source, min=1024, max=65536):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	1839 """return no less than min bytes per chunk while data remains,
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	1840 doubling min after each chunk until it reaches max"""
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1841
7396 526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1842 def log2(x):
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1843 if not x:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1844 return 0
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1845 i = 0
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1846 while x:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1847 x >>= 1
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1848 i += 1
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1849 return i - 1
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1850
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1851 buf = []
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1852 blen = 0
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1853 for chunk in source:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1854 buf.append(chunk)
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1855 blen += len(chunk)
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1856 if blen >= min:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1857 if min < max:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1858 min = min << 1
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1859 nmin = 1 << log2(blen)
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1860 if nmin > min:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1861 min = nmin
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1862 if min > max:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1863 min = max
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1864 yield b''.join(buf)
7396 526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1865 blen = 0
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1866 buf = []
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1867 if buf:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1868 yield b''.join(buf)
7396 526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1869
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1870
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1871 def always(fn):
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1872 return True
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1873
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1874
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1875 def never(fn):
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1876 return False
724 1c0c413cccdd Get add and locate to use new repo and dirstate walk code. Bryan O'Sullivan <bos@serpentine.com> parents: 705 diff changeset	1877
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1878
51587 a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	1879 def nogc(func=None) -> Any:
23495 b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1880 """disable garbage collector
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1881
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1882 Python's garbage collector triggers a GC each time a certain number of
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1883 container objects (the number being defined by gc.get_threshold()) are
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1884 allocated even when marked not to be tracked by the collector. Tracking has
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1885 no effect on when GCs are triggered, only on what objects the GC looks
23543 4dd8a6a1240d spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23495 diff changeset	1886 into. As a workaround, disable GC while building complex (huge)
23495 b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1887 containers.
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1888
33820 05264fc9d8d6 util: make nogc effective for CPython Jun Wu <quark@fb.com> parents: 33814 diff changeset	1889 This garbage collector issue have been fixed in 2.7. But it still affect
05264fc9d8d6 util: make nogc effective for CPython Jun Wu <quark@fb.com> parents: 33814 diff changeset	1890 CPython's performance.
23495 b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1891 """
51587 a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	1892 if func is None:
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	1893 return _nogc_context()
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	1894 else:
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	1895 return _nogc_decorator(func)
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	1896
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	1897
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	1898 @contextlib.contextmanager
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	1899 def _nogc_context():
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	1900 gcenabled = gc.isenabled()
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	1901 gc.disable()
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	1902 try:
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	1903 yield
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	1904 finally:
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	1905 if gcenabled:
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	1906 gc.enable()
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	1907
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	1908
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	1909 def _nogc_decorator(func):
23495 b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1910 def wrapper(args, *kwargs):
51587 a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	1911 with _nogc_context():
23495 b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1912 return func(args, *kwargs)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1913
23495 b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1914 return wrapper
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1915
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1916
33820 05264fc9d8d6 util: make nogc effective for CPython Jun Wu <quark@fb.com> parents: 33814 diff changeset	1917 if pycompat.ispypy:
05264fc9d8d6 util: make nogc effective for CPython Jun Wu <quark@fb.com> parents: 33814 diff changeset	1918 # PyPy runs slower with gc disabled
05264fc9d8d6 util: make nogc effective for CPython Jun Wu <quark@fb.com> parents: 33814 diff changeset	1919 nogc = lambda x: x
05264fc9d8d6 util: make nogc effective for CPython Jun Wu <quark@fb.com> parents: 33814 diff changeset	1920
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1921
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	1922 def pathto(root: bytes, n1: bytes, n2: bytes) -> bytes:
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	1923 """return the relative path from one place to another.
4229 24c22a3f2ef8 pass repo.root to util.pathto() in preparation for the next patch Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4129 diff changeset	1924 root should use os.sep to separate directories
3669 48768b1ab23c fix util.pathto Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 3629 diff changeset	1925 n1 should use os.sep to separate directories
48768b1ab23c fix util.pathto Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 3629 diff changeset	1926 n2 should use "/" to separate directories
48768b1ab23c fix util.pathto Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 3629 diff changeset	1927 returns an os.sep-separated path.
4229 24c22a3f2ef8 pass repo.root to util.pathto() in preparation for the next patch Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4129 diff changeset	1928
24c22a3f2ef8 pass repo.root to util.pathto() in preparation for the next patch Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4129 diff changeset	1929 If n1 is a relative path, it's assumed it's
24c22a3f2ef8 pass repo.root to util.pathto() in preparation for the next patch Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4129 diff changeset	1930 relative to root.
24c22a3f2ef8 pass repo.root to util.pathto() in preparation for the next patch Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4129 diff changeset	1931 n2 should always be relative to root.
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	1932 """
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1933 if not n1:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1934 return localpath(n2)
4230 c93562fb12cc Fix handling of paths when run outside the repo. Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4229 diff changeset	1935 if os.path.isabs(n1):
c93562fb12cc Fix handling of paths when run outside the repo. Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4229 diff changeset	1936 if os.path.splitdrive(root)[0] != os.path.splitdrive(n1)[0]:
c93562fb12cc Fix handling of paths when run outside the repo. Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4229 diff changeset	1937 return os.path.join(root, localpath(n2))
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1938 n2 = b'/'.join((pconvert(root), n2))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1939 a, b = splitpath(n1), n2.split(b'/')
1541 bf4e7ef08741 fixed some stuff pychecker shows, marked unclear/wrong stuff with XXX twaldmann@thinkmo.de parents: 1528 diff changeset	1940 a.reverse()
bf4e7ef08741 fixed some stuff pychecker shows, marked unclear/wrong stuff with XXX twaldmann@thinkmo.de parents: 1528 diff changeset	1941 b.reverse()
884 087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types. Bryan O'Sullivan <bos@serpentine.com> parents: 878 diff changeset	1942 while a and b and a[-1] == b[-1]:
1541 bf4e7ef08741 fixed some stuff pychecker shows, marked unclear/wrong stuff with XXX twaldmann@thinkmo.de parents: 1528 diff changeset	1943 a.pop()
bf4e7ef08741 fixed some stuff pychecker shows, marked unclear/wrong stuff with XXX twaldmann@thinkmo.de parents: 1528 diff changeset	1944 b.pop()
884 087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types. Bryan O'Sullivan <bos@serpentine.com> parents: 878 diff changeset	1945 b.reverse()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1946 return pycompat.ossep.join(([b'..'] * len(a)) + b) or b'.'
884 087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types. Bryan O'Sullivan <bos@serpentine.com> parents: 878 diff changeset	1947
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1948
44456 09f3e003fc2a phabricator: avoid a stacktrace when command arguments are missing Matt Harbison <matt_harbison@yahoo.com> parents: 44060 diff changeset	1949 def checksignature(func, depth=1):
7388 5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1950 '''wrap a function with code to check for calling errors'''
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1951
7388 5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1952 def check(args, *kwargs):
5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1953 try:
5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1954 return func(args, *kwargs)
5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1955 except TypeError:
44456 09f3e003fc2a phabricator: avoid a stacktrace when command arguments are missing Matt Harbison <matt_harbison@yahoo.com> parents: 44060 diff changeset	1956 if len(traceback.extract_tb(sys.exc_info()[2])) == depth:
7646 e62a456b8dc5 error: move SignatureError Matt Mackall <mpm@selenic.com> parents: 7644 diff changeset	1957 raise error.SignatureError
7388 5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1958 raise
5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1959
5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1960 return check
5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1961
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1962
31581 e506e461c7a9 util: disable hardlink for copyfile if fstype is outside a whitelist Jun Wu <quark@fb.com> parents: 31579 diff changeset	1963 # a whilelist of known filesystems where hardlink works reliably
32331 bd872f64a8ba cleanup: use set literals Martin von Zweigbergk <martinvonz@google.com> parents: 32330 diff changeset	1964 _hardlinkfswhitelist = {
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1965 b'apfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1966 b'btrfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1967 b'ext2',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1968 b'ext3',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1969 b'ext4',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1970 b'hfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1971 b'jfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1972 b'NTFS',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1973 b'reiserfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1974 b'tmpfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1975 b'ufs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1976 b'xfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1977 b'zfs',
32331 bd872f64a8ba cleanup: use set literals Martin von Zweigbergk <martinvonz@google.com> parents: 32330 diff changeset	1978 }
31581 e506e461c7a9 util: disable hardlink for copyfile if fstype is outside a whitelist Jun Wu <quark@fb.com> parents: 31579 diff changeset	1979
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1980
47406 9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47244 diff changeset	1981 def copyfile(
47451 d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47423 diff changeset	1982 src,
d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47423 diff changeset	1983 dest,
d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47423 diff changeset	1984 hardlink=False,
d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47423 diff changeset	1985 copystat=False,
d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47423 diff changeset	1986 checkambig=False,
d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47423 diff changeset	1987 nb_bytes=None,
d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47423 diff changeset	1988 no_hardlink_cb=None,
47452 09ff5d532a25 copyfiles: add a way to relax the file system checking for hardlink Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47451 diff changeset	1989 check_fs_hardlink=True,
47406 9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47244 diff changeset	1990 ):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	1991 """copy a file, preserving mode and optionally other stat info like
29367 4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1992 atime/mtime
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1993
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1994 checkambig argument is used with filestat, and is useful only if
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1995 destination file is guarded by any lock (e.g. repo.lock or
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1996 repo.wlock).
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1997
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1998 copystat and checkambig should be exclusive.
47406 9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47244 diff changeset	1999
9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47244 diff changeset	2000 nb_bytes: if set only copy the first `nb_bytes` of the source file.
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2001 """
29204 ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	2002 assert not (copystat and checkambig)
ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	2003 oldstat = None
18326 614f769e6aa7 util: copyfile: remove dest before copying Mads Kiilerich <mads@kiilerich.com> parents: 18026 diff changeset	2004 if os.path.lexists(dest):
29204 ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	2005 if checkambig:
32792 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	2006 oldstat = checkambig and filestat.frompath(dest)
18326 614f769e6aa7 util: copyfile: remove dest before copying Mads Kiilerich <mads@kiilerich.com> parents: 18026 diff changeset	2007 unlink(dest)
47452 09ff5d532a25 copyfiles: add a way to relax the file system checking for hardlink Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47451 diff changeset	2008 if hardlink and check_fs_hardlink:
31581 e506e461c7a9 util: disable hardlink for copyfile if fstype is outside a whitelist Jun Wu <quark@fb.com> parents: 31579 diff changeset	2009 # Hardlinks are problematic on CIFS (issue4546), do not allow hardlinks
e506e461c7a9 util: disable hardlink for copyfile if fstype is outside a whitelist Jun Wu <quark@fb.com> parents: 31579 diff changeset	2010 # unless we are confident that dest is on a whitelisted filesystem.
31683 1ed57a7dd904 statfs: make getfstype() raise OSError Yuya Nishihara <yuya@tcha.org> parents: 31667 diff changeset	2011 try:
1ed57a7dd904 statfs: make getfstype() raise OSError Yuya Nishihara <yuya@tcha.org> parents: 31667 diff changeset	2012 fstype = getfstype(os.path.dirname(dest))
1ed57a7dd904 statfs: make getfstype() raise OSError Yuya Nishihara <yuya@tcha.org> parents: 31667 diff changeset	2013 except OSError:
1ed57a7dd904 statfs: make getfstype() raise OSError Yuya Nishihara <yuya@tcha.org> parents: 31667 diff changeset	2014 fstype = None
31581 e506e461c7a9 util: disable hardlink for copyfile if fstype is outside a whitelist Jun Wu <quark@fb.com> parents: 31579 diff changeset	2015 if fstype not in _hardlinkfswhitelist:
47451 d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47423 diff changeset	2016 if no_hardlink_cb is not None:
d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47423 diff changeset	2017 no_hardlink_cb()
31581 e506e461c7a9 util: disable hardlink for copyfile if fstype is outside a whitelist Jun Wu <quark@fb.com> parents: 31579 diff changeset	2018 hardlink = False
31583 e7a02e9ad162 util: enable hardlink for copyfile Jun Wu <quark@fb.com> parents: 31581 diff changeset	2019 if hardlink:
23899 4e451d1359de copyfile: allow optional hardlinking Pierre-Yves David <pierre-yves.david@fb.com> parents: 23864 diff changeset	2020 try:
4e451d1359de copyfile: allow optional hardlinking Pierre-Yves David <pierre-yves.david@fb.com> parents: 23864 diff changeset	2021 oslink(src, dest)
47406 9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47244 diff changeset	2022 if nb_bytes is not None:
9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47244 diff changeset	2023 m = "the `nb_bytes` argument is incompatible with `hardlink`"
9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47244 diff changeset	2024 raise error.ProgrammingError(m)
23899 4e451d1359de copyfile: allow optional hardlinking Pierre-Yves David <pierre-yves.david@fb.com> parents: 23864 diff changeset	2025 return
52665 24ee91ba9aa8 pyupgrade: drop usage of py3 aliases for `OSError` Matt Harbison <matt_harbison@yahoo.com> parents: 52392 diff changeset	2026 except OSError as exc:
47451 d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47423 diff changeset	2027 if exc.errno != errno.EEXIST and no_hardlink_cb is not None:
d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47423 diff changeset	2028 no_hardlink_cb()
d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47423 diff changeset	2029 # fall back to normal copy
4271 1eaa8d90c689 fix util.copyfile to deal with symlinks Eric St-Jean <esj@wwd.ca> parents: 4256 diff changeset	2030 if os.path.islink(src):
1eaa8d90c689 fix util.copyfile to deal with symlinks Eric St-Jean <esj@wwd.ca> parents: 4256 diff changeset	2031 os.symlink(os.readlink(src), dest)
27369 c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	2032 # copytime is ignored for symlinks, but in general copytime isn't needed
c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	2033 # for them anyway
47406 9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47244 diff changeset	2034 if nb_bytes is not None:
9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47244 diff changeset	2035 m = "cannot use `nb_bytes` on a symlink"
9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47244 diff changeset	2036 raise error.ProgrammingError(m)
4271 1eaa8d90c689 fix util.copyfile to deal with symlinks Eric St-Jean <esj@wwd.ca> parents: 4256 diff changeset	2037 else:
1eaa8d90c689 fix util.copyfile to deal with symlinks Eric St-Jean <esj@wwd.ca> parents: 4256 diff changeset	2038 try:
1eaa8d90c689 fix util.copyfile to deal with symlinks Eric St-Jean <esj@wwd.ca> parents: 4256 diff changeset	2039 shutil.copyfile(src, dest)
27369 c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	2040 if copystat:
c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	2041 # copystat also copies mode
c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	2042 shutil.copystat(src, dest)
c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	2043 else:
c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	2044 shutil.copymode(src, dest)
29204 ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	2045 if oldstat and oldstat.stat:
32792 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	2046 newstat = filestat.frompath(dest)
29204 ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	2047 if newstat.isambig(oldstat):
ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	2048 # stat of copied file is ambiguous to original one
36789 ffa3026d4196 cleanup: use stat_result[stat.ST_MTIME] instead of stat_result.st_mtime Augie Fackler <augie@google.com> parents: 36747 diff changeset	2049 advanced = (
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2050 oldstat.stat[stat.ST_MTIME] + 1
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2051 ) & 0x7FFFFFFF
29204 ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	2052 os.utime(dest, (advanced, advanced))
47406 9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47244 diff changeset	2053 # We could do something smarter using `copy_file_range` call or similar
9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47244 diff changeset	2054 if nb_bytes is not None:
9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47244 diff changeset	2055 with open(dest, mode='r+') as f:
9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47244 diff changeset	2056 f.truncate(nb_bytes)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25420 diff changeset	2057 except shutil.Error as inst:
43440 5be128f669d4 util: convert an exception to bytes when passing to Abort() Matt Harbison <matt_harbison@yahoo.com> parents: 43117 diff changeset	2058 raise error.Abort(stringutil.forcebytestr(inst))
3629 4cfb72bcb978 util: add copyfile function Matt Mackall <mpm@selenic.com> parents: 3568 diff changeset	2059
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2060
38386 63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38323 diff changeset	2061 def copyfiles(src, dst, hardlink=None, progress=None):
24439 2ddfac2f163e util: add progress callback support to copyfiles Augie Fackler <augie@google.com> parents: 24236 diff changeset	2062 """Copy a directory tree using hardlinks if possible."""
2ddfac2f163e util: add progress callback support to copyfiles Augie Fackler <augie@google.com> parents: 24236 diff changeset	2063 num = 0
1241 3b4f05ff3130 Add support for cloning with hardlinks on windows. Stephen Darnell parents: 1207 diff changeset	2064
38386 63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38323 diff changeset	2065 def settopic():
63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38323 diff changeset	2066 if progress:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2067 progress.topic = _(b'linking') if hardlink else _(b'copying')
698 df78d8ccac4c Use python function instead of external 'cp' command when cloning repos. Thomas Arendsen Hein <thomas@intevation.de> parents: 667 diff changeset	2068
1207 a7b8812973d9 Rewrite copytree as copyfiles mpm@selenic.com parents: 1200 diff changeset	2069 if os.path.isdir(src):
31724 456efd1b51fd hardlink: duplicate hardlink detection for copying files and directories Jun Wu <quark@fb.com> parents: 31723 diff changeset	2070 if hardlink is None:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2071 hardlink = (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2072 os.stat(src).st_dev == os.stat(os.path.dirname(dst)).st_dev
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2073 )
38386 63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38323 diff changeset	2074 settopic()
1207 a7b8812973d9 Rewrite copytree as copyfiles mpm@selenic.com parents: 1200 diff changeset	2075 os.mkdir(dst)
32248 d74b0cff94a9 osutil: proxy through util (and platform) modules (API) Yuya Nishihara <yuya@tcha.org> parents: 32245 diff changeset	2076 for name, kind in listdir(src):
1207 a7b8812973d9 Rewrite copytree as copyfiles mpm@selenic.com parents: 1200 diff changeset	2077 srcname = os.path.join(src, name)
a7b8812973d9 Rewrite copytree as copyfiles mpm@selenic.com parents: 1200 diff changeset	2078 dstname = os.path.join(dst, name)
38386 63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38323 diff changeset	2079 hardlink, n = copyfiles(srcname, dstname, hardlink, progress)
11251 c61442f6d106 clone: print number of linked/copied files on --debug Adrian Buehlmann <adrian@cadifra.com> parents: 11232 diff changeset	2080 num += n
1207 a7b8812973d9 Rewrite copytree as copyfiles mpm@selenic.com parents: 1200 diff changeset	2081 else:
31724 456efd1b51fd hardlink: duplicate hardlink detection for copying files and directories Jun Wu <quark@fb.com> parents: 31723 diff changeset	2082 if hardlink is None:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2083 hardlink = (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2084 os.stat(os.path.dirname(src)).st_dev
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2085 == os.stat(os.path.dirname(dst)).st_dev
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2086 )
38386 63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38323 diff changeset	2087 settopic()
31724 456efd1b51fd hardlink: duplicate hardlink detection for copying files and directories Jun Wu <quark@fb.com> parents: 31723 diff changeset	2088
1241 3b4f05ff3130 Add support for cloning with hardlinks on windows. Stephen Darnell parents: 1207 diff changeset	2089 if hardlink:
3b4f05ff3130 Add support for cloning with hardlinks on windows. Stephen Darnell parents: 1207 diff changeset	2090 try:
14235 b9e1b041744f rename util.os_link to oslink Adrian Buehlmann <adrian@cadifra.com> parents: 14234 diff changeset	2091 oslink(src, dst)
52665 24ee91ba9aa8 pyupgrade: drop usage of py3 aliases for `OSError` Matt Harbison <matt_harbison@yahoo.com> parents: 52392 diff changeset	2092 except OSError as exc:
47423 9ea525216edb copyfiles: deal with existing file when hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47406 diff changeset	2093 if exc.errno != errno.EEXIST:
9ea525216edb copyfiles: deal with existing file when hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47406 diff changeset	2094 hardlink = False
9ea525216edb copyfiles: deal with existing file when hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47406 diff changeset	2095 # XXX maybe try to relink if the file exist ?
1591 5a3229cf1492 do not copy atime and mtime in util.copyfiles Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 1585 diff changeset	2096 shutil.copy(src, dst)
1241 3b4f05ff3130 Add support for cloning with hardlinks on windows. Stephen Darnell parents: 1207 diff changeset	2097 else:
1591 5a3229cf1492 do not copy atime and mtime in util.copyfiles Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 1585 diff changeset	2098 shutil.copy(src, dst)
11251 c61442f6d106 clone: print number of linked/copied files on --debug Adrian Buehlmann <adrian@cadifra.com> parents: 11232 diff changeset	2099 num += 1
38386 63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38323 diff changeset	2100 if progress:
63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38323 diff changeset	2101 progress.increment()
698 df78d8ccac4c Use python function instead of external 'cp' command when cloning repos. Thomas Arendsen Hein <thomas@intevation.de> parents: 667 diff changeset	2102
11251 c61442f6d106 clone: print number of linked/copied files on --debug Adrian Buehlmann <adrian@cadifra.com> parents: 11232 diff changeset	2103 return hardlink, num
11254 640d419725d0 util.copyfiles: don't try os_link() again if it failed before Adrian Buehlmann <adrian@cadifra.com> parents: 11010 diff changeset	2104
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2105
34069 ca6a3852daf0 util: use set for reserved Windows filenames Gregory Szorc <gregory.szorc@gmail.com> parents: 34020 diff changeset	2106 _winreservednames = {
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2107 b'con',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2108 b'prn',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2109 b'aux',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2110 b'nul',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2111 b'com1',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2112 b'com2',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2113 b'com3',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2114 b'com4',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2115 b'com5',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2116 b'com6',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2117 b'com7',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2118 b'com8',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2119 b'com9',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2120 b'lpt1',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2121 b'lpt2',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2122 b'lpt3',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2123 b'lpt4',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2124 b'lpt5',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2125 b'lpt6',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2126 b'lpt7',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2127 b'lpt8',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2128 b'lpt9',
34069 ca6a3852daf0 util: use set for reserved Windows filenames Gregory Szorc <gregory.szorc@gmail.com> parents: 34020 diff changeset	2129 }
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2130 _winreservedchars = b':*?"<>\|'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2131
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2132
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	2133 def checkwinfilename(path: bytes) -> Optional[bytes]:
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2134 r"""Check that the base-relative path is a valid filename on Windows.
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2135 Returns None if the path is ok, or a UI string describing the problem.
98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2136
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2137 >>> checkwinfilename(b"just/a/normal/path")
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2138 >>> checkwinfilename(b"foo/bar/con.xml")
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2139 "filename contains 'con', which is reserved on Windows"
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2140 >>> checkwinfilename(b"foo/con.xml/bar")
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2141 "filename contains 'con', which is reserved on Windows"
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2142 >>> checkwinfilename(b"foo/bar/xml.con")
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2143 >>> checkwinfilename(b"foo/bar/AUX/bla.txt")
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2144 "filename contains 'AUX', which is reserved on Windows"
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2145 >>> checkwinfilename(b"foo/bar/bla:.txt")
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2146 "filename contains ':', which is reserved on Windows"
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2147 >>> checkwinfilename(b"foo/bar/b\07la.txt")
20000 0849d280663e util: warn when adding paths ending with \ Mads Kiilerich <madski@unity3d.com> parents: 19951 diff changeset	2148 "filename contains '\\x07', which is invalid on Windows"
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2149 >>> checkwinfilename(b"foo/bar/bla ")
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2150 "filename ends with ' ', which is not allowed on Windows"
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2151 >>> checkwinfilename(b"../bar")
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2152 >>> checkwinfilename(b"foo\\")
20000 0849d280663e util: warn when adding paths ending with \ Mads Kiilerich <madski@unity3d.com> parents: 19951 diff changeset	2153 "filename ends with '\\', which is invalid on Windows"
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2154 >>> checkwinfilename(b"foo\\/bar")
20000 0849d280663e util: warn when adding paths ending with \ Mads Kiilerich <madski@unity3d.com> parents: 19951 diff changeset	2155 "directory name ends with '\\', which is invalid on Windows"
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2156 """
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2157 if path.endswith(b'\\'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2158 return _(b"filename ends with '\\', which is invalid on Windows")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2159 if b'\\/' in path:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2160 return _(b"directory name ends with '\\', which is invalid on Windows")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2161 for n in path.replace(b'\\', b'/').split(b'/'):
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2162 if not n:
98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2163 continue
32131 377c74ef008d win32mbcs: avoid unintentional failure at colorization FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 31952 diff changeset	2164 for c in _filenamebytestr(n):
14262 23cd7eeff678 util: rename _windows_reserved_filenames and _windows_reserved_chars Adrian Buehlmann <adrian@cadifra.com> parents: 14250 diff changeset	2165 if c in _winreservedchars:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2166 return (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2167 _(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2168 b"filename contains '%s', which is reserved "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2169 b"on Windows"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2170 )
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2171 % c
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2172 )
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2173 if ord(c) <= 31:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2174 return _(
43117 8ff1ecfadcd1 cleanup: join string literals that are already on one line Martin von Zweigbergk <martinvonz@google.com> parents: 43106 diff changeset	2175 b"filename contains '%s', which is invalid on Windows"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2176 ) % stringutil.escapestr(c)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2177 base = n.split(b'.')[0]
14262 23cd7eeff678 util: rename _windows_reserved_filenames and _windows_reserved_chars Adrian Buehlmann <adrian@cadifra.com> parents: 14250 diff changeset	2178 if base and base.lower() in _winreservednames:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2179 return (
43117 8ff1ecfadcd1 cleanup: join string literals that are already on one line Martin von Zweigbergk <martinvonz@google.com> parents: 43106 diff changeset	2180 _(b"filename contains '%s', which is reserved on Windows")
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2181 % base
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2182 )
34362 7508a7dc95c1 py3: replace bytes[n] with slicing in checkwinfilename() Yuya Nishihara <yuya@tcha.org> parents: 34361 diff changeset	2183 t = n[-1:]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2184 if t in b'. ' and n not in b'..':
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2185 return (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2186 _(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2187 b"filename ends with '%s', which is not allowed "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2188 b"on Windows"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2189 )
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2190 % t
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2191 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2192
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2193
44020 f9d29e1d3354 util: avoid referencing `time.clock()` on Windows when missing (issue6238) Matt Harbison <matt_harbison@yahoo.com> parents: 43926 diff changeset	2194 timer = getattr(time, "perf_counter", None)
f9d29e1d3354 util: avoid referencing `time.clock()` on Windows when missing (issue6238) Matt Harbison <matt_harbison@yahoo.com> parents: 43926 diff changeset	2195
34645 75979c8d4572 codemod: use pycompat.iswindows Jun Wu <quark@fb.com> parents: 34554 diff changeset	2196 if pycompat.iswindows:
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2197 checkosfilename = checkwinfilename
44020 f9d29e1d3354 util: avoid referencing `time.clock()` on Windows when missing (issue6238) Matt Harbison <matt_harbison@yahoo.com> parents: 43926 diff changeset	2198 if not timer:
51311 7d3b92e8df13 pytype: ignore attribute error for time.clock Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51304 diff changeset	2199 timer = time.clock # pytype: disable=module-attr
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2200 else:
43889 09bcbeacedc7 typing: suppress a warning that mercurial.windows.checkosfilename is missing Matt Harbison <matt_harbison@yahoo.com> parents: 43888 diff changeset	2201 # mercurial.windows doesn't have platform.checkosfilename
09bcbeacedc7 typing: suppress a warning that mercurial.windows.checkosfilename is missing Matt Harbison <matt_harbison@yahoo.com> parents: 43888 diff changeset	2202 checkosfilename = platform.checkosfilename # pytype: disable=module-attr
44020 f9d29e1d3354 util: avoid referencing `time.clock()` on Windows when missing (issue6238) Matt Harbison <matt_harbison@yahoo.com> parents: 43926 diff changeset	2203 if not timer:
f9d29e1d3354 util: avoid referencing `time.clock()` on Windows when missing (issue6238) Matt Harbison <matt_harbison@yahoo.com> parents: 43926 diff changeset	2204 timer = time.time
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2205
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2206
51926 cbd01bf33802 typing: add type annotations to `mercurial.util.makelock()` Matt Harbison <matt_harbison@yahoo.com> parents: 51925 diff changeset	2207 def makelock(info: bytes, pathname: bytes) -> None:
36724 d77c3b023393 lock: block signal interrupt while making a lock file Yuya Nishihara <yuya@tcha.org> parents: 36722 diff changeset	2208 """Create a lock file atomically if possible
d77c3b023393 lock: block signal interrupt while making a lock file Yuya Nishihara <yuya@tcha.org> parents: 36722 diff changeset	2209
d77c3b023393 lock: block signal interrupt while making a lock file Yuya Nishihara <yuya@tcha.org> parents: 36722 diff changeset	2210 This may leave a stale lock file if symlink isn't supported and signal
d77c3b023393 lock: block signal interrupt while making a lock file Yuya Nishihara <yuya@tcha.org> parents: 36722 diff changeset	2211 interrupt is enabled.
d77c3b023393 lock: block signal interrupt while making a lock file Yuya Nishihara <yuya@tcha.org> parents: 36722 diff changeset	2212 """
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2213 try:
e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2214 return os.symlink(info, pathname)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25420 diff changeset	2215 except OSError as why:
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2216 if why.errno == errno.EEXIST:
e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2217 raise
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2218 except AttributeError: # no symlink in os
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2219 pass
e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2220
36791 1fbbb8e83392 py3: read/write plain lock file in binary mode Yuya Nishihara <yuya@tcha.org> parents: 36790 diff changeset	2221 flags = os.O_CREAT \| os.O_WRONLY \| os.O_EXCL \| getattr(os, 'O_BINARY', 0)
1fbbb8e83392 py3: read/write plain lock file in binary mode Yuya Nishihara <yuya@tcha.org> parents: 36790 diff changeset	2222 ld = os.open(pathname, flags)
51925 f833ad92ee44 util: avoid a leaked file descriptor in `util.makelock()` exceptional case Matt Harbison <matt_harbison@yahoo.com> parents: 51924 diff changeset	2223 try:
f833ad92ee44 util: avoid a leaked file descriptor in `util.makelock()` exceptional case Matt Harbison <matt_harbison@yahoo.com> parents: 51924 diff changeset	2224 os.write(ld, info)
f833ad92ee44 util: avoid a leaked file descriptor in `util.makelock()` exceptional case Matt Harbison <matt_harbison@yahoo.com> parents: 51924 diff changeset	2225 finally:
f833ad92ee44 util: avoid a leaked file descriptor in `util.makelock()` exceptional case Matt Harbison <matt_harbison@yahoo.com> parents: 51924 diff changeset	2226 os.close(ld)
704 5ca319a641e1 Make makelock and readlock work on filesystems without symlink support. Thomas Arendsen Hein <thomas@intevation.de> parents: 698 diff changeset	2227
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2228
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	2229 def readlock(pathname: bytes) -> bytes:
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2230 try:
39909 5fe0b880200e py3: convert os.readlink() path to native strings on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 39793 diff changeset	2231 return readlink(pathname)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25420 diff changeset	2232 except OSError as why:
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2233 if why.errno not in (errno.EINVAL, errno.ENOSYS):
e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2234 raise
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2235 except AttributeError: # no symlink in os
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2236 pass
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2237 with posixfile(pathname, b'rb') as fp:
39910 4017968f0a1d util: use a context manager in readlock() Matt Harbison <matt_harbison@yahoo.com> parents: 39909 diff changeset	2238 return fp.read()
704 5ca319a641e1 Make makelock and readlock work on filesystems without symlink support. Thomas Arendsen Hein <thomas@intevation.de> parents: 698 diff changeset	2239
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2240
2176 9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2241 def fstat(fp):
9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2242 '''stat file object that may not have fileno method.'''
9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2243 try:
9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2244 return os.fstat(fp.fileno())
9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2245 except AttributeError:
9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2246 return os.stat(fp.name)
9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2247
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2248
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2249 # File system features
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2250
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2251
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	2252 def fscasesensitive(path: bytes) -> bool:
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2253 """
18911 451eb1c27c1b util: improve doc for checkcase Mads Kiilerich <mads@kiilerich.com> parents: 18868 diff changeset	2254 Return true if the given path is on a case-sensitive filesystem
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2255
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2256 Requires a path (like /foo/.hg) ending with a foldable final
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2257 directory component.
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2258 """
24902 986a5c23b1c1 util.checkcase: don't abort on broken symlinks Siddharth Agarwal <sid0@fb.com> parents: 24692 diff changeset	2259 s1 = os.lstat(path)
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2260 d, b = os.path.split(path)
15667 eacfd851cb9e icasefs: consider as case sensitive if there is no counterevidence, for safety FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15609 diff changeset	2261 b2 = b.upper()
eacfd851cb9e icasefs: consider as case sensitive if there is no counterevidence, for safety FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15609 diff changeset	2262 if b == b2:
eacfd851cb9e icasefs: consider as case sensitive if there is no counterevidence, for safety FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15609 diff changeset	2263 b2 = b.lower()
eacfd851cb9e icasefs: consider as case sensitive if there is no counterevidence, for safety FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15609 diff changeset	2264 if b == b2:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2265 return True # no evidence against case sensitivity
15667 eacfd851cb9e icasefs: consider as case sensitive if there is no counterevidence, for safety FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15609 diff changeset	2266 p2 = os.path.join(d, b2)
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2267 try:
24902 986a5c23b1c1 util.checkcase: don't abort on broken symlinks Siddharth Agarwal <sid0@fb.com> parents: 24692 diff changeset	2268 s2 = os.lstat(p2)
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2269 if s2 == s1:
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2270 return False
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2271 return True
14004 97ed99d1f419 eliminate various naked except clauses Idan Kamara <idankk86@gmail.com> parents: 13985 diff changeset	2272 except OSError:
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2273 return True
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2274
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2275
46422 112826b59476 re2: feed unicode string to re2 module when necessary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46372 diff changeset	2276 _re2_input = lambda x: x
51486 6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	2277 # google-re2 will need to be tell to not output error on its own
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	2278 _re2_options = None
16943 8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2279 try:
43885 40bd667491a7 pytype: suppress the import-error in util.py when importing re2 Matt Harbison <matt_harbison@yahoo.com> parents: 43774 diff changeset	2280 import re2 # pytype: disable=import-error
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2281
16943 8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2282 _re2 = None
8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2283 except ImportError:
8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2284 _re2 = False
8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2285
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2286
50398 a45460e235a2 re2: fix reporting of availability in `hg debuginstall` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50397 diff changeset	2287 def has_re2():
a45460e235a2 re2: fix reporting of availability in `hg debuginstall` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50397 diff changeset	2288 """return True is re2 is available, False otherwise"""
a45460e235a2 re2: fix reporting of availability in `hg debuginstall` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50397 diff changeset	2289 if _re2 is None:
a45460e235a2 re2: fix reporting of availability in `hg debuginstall` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50397 diff changeset	2290 _re._checkre2()
a45460e235a2 re2: fix reporting of availability in `hg debuginstall` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50397 diff changeset	2291 return _re2
a45460e235a2 re2: fix reporting of availability in `hg debuginstall` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50397 diff changeset	2292
a45460e235a2 re2: fix reporting of availability in `hg debuginstall` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50397 diff changeset	2293
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49031 diff changeset	2294 class _re:
50397 293e1763982e re: make _checkre2 a static method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50396 diff changeset	2295 @staticmethod
293e1763982e re: make _checkre2 a static method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50396 diff changeset	2296 def _checkre2():
21913 50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2297 global _re2
46422 112826b59476 re2: feed unicode string to re2 module when necessary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46372 diff changeset	2298 global _re2_input
51486 6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	2299 global _re2_options
50396 82cf392c99f6 re2: exit `_checkre2` early if calling it is useless Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49943 diff changeset	2300 if _re2 is not None:
82cf392c99f6 re2: exit `_checkre2` early if calling it is useless Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49943 diff changeset	2301 # we already have the answer
82cf392c99f6 re2: exit `_checkre2` early if calling it is useless Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49943 diff changeset	2302 return
46926 9c3e84569071 re2: adjust local variable assignment scope Matt Harbison <matt_harbison@yahoo.com> parents: 46907 diff changeset	2303
9c3e84569071 re2: adjust local variable assignment scope Matt Harbison <matt_harbison@yahoo.com> parents: 46907 diff changeset	2304 check_pattern = br'\[([^\[]+)\]'
9c3e84569071 re2: adjust local variable assignment scope Matt Harbison <matt_harbison@yahoo.com> parents: 46907 diff changeset	2305 check_input = b'[ui]'
21913 50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2306 try:
50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2307 # check if match works, see issue3964
46422 112826b59476 re2: feed unicode string to re2 module when necessary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46372 diff changeset	2308 _re2 = bool(re2.match(check_pattern, check_input))
21913 50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2309 except ImportError:
50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2310 _re2 = False
46422 112826b59476 re2: feed unicode string to re2 module when necessary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46372 diff changeset	2311 except TypeError:
112826b59476 re2: feed unicode string to re2 module when necessary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46372 diff changeset	2312 # the `pyre-2` project provides a re2 module that accept bytes
112826b59476 re2: feed unicode string to re2 module when necessary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46372 diff changeset	2313 # the `fb-re2` project provides a re2 module that acccept sysstr
112826b59476 re2: feed unicode string to re2 module when necessary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46372 diff changeset	2314 check_pattern = pycompat.sysstr(check_pattern)
112826b59476 re2: feed unicode string to re2 module when necessary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46372 diff changeset	2315 check_input = pycompat.sysstr(check_input)
112826b59476 re2: feed unicode string to re2 module when necessary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46372 diff changeset	2316 _re2 = bool(re2.match(check_pattern, check_input))
112826b59476 re2: feed unicode string to re2 module when necessary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46372 diff changeset	2317 _re2_input = pycompat.sysstr
51486 6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	2318 try:
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	2319 quiet = re2.Options()
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	2320 quiet.log_errors = False
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	2321 _re2_options = quiet
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	2322 except AttributeError:
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	2323 pass
21913 50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2324
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2325 def compile(self, pat, flags=0):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2326 """Compile a regular expression, using re2 if possible
16943 8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2327
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2328 For best performance, use only re2-compatible regexp features. The
cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2329 only flags from the re module that are re2-compatible are
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2330 IGNORECASE and MULTILINE."""
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2331 if _re2 is None:
21913 50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2332 self._checkre2()
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2333 if _re2 and (flags & ~(remod.IGNORECASE \| remod.MULTILINE)) == 0:
cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2334 if flags & remod.IGNORECASE:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2335 pat = b'(?i)' + pat
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2336 if flags & remod.MULTILINE:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2337 pat = b'(?m)' + pat
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2338 try:
51486 6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	2339 input_regex = _re2_input(pat)
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	2340 if _re2_options is not None:
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	2341 compiled = re2.compile(input_regex, options=_re2_options)
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	2342 else:
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	2343 compiled = re2.compile(input_regex)
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51428 diff changeset	2344 return compiled
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2345 except re2.error:
cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2346 pass
cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2347 return remod.compile(pat, flags)
cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2348
21914 10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2349 @propertycache
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2350 def escape(self):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2351 """Return the version of escape corresponding to self.compile.
21914 10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2352
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2353 This is imperfect because whether re2 or re is used for a particular
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2354 function depends on the flags, etc, but it's the best we can do.
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2355 """
21914 10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2356 global _re2
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2357 if _re2 is None:
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2358 self._checkre2()
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2359 if _re2:
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2360 return re2.escape
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2361 else:
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2362 return remod.escape
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2363
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2364
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2365 re = _re()
16943 8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2366
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2367 _fspathcache = {}
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2368
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2369
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	2370 def fspath(name: bytes, root: bytes) -> bytes:
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2371 """Get name in the case stored in the filesystem
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2372
15710 f63e40047372 icasefs: avoid path-absoluteness/existance check in util.fspath() for efficiency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15709 diff changeset	2373 The name should be relative to root, and be normcase-ed for efficiency.
f63e40047372 icasefs: avoid path-absoluteness/existance check in util.fspath() for efficiency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15709 diff changeset	2374
f63e40047372 icasefs: avoid path-absoluteness/existance check in util.fspath() for efficiency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15709 diff changeset	2375 Note that this function is unnecessary, and should not be
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2376 called, for case-sensitive filesystems (simply because it's expensive).
15670 d6c19cfa03ce icasefs: avoid normcase()-ing in util.fspath() for efficiency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15669 diff changeset	2377
15710 f63e40047372 icasefs: avoid path-absoluteness/existance check in util.fspath() for efficiency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15709 diff changeset	2378 The root should be normcase-ed, too.
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2379 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2380
23097 30124c40d11f util.fspath: use a dict rather than a linear scan for lookups Siddharth Agarwal <sid0@fb.com> parents: 23076 diff changeset	2381 def _makefspathcacheentry(dir):
44470 9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things Augie Fackler <augie@google.com> parents: 44456 diff changeset	2382 return {normcase(n): n for n in os.listdir(dir)}
15709 a1f4bd47d18e icasefs: retry directory scan once for already invalidated cache FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15674 diff changeset	2383
30618 1112ff99d965 py3: replace os.sep with pycompat.ossep (part 1 of 4) Pulkit Goyal <7895pulkit@gmail.com> parents: 30545 diff changeset	2384 seps = pycompat.ossep
30630 bcf4a975f93d py3: replace os.altsep with pycompat.altsep Pulkit Goyal <7895pulkit@gmail.com> parents: 30618 diff changeset	2385 if pycompat.osaltsep:
bcf4a975f93d py3: replace os.altsep with pycompat.altsep Pulkit Goyal <7895pulkit@gmail.com> parents: 30618 diff changeset	2386 seps = seps + pycompat.osaltsep
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2387 # Protect backslashes. This gets silly very quickly.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2388 seps.replace(b'\\', b'\\\\')
31503 670166e0fcaa util: use bytes re on bytes input in fspath Augie Fackler <augie@google.com> parents: 31502 diff changeset	2389 pattern = remod.compile(br'([^%s]+)\|([%s]+)' % (seps, seps))
15669 390bcd01775a icasefs: use util.normcase() instead of lower() or os.path.normcase in fspath FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15667 diff changeset	2390 dir = os.path.normpath(root)
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2391 result = []
33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2392 for part, sep in pattern.findall(name):
33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2393 if sep:
33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2394 result.append(sep)
33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2395 continue
33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2396
15719 1dd60426b061 icasefs: follow standard cache look up pattern FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15718 diff changeset	2397 if dir not in _fspathcache:
23097 30124c40d11f util.fspath: use a dict rather than a linear scan for lookups Siddharth Agarwal <sid0@fb.com> parents: 23076 diff changeset	2398 _fspathcache[dir] = _makefspathcacheentry(dir)
15719 1dd60426b061 icasefs: follow standard cache look up pattern FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15718 diff changeset	2399 contents = _fspathcache[dir]
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2400
23097 30124c40d11f util.fspath: use a dict rather than a linear scan for lookups Siddharth Agarwal <sid0@fb.com> parents: 23076 diff changeset	2401 found = contents.get(part)
15709 a1f4bd47d18e icasefs: retry directory scan once for already invalidated cache FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15674 diff changeset	2402 if not found:
15720 3bcfea777efc icasefs: rewrite comment to explain situtation precisely FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15719 diff changeset	2403 # retry "once per directory" per "dirstate.walk" which
3bcfea777efc icasefs: rewrite comment to explain situtation precisely FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15719 diff changeset	2404 # may take place for each patches of "hg qpush", for example
23097 30124c40d11f util.fspath: use a dict rather than a linear scan for lookups Siddharth Agarwal <sid0@fb.com> parents: 23076 diff changeset	2405 _fspathcache[dir] = contents = _makefspathcacheentry(dir)
30124c40d11f util.fspath: use a dict rather than a linear scan for lookups Siddharth Agarwal <sid0@fb.com> parents: 23076 diff changeset	2406 found = contents.get(part)
15709 a1f4bd47d18e icasefs: retry directory scan once for already invalidated cache FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15674 diff changeset	2407
a1f4bd47d18e icasefs: retry directory scan once for already invalidated cache FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15674 diff changeset	2408 result.append(found or part)
15669 390bcd01775a icasefs: use util.normcase() instead of lower() or os.path.normcase in fspath FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15667 diff changeset	2409 dir = os.path.join(dir, part)
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2410
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2411 return b''.join(result)
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2412
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2413
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	2414 def checknlink(testfile: bytes) -> bool:
12938 bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2415 '''check whether hardlink count reporting works properly'''
bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2416
13204 5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2417 # testfile may be open, so we need a separate file for checking to
5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2418 # work around issue2543 (or testfile may get lost on Samba shares)
34101 6c5cdb02f2f9 checknlink: rename file object from 'fd' to 'fp' Jun Wu <quark@fb.com> parents: 34096 diff changeset	2419 f1, f2, fp = None, None, None
12938 bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2420 try:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2421 fd, f1 = pycompat.mkstemp(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2422 prefix=b'.%s-' % os.path.basename(testfile),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2423 suffix=b'1~',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2424 dir=os.path.dirname(testfile),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2425 )
34096 1104718fb090 checknlink: use a random temp file name for checking Jun Wu <quark@fb.com> parents: 34089 diff changeset	2426 os.close(fd)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2427 f2 = b'%s2~' % f1[:-2]
34096 1104718fb090 checknlink: use a random temp file name for checking Jun Wu <quark@fb.com> parents: 34089 diff changeset	2428
25088 754df8e932d3 util: use try/except/finally Matt Mackall <mpm@selenic.com> parents: 24902 diff changeset	2429 oslink(f1, f2)
12938 bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2430 # nlinks() may behave differently for files on Windows shares if
bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2431 # the file is open.
34101 6c5cdb02f2f9 checknlink: rename file object from 'fd' to 'fp' Jun Wu <quark@fb.com> parents: 34096 diff changeset	2432 fp = posixfile(f2)
13204 5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2433 return nlinks(f2) > 1
25088 754df8e932d3 util: use try/except/finally Matt Mackall <mpm@selenic.com> parents: 24902 diff changeset	2434 except OSError:
754df8e932d3 util: use try/except/finally Matt Mackall <mpm@selenic.com> parents: 24902 diff changeset	2435 return False
12938 bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2436 finally:
34101 6c5cdb02f2f9 checknlink: rename file object from 'fd' to 'fp' Jun Wu <quark@fb.com> parents: 34096 diff changeset	2437 if fp is not None:
6c5cdb02f2f9 checknlink: rename file object from 'fd' to 'fp' Jun Wu <quark@fb.com> parents: 34096 diff changeset	2438 fp.close()
13204 5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2439 for f in (f1, f2):
5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2440 try:
34096 1104718fb090 checknlink: use a random temp file name for checking Jun Wu <quark@fb.com> parents: 34089 diff changeset	2441 if f is not None:
1104718fb090 checknlink: use a random temp file name for checking Jun Wu <quark@fb.com> parents: 34089 diff changeset	2442 os.unlink(f)
13204 5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2443 except OSError:
5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2444 pass
12938 bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2445
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2446
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	2447 def endswithsep(path: bytes) -> bool:
5843 83c354c4d529 Add endswithsep() and use it instead of using os.sep and os.altsep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5802 diff changeset	2448 '''Check path ends with os.sep or os.altsep.'''
46684 7711853110b9 typing: add some type annotations to mercurial/util.py Matt Harbison <matt_harbison@yahoo.com> parents: 46114 diff changeset	2449 return bool( # help pytype
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2450 path.endswith(pycompat.ossep)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2451 or pycompat.osaltsep
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2452 and path.endswith(pycompat.osaltsep)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2453 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2454
5843 83c354c4d529 Add endswithsep() and use it instead of using os.sep and os.altsep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5802 diff changeset	2455
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	2456 def splitpath(path: bytes) -> List[bytes]:
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2457 """Split path by os.sep.
5844 07d8eb78dd68 Add util.splitpath() and use it instead of using os.sep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5843 diff changeset	2458 Note that this function does not use os.altsep because this is
07d8eb78dd68 Add util.splitpath() and use it instead of using os.sep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5843 diff changeset	2459 an alternative of simple "xxx.split(os.sep)".
07d8eb78dd68 Add util.splitpath() and use it instead of using os.sep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5843 diff changeset	2460 It is recommended to use os.path.normpath() before using this
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2461 function if need."""
30618 1112ff99d965 py3: replace os.sep with pycompat.ossep (part 1 of 4) Pulkit Goyal <7895pulkit@gmail.com> parents: 30545 diff changeset	2462 return path.split(pycompat.ossep)
5844 07d8eb78dd68 Add util.splitpath() and use it instead of using os.sep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5843 diff changeset	2463
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2464
51843 62806be5cbda typing: add hints to `mercurial.util.mktempcopy()` Matt Harbison <matt_harbison@yahoo.com> parents: 51800 diff changeset	2465 def mktempcopy(
62806be5cbda typing: add hints to `mercurial.util.mktempcopy()` Matt Harbison <matt_harbison@yahoo.com> parents: 51800 diff changeset	2466 name: bytes,
62806be5cbda typing: add hints to `mercurial.util.mktempcopy()` Matt Harbison <matt_harbison@yahoo.com> parents: 51800 diff changeset	2467 emptyok: bool = False,
62806be5cbda typing: add hints to `mercurial.util.mktempcopy()` Matt Harbison <matt_harbison@yahoo.com> parents: 51800 diff changeset	2468 createmode: Optional[int] = None,
62806be5cbda typing: add hints to `mercurial.util.mktempcopy()` Matt Harbison <matt_harbison@yahoo.com> parents: 51800 diff changeset	2469 enforcewritable: bool = False,
62806be5cbda typing: add hints to `mercurial.util.mktempcopy()` Matt Harbison <matt_harbison@yahoo.com> parents: 51800 diff changeset	2470 ) -> bytes:
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2471 """Create a temporary file with the same contents from name
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2472
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2473 The permission bits are copied from the original file.
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2474
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2475 If the temporary file is going to be truncated immediately, you
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2476 can use emptyok=True as an optimization.
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2477
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2478 Returns the name of the temporary file.
2176 9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2479 """
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2480 d, fn = os.path.split(name)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2481 fd, temp = pycompat.mkstemp(prefix=b'.%s-' % fn, suffix=b'~', dir=d)
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2482 os.close(fd)
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2483 # Temporary files are created with mode 0600, which is usually not
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2484 # what we want. If the original file already exists, just copy
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2485 # its mode. Otherwise, manually obey umask.
41289 593f6359681d update: fix edge-case with update.atomic-file and read-only files Boris Feld <boris.feld@octobus.net> parents: 40898 diff changeset	2486 copymode(name, temp, createmode, enforcewritable)
593f6359681d update: fix edge-case with update.atomic-file and read-only files Boris Feld <boris.feld@octobus.net> parents: 40898 diff changeset	2487
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2488 if emptyok:
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2489 return temp
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2490 try:
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2491 try:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2492 ifp = posixfile(name, b"rb")
52665 24ee91ba9aa8 pyupgrade: drop usage of py3 aliases for `OSError` Matt Harbison <matt_harbison@yahoo.com> parents: 52392 diff changeset	2493 except OSError as inst:
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2494 if inst.errno == errno.ENOENT:
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2495 return temp
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2496 if not getattr(inst, 'filename', None):
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2497 inst.filename = name
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2498 raise
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2499 ofp = posixfile(temp, b"wb")
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2500 for chunk in filechunkiter(ifp):
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2501 ofp.write(chunk)
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2502 ifp.close()
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2503 ofp.close()
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2504 except: # re-raises
34436 5326e4ef1dab style: never put multiple statements on one line Alex Gaynor <agaynor@mozilla.com> parents: 34363 diff changeset	2505 try:
5326e4ef1dab style: never put multiple statements on one line Alex Gaynor <agaynor@mozilla.com> parents: 34363 diff changeset	2506 os.unlink(temp)
5326e4ef1dab style: never put multiple statements on one line Alex Gaynor <agaynor@mozilla.com> parents: 34363 diff changeset	2507 except OSError:
5326e4ef1dab style: never put multiple statements on one line Alex Gaynor <agaynor@mozilla.com> parents: 34363 diff changeset	2508 pass
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2509 raise
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2510 return temp
2176 9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2511
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2512
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49031 diff changeset	2513 class filestat:
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2514 """help to exactly detect change of a file
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2515
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2516 'stat' attribute is result of 'os.stat()' if specified 'path'
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2517 exists. Otherwise, it is None. This can avoid preparative
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2518 'exists()' examination on client side of this class.
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2519 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2520
51924 1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51917 diff changeset	2521 def __init__(self, stat: Optional[os.stat_result]) -> None:
32792 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	2522 self.stat = stat
7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	2523
7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	2524 @classmethod
51924 1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51917 diff changeset	2525 def frompath(cls: Type[_Tfilestat], path: bytes) -> _Tfilestat:
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2526 try:
32792 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	2527 stat = os.stat(path)
49314 2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT Manuel Jacob <me@manueljacob.de> parents: 49286 diff changeset	2528 except FileNotFoundError:
32792 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	2529 stat = None
7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	2530 return cls(stat)
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2531
32835 1b25c648d5b7 fsmonitor: don't write out state if identity has changed (issue5581) Siddharth Agarwal <sid0@fb.com> parents: 32792 diff changeset	2532 @classmethod
51924 1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51917 diff changeset	2533 def fromfp(cls: Type[_Tfilestat], fp: BinaryIO) -> _Tfilestat:
32835 1b25c648d5b7 fsmonitor: don't write out state if identity has changed (issue5581) Siddharth Agarwal <sid0@fb.com> parents: 32792 diff changeset	2534 stat = os.fstat(fp.fileno())
1b25c648d5b7 fsmonitor: don't write out state if identity has changed (issue5581) Siddharth Agarwal <sid0@fb.com> parents: 32792 diff changeset	2535 return cls(stat)
1b25c648d5b7 fsmonitor: don't write out state if identity has changed (issue5581) Siddharth Agarwal <sid0@fb.com> parents: 32792 diff changeset	2536
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2537 __hash__ = object.__hash__
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2538
51924 1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51917 diff changeset	2539 def __eq__(self, old) -> bool:
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2540 try:
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2541 # if ambiguity between stat of new and old file is
30342 318a24b52eeb spelling: fixes of non-dictionary words Mads Kiilerich <madski@unity3d.com> parents: 30339 diff changeset	2542 # avoided, comparison of size, ctime and mtime is enough
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2543 # to exactly detect change of a file regardless of platform
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2544 return (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2545 self.stat.st_size == old.stat.st_size
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2546 and self.stat[stat.ST_CTIME] == old.stat[stat.ST_CTIME]
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2547 and self.stat[stat.ST_MTIME] == old.stat[stat.ST_MTIME]
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2548 )
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2549 except AttributeError:
32769 b5524fd9a4e3 util: make filestat.__eq__ return True if both of self and old have None stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32766 diff changeset	2550 pass
b5524fd9a4e3 util: make filestat.__eq__ return True if both of self and old have None stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32766 diff changeset	2551 try:
b5524fd9a4e3 util: make filestat.__eq__ return True if both of self and old have None stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32766 diff changeset	2552 return self.stat is None and old.stat is None
b5524fd9a4e3 util: make filestat.__eq__ return True if both of self and old have None stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32766 diff changeset	2553 except AttributeError:
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2554 return False
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2555
51924 1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51917 diff changeset	2556 def isambig(self, old: _Tfilestat) -> bool:
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2557 """Examine whether new (= self) stat is ambiguous against old one
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2558
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2559 "S[N]" below means stat of a file at N-th change:
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2560
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2561 - S[n-1].ctime < S[n].ctime: can detect change of a file
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2562 - S[n-1].ctime == S[n].ctime
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2563 - S[n-1].ctime < S[n].mtime: means natural advancing (*1)
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2564 - S[n-1].ctime == S[n].mtime: is ambiguous (*2)
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2565 - S[n-1].ctime > S[n].mtime: never occurs naturally (don't care)
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2566 - S[n-1].ctime > S[n].ctime: never occurs naturally (don't care)
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2567
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2568 Case (*2) above means that a file was changed twice or more at
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2569 same time in sec (= S[n-1].ctime), and comparison of timestamp
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2570 is ambiguous.
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2571
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2572 Base idea to avoid such ambiguity is "advance mtime 1 sec, if
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2573 timestamp is ambiguous".
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2574
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2575 But advancing mtime only in case (*2) doesn't work as
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2576 expected, because naturally advanced S[n].mtime in case (*1)
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2577 might be equal to manually advanced S[n-1 or earlier].mtime.
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2578
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2579 Therefore, all "S[n-1].ctime == S[n].ctime" cases should be
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2580 treated as ambiguous regardless of mtime, to avoid overlooking
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2581 by confliction between such mtime.
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2582
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2583 Advancing mtime "if isambig(oldstat)" ensures "S[n-1].mtime !=
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2584 S[n].mtime", even if size of a file isn't changed.
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2585 """
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2586 try:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2587 return self.stat[stat.ST_CTIME] == old.stat[stat.ST_CTIME]
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2588 except AttributeError:
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2589 return False
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2590
51924 1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51917 diff changeset	2591 def avoidambig(self, path: bytes, old: _Tfilestat) -> bool:
30253 b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2592 """Change file stat of specified path to avoid ambiguity
b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2593
b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2594 'old' should be previous filestat of 'path'.
b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2595
b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2596 This skips avoiding ambiguity, if a process doesn't have
32766 77f354ae1123 util: make filestat.avoidambig() return whether ambiguity is avoided or not FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32674 diff changeset	2597 appropriate privileges for 'path'. This returns False in this
77f354ae1123 util: make filestat.avoidambig() return whether ambiguity is avoided or not FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32674 diff changeset	2598 case.
77f354ae1123 util: make filestat.avoidambig() return whether ambiguity is avoided or not FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32674 diff changeset	2599
77f354ae1123 util: make filestat.avoidambig() return whether ambiguity is avoided or not FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32674 diff changeset	2600 Otherwise, this returns True, as "ambiguity is avoided".
30253 b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2601 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2602 advanced = (old.stat[stat.ST_MTIME] + 1) & 0x7FFFFFFF
30253 b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2603 try:
b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2604 os.utime(path, (advanced, advanced))
49316 d2adebe35635 py3: catch PermissionError instead of checking errno == EPERM Manuel Jacob <me@manueljacob.de> parents: 49314 diff changeset	2605 except PermissionError:
d2adebe35635 py3: catch PermissionError instead of checking errno == EPERM Manuel Jacob <me@manueljacob.de> parents: 49314 diff changeset	2606 # utime() on the file created by another user causes EPERM,
d2adebe35635 py3: catch PermissionError instead of checking errno == EPERM Manuel Jacob <me@manueljacob.de> parents: 49314 diff changeset	2607 # if a process doesn't have appropriate privileges
d2adebe35635 py3: catch PermissionError instead of checking errno == EPERM Manuel Jacob <me@manueljacob.de> parents: 49314 diff changeset	2608 return False
32766 77f354ae1123 util: make filestat.avoidambig() return whether ambiguity is avoided or not FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32674 diff changeset	2609 return True
30253 b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2610
51924 1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51917 diff changeset	2611 def __ne__(self, other) -> bool:
29298 82f6193ff2bc util: add __ne__ to filestat class for consistency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29261 diff changeset	2612 return not self == other
82f6193ff2bc util: add __ne__ to filestat class for consistency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29261 diff changeset	2613
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2614
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49031 diff changeset	2615 class atomictempfile:
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2616 """writable file object that atomically updates a file
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2617
14008 da65edcac72a atomictempfile: rewrite docstring to clarify rename() vs. close(). Greg Ward <greg@gerg.ca> parents: 14007 diff changeset	2618 All writes will go to a temporary copy of the original file. Call
15057 774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2619 close() when you are done writing, and atomictempfile will rename
774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2620 the temporary copy to the original name, making the changes
774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2621 visible. If the object is destroyed without being closed, all your
774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2622 writes are discarded.
29367 4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	2623
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	2624 checkambig argument of constructor is used with filestat, and is
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	2625 useful only if target file is guarded by any lock (e.g. repo.lock
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	2626 or repo.wlock).
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2627 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2628
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2629 def __init__(self, name, mode=b'w+b', createmode=None, checkambig=False):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2630 self.__name = name # permanent name
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2631 self._tempname = mktempcopy(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2632 name,
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2633 emptyok=(b'w' in mode),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2634 createmode=createmode,
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2635 enforcewritable=(b'w' in mode),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2636 )
41289 593f6359681d update: fix edge-case with update.atomic-file and read-only files Boris Feld <boris.feld@octobus.net> parents: 40898 diff changeset	2637
14007 d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2638 self._fp = posixfile(self._tempname, mode)
29201 a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2639 self._checkambig = checkambig
8327 aa25be1c2889 atomictempfile: delegate to posixfile instead of inheriting from it Bryan O'Sullivan <bos@serpentine.com> parents: 8312 diff changeset	2640
14007 d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2641 # delegated methods
29393 50269a4dce61 atomictempfile: add read to the supported file operations Martijn Pieters <mjpieters@fb.com> parents: 29367 diff changeset	2642 self.read = self._fp.read
14007 d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2643 self.write = self._fp.write
49721 c4f07a011714 util: implement `writelines()` on atomictempfile Matt Harbison <matt_harbison@yahoo.com> parents: 49316 diff changeset	2644 self.writelines = self._fp.writelines
17237 e73128535105 util: delegate seek and tell methods of atomictempfile Bryan O'Sullivan <bryano@fb.com> parents: 17203 diff changeset	2645 self.seek = self._fp.seek
e73128535105 util: delegate seek and tell methods of atomictempfile Bryan O'Sullivan <bryano@fb.com> parents: 17203 diff changeset	2646 self.tell = self._fp.tell
14007 d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2647 self.fileno = self._fp.fileno
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2648
15057 774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2649 def close(self):
8785 7a9151bc5b37 atomictempfile: fix exception in __del__ if mktempcopy fails (self._fp is None) Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8778 diff changeset	2650 if not self._fp.closed:
8327 aa25be1c2889 atomictempfile: delegate to posixfile instead of inheriting from it Bryan O'Sullivan <bos@serpentine.com> parents: 8312 diff changeset	2651 self._fp.close()
29201 a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2652 filename = localpath(self.__name)
32792 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	2653 oldstat = self._checkambig and filestat.frompath(filename)
29201 a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2654 if oldstat and oldstat.stat:
a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2655 rename(self._tempname, filename)
32792 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	2656 newstat = filestat.frompath(filename)
29201 a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2657 if newstat.isambig(oldstat):
a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2658 # stat of changed file is ambiguous to original one
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2659 advanced = (oldstat.stat[stat.ST_MTIME] + 1) & 0x7FFFFFFF
29201 a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2660 os.utime(filename, (advanced, advanced))
a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2661 else:
a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2662 rename(self._tempname, filename)
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2663
15057 774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2664 def discard(self):
8785 7a9151bc5b37 atomictempfile: fix exception in __del__ if mktempcopy fails (self._fp is None) Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8778 diff changeset	2665 if not self._fp.closed:
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2666 try:
14007 d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2667 os.unlink(self._tempname)
d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2668 except OSError:
d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2669 pass
8327 aa25be1c2889 atomictempfile: delegate to posixfile instead of inheriting from it Bryan O'Sullivan <bos@serpentine.com> parents: 8312 diff changeset	2670 self._fp.close()
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2671
13098 f7d6750dcd01 util: make atomicfiles closable Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 13072 diff changeset	2672 def __del__(self):
50951 d718eddf01d9 safehasattr: drop usage in favor of hasattr Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50749 diff changeset	2673 if hasattr(self, '_fp'): # constructor actually did something
15057 774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2674 self.discard()
13098 f7d6750dcd01 util: make atomicfiles closable Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 13072 diff changeset	2675
29394 6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2676 def __enter__(self):
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2677 return self
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2678
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2679 def __exit__(self, exctype, excvalue, traceback):
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2680 if exctype is not None:
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2681 self.discard()
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2682 else:
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2683 self.close()
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2684
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2685
49222 4ff4e23de7df clone: use better names for temp files Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 49070 diff changeset	2686 def tryrmdir(f):
4ff4e23de7df clone: use better names for temp files Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 49070 diff changeset	2687 try:
4ff4e23de7df clone: use better names for temp files Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 49070 diff changeset	2688 removedirs(f)
4ff4e23de7df clone: use better names for temp files Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 49070 diff changeset	2689 except OSError as e:
4ff4e23de7df clone: use better names for temp files Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 49070 diff changeset	2690 if e.errno != errno.ENOENT and e.errno != errno.ENOTEMPTY:
4ff4e23de7df clone: use better names for temp files Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 49070 diff changeset	2691 raise
4ff4e23de7df clone: use better names for temp files Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 49070 diff changeset	2692
4ff4e23de7df clone: use better names for temp files Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 49070 diff changeset	2693
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	2694 def unlinkpath(
f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	2695 f: bytes, ignoremissing: bool = False, rmdir: bool = True
f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	2696 ) -> None:
31545 52361c4f4dac util: unify unlinkpath Ryan McElroy <rmcelroy@fb.com> parents: 31539 diff changeset	2697 """unlink and remove the directory if it is empty"""
31547 bd9daafbf87c util: use tryunlink in unlinkpath Ryan McElroy <rmcelroy@fb.com> parents: 31546 diff changeset	2698 if ignoremissing:
bd9daafbf87c util: use tryunlink in unlinkpath Ryan McElroy <rmcelroy@fb.com> parents: 31546 diff changeset	2699 tryunlink(f)
bd9daafbf87c util: use tryunlink in unlinkpath Ryan McElroy <rmcelroy@fb.com> parents: 31546 diff changeset	2700 else:
31545 52361c4f4dac util: unify unlinkpath Ryan McElroy <rmcelroy@fb.com> parents: 31539 diff changeset	2701 unlink(f)
38497 da2a7d8354b2 unlinkpath: make empty directory removal optional (issue5901) (issue5826) Kyle Lippincott <spectral@google.com> parents: 38386 diff changeset	2702 if rmdir:
da2a7d8354b2 unlinkpath: make empty directory removal optional (issue5901) (issue5826) Kyle Lippincott <spectral@google.com> parents: 38386 diff changeset	2703 # try removing directories that might now be empty
da2a7d8354b2 unlinkpath: make empty directory removal optional (issue5901) (issue5826) Kyle Lippincott <spectral@google.com> parents: 38386 diff changeset	2704 try:
da2a7d8354b2 unlinkpath: make empty directory removal optional (issue5901) (issue5826) Kyle Lippincott <spectral@google.com> parents: 38386 diff changeset	2705 removedirs(os.path.dirname(f))
da2a7d8354b2 unlinkpath: make empty directory removal optional (issue5901) (issue5826) Kyle Lippincott <spectral@google.com> parents: 38386 diff changeset	2706 except OSError:
da2a7d8354b2 unlinkpath: make empty directory removal optional (issue5901) (issue5826) Kyle Lippincott <spectral@google.com> parents: 38386 diff changeset	2707 pass
31545 52361c4f4dac util: unify unlinkpath Ryan McElroy <rmcelroy@fb.com> parents: 31539 diff changeset	2708
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2709
51428 187c5769a629 vfs: have tryunlink tell what it did Georges Racinet <georges.racinet@octobus.net> parents: 51315 diff changeset	2710 def tryunlink(f: bytes) -> bool:
187c5769a629 vfs: have tryunlink tell what it did Georges Racinet <georges.racinet@octobus.net> parents: 51315 diff changeset	2711 """Attempt to remove a file, ignoring FileNotFoundError.
187c5769a629 vfs: have tryunlink tell what it did Georges Racinet <georges.racinet@octobus.net> parents: 51315 diff changeset	2712
187c5769a629 vfs: have tryunlink tell what it did Georges Racinet <georges.racinet@octobus.net> parents: 51315 diff changeset	2713 Returns False in case the file did not exit, True otherwise
187c5769a629 vfs: have tryunlink tell what it did Georges Racinet <georges.racinet@octobus.net> parents: 51315 diff changeset	2714 """
31546 6d5b77abf306 util: add tryunlink function Ryan McElroy <rmcelroy@fb.com> parents: 31545 diff changeset	2715 try:
6d5b77abf306 util: add tryunlink function Ryan McElroy <rmcelroy@fb.com> parents: 31545 diff changeset	2716 unlink(f)
51428 187c5769a629 vfs: have tryunlink tell what it did Georges Racinet <georges.racinet@octobus.net> parents: 51315 diff changeset	2717 return True
49314 2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT Manuel Jacob <me@manueljacob.de> parents: 49286 diff changeset	2718 except FileNotFoundError:
51428 187c5769a629 vfs: have tryunlink tell what it did Georges Racinet <georges.racinet@octobus.net> parents: 51315 diff changeset	2719 return False
31546 6d5b77abf306 util: add tryunlink function Ryan McElroy <rmcelroy@fb.com> parents: 31545 diff changeset	2720
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2721
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	2722 def makedirs(
f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	2723 name: bytes, mode: Optional[int] = None, notindexed: bool = False
f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	2724 ) -> None:
29017 07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2725 """recursive directory creation with parent mode inheritance
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2726
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2727 Newly created directories are marked as "not to be indexed by
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2728 the content indexing service", if ``notindexed`` is specified
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2729 for "write" mode access.
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2730 """
6062 3c3b126e5619 Make files in .hg inherit the permissions from .hg/store Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 6007 diff changeset	2731 try:
18938 e22107cff6bf util: add notindexed optional parameter to makedirs function Angel Ezquerra <angel.ezquerra@gmail.com> parents: 18930 diff changeset	2732 makedir(name, notindexed)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25420 diff changeset	2733 except OSError as err:
6062 3c3b126e5619 Make files in .hg inherit the permissions from .hg/store Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 6007 diff changeset	2734 if err.errno == errno.EEXIST:
3c3b126e5619 Make files in .hg inherit the permissions from .hg/store Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 6007 diff changeset	2735 return
15058 81f33be0ea79 util: postpone and reorder parent calculation in makedirs Adrian Buehlmann <adrian@cadifra.com> parents: 15057 diff changeset	2736 if err.errno != errno.ENOENT or not name:
81f33be0ea79 util: postpone and reorder parent calculation in makedirs Adrian Buehlmann <adrian@cadifra.com> parents: 15057 diff changeset	2737 raise
47622 bb917eea1605 windows: introduce a `util.abspath` to replace os.path.abspath Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47452 diff changeset	2738 parent = os.path.dirname(abspath(name))
15058 81f33be0ea79 util: postpone and reorder parent calculation in makedirs Adrian Buehlmann <adrian@cadifra.com> parents: 15057 diff changeset	2739 if parent == name:
6062 3c3b126e5619 Make files in .hg inherit the permissions from .hg/store Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 6007 diff changeset	2740 raise
18938 e22107cff6bf util: add notindexed optional parameter to makedirs function Angel Ezquerra <angel.ezquerra@gmail.com> parents: 18930 diff changeset	2741 makedirs(parent, mode, notindexed)
29017 07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2742 try:
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2743 makedir(name, notindexed)
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2744 except OSError as err:
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2745 # Catch EEXIST to handle races
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2746 if err.errno == errno.EEXIST:
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2747 return
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2748 raise
18678 423eee0b0b14 util: make ensuredirs safer against races Bryan O'Sullivan <bryano@fb.com> parents: 18668 diff changeset	2749 if mode is not None:
423eee0b0b14 util: make ensuredirs safer against races Bryan O'Sullivan <bryano@fb.com> parents: 18668 diff changeset	2750 os.chmod(name, mode)
18668 4034b8d551b1 scmutil: create directories in a race-safe way during update Bryan O'Sullivan <bryano@fb.com> parents: 18614 diff changeset	2751
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2752
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	2753 def readfile(path: bytes) -> bytes:
52392 fca7d38e040b util: stop using the `pycompat.open()` shim Matt Harbison <matt_harbison@yahoo.com> parents: 52116 diff changeset	2754 with open(path, 'rb') as fp:
14100 3e9e02a41dfb util: really drop size from readfile Matt Mackall <mpm@selenic.com> parents: 14099 diff changeset	2755 return fp.read()
14099 0824a0a3cefc util: add readfile() & writefile() helper functions Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 14077 diff changeset	2756
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2757
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	2758 def writefile(path: bytes, text: bytes) -> None:
52392 fca7d38e040b util: stop using the `pycompat.open()` shim Matt Harbison <matt_harbison@yahoo.com> parents: 52116 diff changeset	2759 with open(path, 'wb') as fp:
14167 0e4753807c93 util & scmutil: adapt read/write helpers as request by mpm Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 14147 diff changeset	2760 fp.write(text)
0e4753807c93 util & scmutil: adapt read/write helpers as request by mpm Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 14147 diff changeset	2761
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2762
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	2763 def appendfile(path: bytes, text: bytes) -> None:
52392 fca7d38e040b util: stop using the `pycompat.open()` shim Matt Harbison <matt_harbison@yahoo.com> parents: 52116 diff changeset	2764 with open(path, 'ab') as fp:
14099 0824a0a3cefc util: add readfile() & writefile() helper functions Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 14077 diff changeset	2765 fp.write(text)
0824a0a3cefc util: add readfile() & writefile() helper functions Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 14077 diff changeset	2766
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2767
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49031 diff changeset	2768 class chunkbuffer:
1199 78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an Eric Hopper <hopper@omnifarious.org> parents: 1169 diff changeset	2769 """Allow arbitrary sized chunks of data to be efficiently read from an
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an Eric Hopper <hopper@omnifarious.org> parents: 1169 diff changeset	2770 iterator over chunks of arbitrary size."""
1200 333de1d53846 Minor cleanups. Bryan O'Sullivan <bos@serpentine.com> parents: 1199 diff changeset	2771
5446 fa836e050c50 chunkbuffer: removed unused method and arg Matt Mackall <mpm@selenic.com> parents: 5420 diff changeset	2772 def __init__(self, in_iter):
32176 5f53c267e362 util: remove doc of long gone 'targetsize' argument Martin von Zweigbergk <martinvonz@google.com> parents: 31952 diff changeset	2773 """in_iter is the iterator that's iterating over the input chunks."""
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2774
11670 1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2775 def splitbig(chunks):
1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2776 for chunk in chunks:
51762 ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51761 diff changeset	2777 if len(chunk) > 2**20:
11670 1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2778 pos = 0
1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2779 while pos < len(chunk):
51762 ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51761 diff changeset	2780 end = pos + 2**18
11670 1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2781 yield chunk[pos:end]
1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2782 pos = end
1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2783 else:
1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2784 yield chunk
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2785
11670 1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2786 self.iter = splitbig(in_iter)
25113 0ca8410ea345 util: drop alias for collections.deque Martin von Zweigbergk <martinvonz@google.com> parents: 25112 diff changeset	2787 self._queue = collections.deque()
26480 6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2788 self._chunkoffset = 0
1200 333de1d53846 Minor cleanups. Bryan O'Sullivan <bos@serpentine.com> parents: 1199 diff changeset	2789
21018 c848bfd02366 util: support None size in chunkbuffer.read() Pierre-Yves David <pierre-yves.david@fb.com> parents: 20835 diff changeset	2790 def read(self, l=None):
1200 333de1d53846 Minor cleanups. Bryan O'Sullivan <bos@serpentine.com> parents: 1199 diff changeset	2791 """Read L bytes of data from the iterator of chunks of data.
21018 c848bfd02366 util: support None size in chunkbuffer.read() Pierre-Yves David <pierre-yves.david@fb.com> parents: 20835 diff changeset	2792 Returns less than L bytes if the iterator runs dry.
c848bfd02366 util: support None size in chunkbuffer.read() Pierre-Yves David <pierre-yves.david@fb.com> parents: 20835 diff changeset	2793
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23097 diff changeset	2794 If size parameter is omitted, read everything"""
26478 a3f7e5461dbd util.chunkbuffer: special case reading everything Gregory Szorc <gregory.szorc@gmail.com> parents: 26450 diff changeset	2795 if l is None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2796 return b''.join(self.iter)
26478 a3f7e5461dbd util.chunkbuffer: special case reading everything Gregory Szorc <gregory.szorc@gmail.com> parents: 26450 diff changeset	2797
11758 a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2798 left = l
17962 4c29668ca316 util: make chunkbuffer non-quadratic on Windows Matt Mackall <mpm@selenic.com> parents: 17560 diff changeset	2799 buf = []
16873 37e081609828 util: simplify queue management in chunkbuffer Bryan O'Sullivan <bryano@fb.com> parents: 16834 diff changeset	2800 queue = self._queue
26478 a3f7e5461dbd util.chunkbuffer: special case reading everything Gregory Szorc <gregory.szorc@gmail.com> parents: 26450 diff changeset	2801 while left > 0:
11758 a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2802 # refill the queue
a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2803 if not queue:
51762 ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51761 diff changeset	2804 target = 2**18
11758 a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2805 for chunk in self.iter:
a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2806 queue.append(chunk)
a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2807 target -= len(chunk)
a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2808 if target <= 0:
a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2809 break
a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2810 if not queue:
1199 78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an Eric Hopper <hopper@omnifarious.org> parents: 1169 diff changeset	2811 break
11758 a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2812
26480 6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2813 # The easy way to do this would be to queue.popleft(), modify the
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2814 # chunk (if necessary), then queue.appendleft(). However, for cases
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2815 # where we read partial chunk content, this incurs 2 dequeue
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2816 # mutations and creates a new str for the remaining chunk in the
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2817 # queue. Our code below avoids this overhead.
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2818
26479 46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2819 chunk = queue[0]
46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2820 chunkl = len(chunk)
26480 6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2821 offset = self._chunkoffset
26479 46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2822
46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2823 # Use full chunk.
26480 6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2824 if offset == 0 and left >= chunkl:
26479 46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2825 left -= chunkl
46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2826 queue.popleft()
46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2827 buf.append(chunk)
26480 6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2828 # self._chunkoffset remains at 0.
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2829 continue
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2830
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2831 chunkremaining = chunkl - offset
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2832
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2833 # Use all of unconsumed part of chunk.
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2834 if left >= chunkremaining:
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2835 left -= chunkremaining
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2836 queue.popleft()
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2837 # offset == 0 is enabled by block above, so this won't merely
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2838 # copy via ``chunk[0:]``.
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2839 buf.append(chunk[offset:])
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2840 self._chunkoffset = 0
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2841
26479 46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2842 # Partial chunk needed.
46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2843 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2844 buf.append(chunk[offset : offset + left])
26480 6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2845 self._chunkoffset += left
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2846 left -= chunkremaining
11758 a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2847
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2848 return b''.join(buf)
11758 a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2849
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2850
30181 7356e6b1f5b8 util: increase filechunkiter size to 128k Mads Kiilerich <madski@unity3d.com> parents: 30087 diff changeset	2851 def filechunkiter(f, size=131072, limit=None):
2462 d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2852 """Create a generator that produces the data in the file size
30181 7356e6b1f5b8 util: increase filechunkiter size to 128k Mads Kiilerich <madski@unity3d.com> parents: 30087 diff changeset	2853 (default 131072) bytes at a time, up to optional limit (default is
2462 d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2854 to read all data). Chunks may be less than size bytes if the
d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2855 chunk is the last chunk in the file, or the file is a socket or
d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2856 some other type of file that sometimes reads less data than is
d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2857 requested."""
d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2858 assert size >= 0
d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2859 assert limit is None or limit >= 0
d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2860 while True:
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2861 if limit is None:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2862 nbytes = size
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2863 else:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2864 nbytes = min(limit, size)
2462 d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2865 s = nbytes and f.read(nbytes)
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2866 if not s:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2867 break
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2868 if limit:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2869 limit -= len(s)
1199 78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an Eric Hopper <hopper@omnifarious.org> parents: 1169 diff changeset	2870 yield s
1320 5f277e73778f Fix up representation of dates in hgweb. Bryan O'Sullivan <bos@serpentine.com> parents: 1312 diff changeset	2871
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2872
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49031 diff changeset	2873 class cappedreader:
36395 01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2874 """A file object proxy that allows reading up to N bytes.
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2875
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2876 Given a source file object, instances of this type allow reading up to
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2877 N bytes from that source file object. Attempts to read past the allowed
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2878 limit are treated as EOF.
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2879
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2880 It is assumed that I/O is not performed on the original file object
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2881 in addition to I/O that is performed by this instance. If there is,
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2882 state tracking will get out of sync and unexpected results will ensue.
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2883 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2884
36395 01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2885 def __init__(self, fh, limit):
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2886 """Allow reading up to <limit> bytes from <fh>."""
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2887 self._fh = fh
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2888 self._left = limit
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2889
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2890 def read(self, n=-1):
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2891 if not self._left:
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2892 return b''
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2893
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2894 if n < 0:
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2895 n = self._left
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2896
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2897 data = self._fh.read(min(n, self._left))
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2898 self._left -= len(data)
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2899 assert self._left >= 0
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2900
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2901 return data
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2902
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37047 diff changeset	2903 def readinto(self, b):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37047 diff changeset	2904 res = self.read(len(b))
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37047 diff changeset	2905 if res is None:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37047 diff changeset	2906 return None
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37047 diff changeset	2907
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2908 b[0 : len(res)] = res
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37047 diff changeset	2909 return len(res)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37047 diff changeset	2910
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2911
18735 716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2912 def unitcountfn(*unittable):
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2913 '''return a function that renders a readable count of some quantity'''
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2914
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2915 def go(count):
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2916 for multiplier, divisor, format in unittable:
31946 f3b80537a70d util: fix human-readable printing of negative byte counts G?bor Stefanik <gabor.stefanik@nng.com> parents: 31934 diff changeset	2917 if abs(count) >= divisor * multiplier:
18735 716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2918 return format % (count / float(divisor))
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2919 return unittable[-1][2] % count
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2920
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2921 return go
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2922
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2923
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	2924 def processlinerange(fromline: int, toline: int) -> Tuple[int, int]:
31667 080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2925 """Check that linerange <fromline>:<toline> makes sense and return a
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2926 0-based range.
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2927
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2928 >>> processlinerange(10, 20)
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2929 (9, 20)
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2930 >>> processlinerange(2, 1)
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2931 Traceback (most recent call last):
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2932 ...
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2933 ParseError: line range must be positive
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2934 >>> processlinerange(0, 5)
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2935 Traceback (most recent call last):
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2936 ...
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2937 ParseError: fromline must be strictly positive
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2938 """
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2939 if toline - fromline < 0:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2940 raise error.ParseError(_(b"line range must be positive"))
31667 080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2941 if fromline < 1:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2942 raise error.ParseError(_(b"fromline must be strictly positive"))
31667 080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2943 return fromline - 1, toline
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2944
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2945
18735 716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2946 bytecount = unitcountfn(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2947 (100, 1 << 30, _(b'%.0f GB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2948 (10, 1 << 30, _(b'%.1f GB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2949 (1, 1 << 30, _(b'%.2f GB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2950 (100, 1 << 20, _(b'%.0f MB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2951 (10, 1 << 20, _(b'%.1f MB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2952 (1, 1 << 20, _(b'%.2f MB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2953 (100, 1 << 10, _(b'%.0f KB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2954 (10, 1 << 10, _(b'%.1f KB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2955 (1, 1 << 10, _(b'%.2f KB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2956 (1, 1, _(b'%.0f bytes')),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2957 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2958
16397 f0f7f3fab315 util: create bytecount array just once Matt Mackall <mpm@selenic.com> parents: 16383 diff changeset	2959
51780 ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	2960 class transformingwriter(typelib.BinaryIO_Proxy):
36845 472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2961 """Writable file wrapper to transform data by function"""
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2962
51780 ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	2963 def __init__(self, fp: BinaryIO, encode: Callable[[bytes], bytes]) -> None:
36845 472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2964 self._fp = fp
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2965 self._encode = encode
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2966
51780 ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	2967 def close(self) -> None:
36845 472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2968 self._fp.close()
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2969
51780 ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	2970 def flush(self) -> None:
36845 472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2971 self._fp.flush()
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2972
51780 ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	2973 def write(self, data: bytes) -> int:
36845 472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2974 return self._fp.write(self._encode(data))
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2975
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2976
31779 fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31725 diff changeset	2977 # Matches a single EOL which can either be a CRLF where repeated CR
fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31725 diff changeset	2978 # are removed or a LF. We do not care about old Macintosh files, so a
fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31725 diff changeset	2979 # stray CR is an error.
fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31725 diff changeset	2980 _eolre = remod.compile(br'\r*\n')
fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31725 diff changeset	2981
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2982
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	2983 def tolf(s: bytes) -> bytes:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2984 return _eolre.sub(b'\n', s)
31779 fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31725 diff changeset	2985
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2986
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	2987 def tocrlf(s: bytes) -> bytes:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2988 return _eolre.sub(b'\r\n', s)
31779 fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31725 diff changeset	2989
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2990
51780 ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	2991 def _crlfwriter(fp: typelib.BinaryIO_Proxy) -> typelib.BinaryIO_Proxy:
36845 472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2992 return transformingwriter(fp, tocrlf)
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2993
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2994
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2995 if pycompat.oslinesep == b'\r\n':
31780 6a5b69b0abec util: add helper to convert between LF and native EOL Yuya Nishihara <yuya@tcha.org> parents: 31779 diff changeset	2996 tonativeeol = tocrlf
6a5b69b0abec util: add helper to convert between LF and native EOL Yuya Nishihara <yuya@tcha.org> parents: 31779 diff changeset	2997 fromnativeeol = tolf
36845 472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2998 nativeeolwriter = _crlfwriter
31780 6a5b69b0abec util: add helper to convert between LF and native EOL Yuya Nishihara <yuya@tcha.org> parents: 31779 diff changeset	2999 else:
6a5b69b0abec util: add helper to convert between LF and native EOL Yuya Nishihara <yuya@tcha.org> parents: 31779 diff changeset	3000 tonativeeol = pycompat.identity
6a5b69b0abec util: add helper to convert between LF and native EOL Yuya Nishihara <yuya@tcha.org> parents: 31779 diff changeset	3001 fromnativeeol = pycompat.identity
36845 472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	3002 nativeeolwriter = pycompat.identity
31780 6a5b69b0abec util: add helper to convert between LF and native EOL Yuya Nishihara <yuya@tcha.org> parents: 31779 diff changeset	3003
51780 ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	3004 if typing.TYPE_CHECKING:
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	3005 # Replace the various overloads that come along with aliasing other methods
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	3006 # with the narrow definition that we care about in the type checking phase
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	3007 # only. This ensures that both Windows and POSIX see only the definition
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	3008 # that is actually available.
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	3009
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	3010 def tonativeeol(s: bytes) -> bytes:
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	3011 raise NotImplementedError
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	3012
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	3013 def fromnativeeol(s: bytes) -> bytes:
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	3014 raise NotImplementedError
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	3015
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	3016 def nativeeolwriter(fp: typelib.BinaryIO_Proxy) -> typelib.BinaryIO_Proxy:
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	3017 raise NotImplementedError
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	3018
49031 2974cdda819b util: remove iterfile() variant for buggy EINTR handling Gregory Szorc <gregory.szorc@gmail.com> parents: 49004 diff changeset	3019
2974cdda819b util: remove iterfile() variant for buggy EINTR handling Gregory Szorc <gregory.szorc@gmail.com> parents: 49004 diff changeset	3020 # TODO delete since workaround variant for Python 2 no longer needed.
2974cdda819b util: remove iterfile() variant for buggy EINTR handling Gregory Szorc <gregory.szorc@gmail.com> parents: 49004 diff changeset	3021 def iterfile(fp):
2974cdda819b util: remove iterfile() variant for buggy EINTR handling Gregory Szorc <gregory.szorc@gmail.com> parents: 49004 diff changeset	3022 return fp
30405 10514a92860e util: add iterfile to workaround a fileobj.__iter__ issue with EINTR Jun Wu <quark@fb.com> parents: 30369 diff changeset	3023
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3024
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	3025 def iterlines(iterator: Iterable[bytes]) -> Iterator[bytes]:
7879 5c4026a289a4 templater: ability to display diffstat for log-like commands Alexander Solovyov <piranha at piranha.org.ua> parents: 7875 diff changeset	3026 for chunk in iterator:
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52668 diff changeset	3027 yield from chunk.splitlines()
9610 d78fe60f6bda make path expanding more consistent Alexander Solovyov <piranha@piranha.org.ua> parents: 9569 diff changeset	3028
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3029
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	3030 def expandpath(path: bytes) -> bytes:
9610 d78fe60f6bda make path expanding more consistent Alexander Solovyov <piranha@piranha.org.ua> parents: 9569 diff changeset	3031 return os.path.expanduser(os.path.expandvars(path))
10239 8e4be44a676f Find right hg command for detached process Patrick Mezard <pmezard@gmail.com> parents: 10199 diff changeset	3032
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3033
13392 777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	3034 def interpolate(prefix, mapping, s, fn=None, escape_prefix=False):
11988 8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3035 """Return the result of interpolating items in the mapping into string s.
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3036
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3037 prefix is a single character string, or a two character string with
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3038 a backslash as the first character if the prefix needs to be escaped in
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3039 a regular expression.
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3040
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3041 fn is an optional function that will be applied to the replacement text
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3042 just before replacement.
13392 777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	3043
777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	3044 escape_prefix is an optional flag that allows using doubled prefix for
777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	3045 its escaping.
11988 8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3046 """
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3047 fn = fn or (lambda s: s)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3048 patterns = b'\|'.join(mapping.keys())
13392 777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	3049 if escape_prefix:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3050 patterns += b'\|' + prefix
13392 777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	3051 if len(prefix) > 1:
777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	3052 prefix_char = prefix[1:]
777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	3053 else:
777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	3054 prefix_char = prefix
777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	3055 mapping[prefix_char] = prefix_char
35166 25c543944bc0 py3: add b'' to regular expressions which are raw strings Pulkit Goyal <7895pulkit@gmail.com> parents: 35053 diff changeset	3056 r = remod.compile(br'%s(%s)' % (prefix, patterns))
11988 8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3057 return r.sub(lambda x: fn(mapping[x.group()[1:]]), s)
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3058
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3059
18736 af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3060 timecount = unitcountfn(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3061 (1, 1e3, _(b'%.0f s')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3062 (100, 1, _(b'%.1f s')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3063 (10, 1, _(b'%.2f s')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3064 (1, 1, _(b'%.3f s')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3065 (100, 0.001, _(b'%.1f ms')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3066 (10, 0.001, _(b'%.2f ms')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3067 (1, 0.001, _(b'%.3f ms')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3068 (100, 0.000001, _(b'%.1f us')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3069 (10, 0.000001, _(b'%.2f us')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3070 (1, 0.000001, _(b'%.3f us')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3071 (100, 0.000000001, _(b'%.1f ns')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3072 (10, 0.000000001, _(b'%.2f ns')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3073 (1, 0.000000001, _(b'%.3f ns')),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3074 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3075
18736 af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3076
38837 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3077 @attr.s
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49031 diff changeset	3078 class timedcmstats:
38837 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3079 """Stats information produced by the timedcm context manager on entering."""
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3080
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3081 # the starting value of the timer as a float (meaning and resulution is
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3082 # platform dependent, see util.timer)
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3083 start = attr.ib(default=attr.Factory(lambda: timer()))
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3084 # the number of seconds as a floating point value; starts at 0, updated when
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3085 # the context is exited.
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3086 elapsed = attr.ib(default=0)
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3087 # the number of nested timedcm context managers.
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3088 level = attr.ib(default=1)
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3089
38852 9d49bb117dde util: make new timedcmstats class Python 3 compatible Martijn Pieters <mj@zopatista.com> parents: 38837 diff changeset	3090 def __bytes__(self):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3091 return timecount(self.elapsed) if self.elapsed else b'<unknown>'
38837 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3092
38852 9d49bb117dde util: make new timedcmstats class Python 3 compatible Martijn Pieters <mj@zopatista.com> parents: 38837 diff changeset	3093 __str__ = encoding.strmethod(__bytes__)
9d49bb117dde util: make new timedcmstats class Python 3 compatible Martijn Pieters <mj@zopatista.com> parents: 38837 diff changeset	3094
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3095
38837 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3096 @contextlib.contextmanager
39287 e00123f63410 util: make timedcm require the label (API) Augie Fackler <augie@google.com> parents: 39286 diff changeset	3097 def timedcm(whencefmt, *whenceargs):
38837 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3098 """A context manager that produces timing information for a given context.
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3099
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3100 On entering a timedcmstats instance is produced.
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3101
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3102 This context manager is reentrant.
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3103
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3104 """
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3105 # track nested context managers
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3106 timedcm._nested += 1
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3107 timing_stats = timedcmstats(level=timedcm._nested)
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3108 try:
39285 497effb0a04a util: make timedcm context manager also emit trace events Augie Fackler <augie@google.com> parents: 39237 diff changeset	3109 with tracing.log(whencefmt, *whenceargs):
497effb0a04a util: make timedcm context manager also emit trace events Augie Fackler <augie@google.com> parents: 39237 diff changeset	3110 yield timing_stats
38837 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3111 finally:
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3112 timing_stats.elapsed = timer() - timing_stats.start
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3113 timedcm._nested -= 1
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3114
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3115
38837 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3116 timedcm._nested = 0
18736 af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3117
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3118
18736 af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3119 def timed(func):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	3120 """Report the execution time of a function call to stderr.
18736 af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3121
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3122 During development, use as a decorator when you need to measure
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3123 the cost of a function, e.g. as follows:
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3124
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3125 @util.timed
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3126 def foo(a, b, c):
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3127 pass
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	3128 """
18736 af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3129
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3130 def wrapper(args, *kwargs):
39286 331ab85e910b cleanup: make all uses of timedcm specify what they're timing Augie Fackler <augie@google.com> parents: 39285 diff changeset	3131 with timedcm(pycompat.bytestr(func.__name__)) as time_stats:
38837 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3132 result = func(args, *kwargs)
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3133 stderr = procutil.stderr
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3134 stderr.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3135 b'%s%s: %s\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3136 % (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3137 b' ' * time_stats.level * 2,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3138 pycompat.bytestr(func.__name__),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3139 time_stats,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3140 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3141 )
38837 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3142 return result
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3143
18736 af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3144 return wrapper
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3145
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3146
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3147 _sizeunits = (
51762 ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51761 diff changeset	3148 (b'm', 2**20),
ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51761 diff changeset	3149 (b'k', 2**10),
ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51761 diff changeset	3150 (b'g', 2**30),
ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51761 diff changeset	3151 (b'kb', 2**10),
ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51761 diff changeset	3152 (b'mb', 2**20),
ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51761 diff changeset	3153 (b'gb', 2**30),
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3154 (b'b', 1),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3155 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3156
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3157
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	3158 def sizetoint(s: bytes) -> int:
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	3159 """Convert a space specifier to a byte count.
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3160
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	3161 >>> sizetoint(b'30')
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3162 30
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	3163 >>> sizetoint(b'2.2kb')
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3164 2252
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	3165 >>> sizetoint(b'6M')
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3166 6291456
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	3167 """
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3168 t = s.strip().lower()
1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3169 try:
1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3170 for k, u in _sizeunits:
1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3171 if t.endswith(k):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3172 return int(float(t[: -len(k)]) * u)
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3173 return int(t)
1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3174 except ValueError:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3175 raise error.ParseError(_(b"couldn't parse size: %s") % s)
19211 3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3176
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3177
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49031 diff changeset	3178 class hooks:
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	3179 """A collection of hook functions that can be used to extend a
26098 ce26928cbe41 spelling: behaviour -> behavior timeless@mozdev.org parents: 25672 diff changeset	3180 function's behavior. Hooks are called in lexicographic order,
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	3181 based on the names of their sources."""
19211 3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3182
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3183 def __init__(self):
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3184 self._hooks = []
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3185
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3186 def add(self, source, hook):
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3187 self._hooks.append((source, hook))
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3188
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3189 def __call__(self, *args):
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3190 self._hooks.sort(key=lambda x: x[0])
21046 cc13addbd62b util: enable "hooks" to return list of the values returned from each hooks FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 21018 diff changeset	3191 results = []
19211 3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3192 for source, hook in self._hooks:
21046 cc13addbd62b util: enable "hooks" to return list of the values returned from each hooks FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 21018 diff changeset	3193 results.append(hook(*args))
cc13addbd62b util: enable "hooks" to return list of the values returned from each hooks FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 21018 diff changeset	3194 return results
20244 47d0843647d1 util: introduce util.debugstacktrace for showing a stack trace without crashing Mads Kiilerich <madski@unity3d.com> parents: 20202 diff changeset	3195
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3196
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3197 def getstackframes(skip=0, line=b' %-*s in %s\n', fileline=b'%s:%d', depth=0):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	3198 """Yields lines for a nicely formatted stacktrace.
31324 78ac7061f840 util: add debugstacktrace depth limit Mads Kiilerich <madski@unity3d.com> parents: 31323 diff changeset	3199 Skips the 'skip' last entries, then return the last 'depth' entries.
28497 906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3200 Each file+linenumber is formatted according to fileline.
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3201 Each line is formatted according to line.
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3202 If line is None, it yields:
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3203 length of longest filepath+line number,
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3204 filepath+linenumber,
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3205 function
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3206
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3207 Not be used in production code but very convenient while developing.
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	3208 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3209 entries = [
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3210 (fileline % (pycompat.sysbytes(fn), ln), pycompat.sysbytes(func))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3211 for fn, ln, func, _text in traceback.extract_stack()[: -skip - 1]
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3212 ][-depth:]
28497 906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3213 if entries:
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3214 fnmax = max(len(entry[0]) for entry in entries)
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3215 for fnln, func in entries:
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3216 if line is None:
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3217 yield (fnmax, fnln, func)
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3218 else:
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3219 yield line % (fnmax, fnln, func)
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3220
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3221
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3222 def debugstacktrace(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3223 msg=b'stacktrace',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3224 skip=0,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3225 f=procutil.stderr,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3226 otherf=procutil.stdout,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3227 depth=0,
43774 02ededbef627 util: add an optional `prefix` argument to debugstacktrace Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43767 diff changeset	3228 prefix=b'',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3229 ):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	3230 """Writes a message to f (stderr) with a nicely formatted stacktrace.
31324 78ac7061f840 util: add debugstacktrace depth limit Mads Kiilerich <madski@unity3d.com> parents: 31323 diff changeset	3231 Skips the 'skip' entries closest to the call, then show 'depth' entries.
78ac7061f840 util: add debugstacktrace depth limit Mads Kiilerich <madski@unity3d.com> parents: 31323 diff changeset	3232 By default it will flush stdout first.
28496 b592564a803c util: reword debugstacktrace comment timeless <timeless@mozdev.org> parents: 28027 diff changeset	3233 It can be used everywhere and intentionally does not require an ui object.
20244 47d0843647d1 util: introduce util.debugstacktrace for showing a stack trace without crashing Mads Kiilerich <madski@unity3d.com> parents: 20202 diff changeset	3234 Not be used in production code but very convenient while developing.
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	3235 """
20542 be27652675ce util: debugstacktrace, flush before and after writing Mads Kiilerich <madski@unity3d.com> parents: 20353 diff changeset	3236 if otherf:
be27652675ce util: debugstacktrace, flush before and after writing Mads Kiilerich <madski@unity3d.com> parents: 20353 diff changeset	3237 otherf.flush()
43774 02ededbef627 util: add an optional `prefix` argument to debugstacktrace Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43767 diff changeset	3238 f.write(b'%s%s at:\n' % (prefix, msg.rstrip()))
31324 78ac7061f840 util: add debugstacktrace depth limit Mads Kiilerich <madski@unity3d.com> parents: 31323 diff changeset	3239 for line in getstackframes(skip + 1, depth=depth):
43774 02ededbef627 util: add an optional `prefix` argument to debugstacktrace Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43767 diff changeset	3240 f.write(prefix + line)
20542 be27652675ce util: debugstacktrace, flush before and after writing Mads Kiilerich <madski@unity3d.com> parents: 20353 diff changeset	3241 f.flush()
20244 47d0843647d1 util: introduce util.debugstacktrace for showing a stack trace without crashing Mads Kiilerich <madski@unity3d.com> parents: 20202 diff changeset	3242
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3243
20244 47d0843647d1 util: introduce util.debugstacktrace for showing a stack trace without crashing Mads Kiilerich <madski@unity3d.com> parents: 20202 diff changeset	3244 # convenient shortcut
47d0843647d1 util: introduce util.debugstacktrace for showing a stack trace without crashing Mads Kiilerich <madski@unity3d.com> parents: 20202 diff changeset	3245 dst = debugstacktrace
34554 6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3246
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3247
34554 6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3248 def safename(f, tag, ctx, others=None):
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3249 """
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3250 Generate a name that it is safe to rename f to in the given context.
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3251
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3252 f: filename to rename
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3253 tag: a string tag that will be included in the new name
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3254 ctx: a context, in which the new name must not exist
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3255 others: a set of other filenames that the new name must not be in
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3256
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3257 Returns a file name of the form oldname~tag[~number] which does not exist
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3258 in the provided context and is not in the set of other names.
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3259 """
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3260 if others is None:
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3261 others = set()
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3262
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3263 fn = b'%s~%s' % (f, tag)
34554 6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3264 if fn not in ctx and fn not in others:
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3265 return fn
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3266 for n in itertools.count(1):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3267 fn = b'%s~%s~%s' % (f, tag, n)
34554 6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3268 if fn not in ctx and fn not in others:
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3269 return fn
35754 fb0be099063f util: move 'readexactly' in the util module Boris Feld <boris.feld@octobus.net> parents: 35582 diff changeset	3270
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3271
35754 fb0be099063f util: move 'readexactly' in the util module Boris Feld <boris.feld@octobus.net> parents: 35582 diff changeset	3272 def readexactly(stream, n):
fb0be099063f util: move 'readexactly' in the util module Boris Feld <boris.feld@octobus.net> parents: 35582 diff changeset	3273 '''read n bytes from stream.read and abort if less was available'''
fb0be099063f util: move 'readexactly' in the util module Boris Feld <boris.feld@octobus.net> parents: 35582 diff changeset	3274 s = stream.read(n)
fb0be099063f util: move 'readexactly' in the util module Boris Feld <boris.feld@octobus.net> parents: 35582 diff changeset	3275 if len(s) < n:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3276 raise error.Abort(
43117 8ff1ecfadcd1 cleanup: join string literals that are already on one line Martin von Zweigbergk <martinvonz@google.com> parents: 43106 diff changeset	3277 _(b"stream ended unexpectedly (got %d bytes, expected %d)")
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3278 % (len(s), n)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3279 )
35754 fb0be099063f util: move 'readexactly' in the util module Boris Feld <boris.feld@octobus.net> parents: 35582 diff changeset	3280 return s
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3281
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3282
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3283 def uvarintencode(value):
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3284 """Encode an unsigned integer value to a varint.
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3285
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3286 A varint is a variable length integer of 1 or more bytes. Each byte
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3287 except the last has the most significant bit set. The lower 7 bits of
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3288 each byte store the 2's complement representation, least significant group
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3289 first.
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3290
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3291 >>> uvarintencode(0)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3292 '\\x00'
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3293 >>> uvarintencode(1)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3294 '\\x01'
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3295 >>> uvarintencode(127)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3296 '\\x7f'
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3297 >>> uvarintencode(1337)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3298 '\\xb9\\n'
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3299 >>> uvarintencode(65536)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3300 '\\x80\\x80\\x04'
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3301 >>> uvarintencode(-1)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3302 Traceback (most recent call last):
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3303 ...
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3304 ProgrammingError: negative value for uvarint: -1
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3305 """
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3306 if value < 0:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3307 raise error.ProgrammingError(b'negative value for uvarint: %d' % value)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3308 bits = value & 0x7F
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3309 value >>= 7
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3310 bytes = []
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3311 while value:
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3312 bytes.append(pycompat.bytechr(0x80 \| bits))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3313 bits = value & 0x7F
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3314 value >>= 7
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3315 bytes.append(pycompat.bytechr(bits))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3316
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3317 return b''.join(bytes)
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3318
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3319
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3320 def uvarintdecodestream(fh):
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3321 """Decode an unsigned variable length integer from a stream.
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3322
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3323 The passed argument is anything that has a ``.read(N)`` method.
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3324
49908 54114bba7c7e tests: drop some obsolete py2 handling in util.py doctest Matt Harbison <matt_harbison@yahoo.com> parents: 49721 diff changeset	3325 >>> from io import BytesIO
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3326 >>> uvarintdecodestream(BytesIO(b'\\x00'))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3327 0
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3328 >>> uvarintdecodestream(BytesIO(b'\\x01'))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3329 1
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3330 >>> uvarintdecodestream(BytesIO(b'\\x7f'))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3331 127
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3332 >>> uvarintdecodestream(BytesIO(b'\\xb9\\n'))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3333 1337
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3334 >>> uvarintdecodestream(BytesIO(b'\\x80\\x80\\x04'))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3335 65536
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3336 >>> uvarintdecodestream(BytesIO(b'\\x80'))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3337 Traceback (most recent call last):
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3338 ...
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3339 Abort: stream ended unexpectedly (got 0 bytes, expected 1)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3340 """
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3341 result = 0
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3342 shift = 0
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3343 while True:
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3344 byte = ord(readexactly(fh, 1))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3345 result \|= (byte & 0x7F) << shift
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3346 if not (byte & 0x80):
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3347 return result
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3348 shift += 7
44826 1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3349
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3350
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3351 # Passing the '' locale means that the locale should be set according to the
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3352 # user settings (environment variables).
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3353 # Python sometimes avoids setting the global locale settings. When interfacing
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3354 # with C code (e.g. the curses module or the Subversion bindings), the global
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3355 # locale settings must be initialized correctly. Python 2 does not initialize
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3356 # the global locale settings on interpreter startup. Python 3 sometimes
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3357 # initializes LC_CTYPE, but not consistently at least on Windows. Therefore we
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3358 # explicitly initialize it to get consistent behavior if it's not already
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3359 # initialized. Since CPython commit 177d921c8c03d30daa32994362023f777624b10d,
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3360 # LC_CTYPE is always initialized. If we require Python 3.8+, we should re-check
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3361 # if we can remove this code.
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3362 @contextlib.contextmanager
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3363 def with_lc_ctype():
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3364 oldloc = locale.setlocale(locale.LC_CTYPE, None)
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3365 if oldloc == 'C':
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3366 try:
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3367 try:
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3368 locale.setlocale(locale.LC_CTYPE, '')
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3369 except locale.Error:
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3370 # The likely case is that the locale from the environment
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3371 # variables is unknown.
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3372 pass
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3373 yield
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3374 finally:
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3375 locale.setlocale(locale.LC_CTYPE, oldloc)
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3376 else:
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44633 diff changeset	3377 yield
45062 02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3378
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3379
51304 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51302 diff changeset	3380 def _estimatememory() -> Optional[int]:
45062 02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3381 """Provide an estimate for the available system memory in Bytes.
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3382
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3383 If no estimate can be provided on the platform, returns None.
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3384 """
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3385 if pycompat.sysplatform.startswith(b'win'):
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3386 # On Windows, use the GlobalMemoryStatusEx kernel function directly.
51917 8f3cbea2547c util: add a comment to suppress a PyCharm warning about a PEP 8 violation Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	3387 # noinspection PyPep8Naming
45062 02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3388 from ctypes import c_long as DWORD, c_ulonglong as DWORDLONG
46684 7711853110b9 typing: add some type annotations to mercurial/util.py Matt Harbison <matt_harbison@yahoo.com> parents: 46114 diff changeset	3389 from ctypes.wintypes import ( # pytype: disable=import-error
7711853110b9 typing: add some type annotations to mercurial/util.py Matt Harbison <matt_harbison@yahoo.com> parents: 46114 diff changeset	3390 Structure,
7711853110b9 typing: add some type annotations to mercurial/util.py Matt Harbison <matt_harbison@yahoo.com> parents: 46114 diff changeset	3391 byref,
7711853110b9 typing: add some type annotations to mercurial/util.py Matt Harbison <matt_harbison@yahoo.com> parents: 46114 diff changeset	3392 sizeof,
7711853110b9 typing: add some type annotations to mercurial/util.py Matt Harbison <matt_harbison@yahoo.com> parents: 46114 diff changeset	3393 windll,
7711853110b9 typing: add some type annotations to mercurial/util.py Matt Harbison <matt_harbison@yahoo.com> parents: 46114 diff changeset	3394 )
45062 02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3395
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3396 class MEMORYSTATUSEX(Structure):
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3397 _fields_ = [
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3398 ('dwLength', DWORD),
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3399 ('dwMemoryLoad', DWORD),
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3400 ('ullTotalPhys', DWORDLONG),
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3401 ('ullAvailPhys', DWORDLONG),
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3402 ('ullTotalPageFile', DWORDLONG),
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3403 ('ullAvailPageFile', DWORDLONG),
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3404 ('ullTotalVirtual', DWORDLONG),
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3405 ('ullAvailVirtual', DWORDLONG),
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3406 ('ullExtendedVirtual', DWORDLONG),
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3407 ]
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3408
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3409 x = MEMORYSTATUSEX()
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3410 x.dwLength = sizeof(x)
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3411 windll.kernel32.GlobalMemoryStatusEx(byref(x))
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3412 return x.ullAvailPhys
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3413
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3414 # On newer Unix-like systems and Mac OSX, the sysconf interface
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3415 # can be used. _SC_PAGE_SIZE is part of POSIX; _SC_PHYS_PAGES
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3416 # seems to be implemented on most systems.
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3417 try:
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3418 pagesize = os.sysconf(os.sysconf_names['SC_PAGE_SIZE'])
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3419 pages = os.sysconf(os.sysconf_names['SC_PHYS_PAGES'])
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3420 return pagesize * pages
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3421 except OSError: # sysconf can fail
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3422 pass
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3423 except KeyError: # unknown parameter
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45034 diff changeset	3424 pass

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/util.py @ 52669:e627cc25b6f3