mercurial-scm/hg: mercurial/util.py annotate

annotate mercurial/util.py @ 52049:af54626bf358

dirstate-map: add a missing debug wait point when accessing the v2 docket fc8e37c380d3 added synchronization points to the dirstate to allow for race condition testing without actually requiring a time-based race condition to happen. This changes adds the `pre-read-file` wait point before we read the docket, since callers might ask for the parents before anything else is read, leading to the first read being done before the wait point. This removes some differences in test output which were presumed to be speed related, but weren't.

author	Rapha?l Gom?s <rgomes@octobus.net>
date	Mon, 14 Oct 2024 14:14:21 +0200
parents	fa7059f031a9
children	82e2c99c84f3

rev	line source
17515 b5b38d21fe99 spelling: specific timeless@mozdev.org parents: 17237 diff changeset	1 # util.py - Mercurial utility functions and platform specific implementations
8226 8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	2 #
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	3 # Copyright 2005 K. Thananchayan <thananck@yahoo.com>
46819 d4ba4d51f85f contributor: change mentions of mpm to olivia Rapha?l Gom?s <rgomes@octobus.net> parents: 46675 diff changeset	4 # Copyright 2005-2007 Olivia Mackall <olivia@selenic.com>
8226 8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	5 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	6 #
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	7 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 9996 diff changeset	8 # GNU General Public License version 2 or any later version.
1082 ce96e316278a Update util.py docstrings, fix walk test mpm@selenic.com parents: 1081 diff changeset	9
17515 b5b38d21fe99 spelling: specific timeless@mozdev.org parents: 17237 diff changeset	10 """Mercurial utility functions and platform specific implementations.
1082 ce96e316278a Update util.py docstrings, fix walk test mpm@selenic.com parents: 1081 diff changeset	11
8227 0a9542703300 turn some comments back into module docstrings Martin Geisler <mg@lazybytes.net> parents: 8226 diff changeset	12 This contains helper routines that are independent of the SCM core and
0a9542703300 turn some comments back into module docstrings Martin Geisler <mg@lazybytes.net> parents: 8226 diff changeset	13 hide platform-specific details from the core.
1082 ce96e316278a Update util.py docstrings, fix walk test mpm@selenic.com parents: 1081 diff changeset	14 """
419 28511fc21073 [PATCH] file seperator handling for the other 'OS' mpm@selenic.com parents: diff changeset	15
51859 f4733654f144 typing: add `from __future__ import annotations` to most files Matt Harbison <matt_harbison@yahoo.com> parents: 51846 diff changeset	16 from __future__ import annotations
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	17
33793 bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	18 import abc
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	19 import collections
33446 fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	20 import contextlib
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	21 import errno
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	22 import gc
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	23 import hashlib
48873 5aafc3c5bdec py3: use io.BytesIO directly Gregory Szorc <gregory.szorc@gmail.com> parents: 48870 diff changeset	24 import io
34554 6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	25 import itertools
45011 1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	26 import locale
34295 3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34139 diff changeset	27 import mmap
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	28 import os
48993 225659936fff util: restore the util.pickle symbol Matt Harbison <matt_harbison@yahoo.com> parents: 48946 diff changeset	29 import pickle # provides util.pickle symbol
21907 7e5dfa00e3c2 util: rename 're' to 'remod' Siddharth Agarwal <sid0@fb.com> parents: 21857 diff changeset	30 import re as remod
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	31 import shutil
30418 1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30417 diff changeset	32 import stat
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	33 import sys
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	34 import time
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	35 import traceback
51717 ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	36 import typing
31950 cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	37 import warnings
3769 96095d9ff1f8 Add encoding detection Matt Mackall <mpm@selenic.com> parents: 3767 diff changeset	38
51282 9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	39 from typing import (
51546 a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	40 Any,
51717 ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	41 BinaryIO,
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	42 Callable,
51282 9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	43 Iterable,
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	44 Iterator,
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	45 List,
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	46 Optional,
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	47 Tuple,
51883 1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51876 diff changeset	48 Type,
1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51876 diff changeset	49 TypeVar,
51282 9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	50 )
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	51
47012 d55b71393907 node: replace nullid and friends with nodeconstants class Joerg Sonnenberger <joerg@bec.de> parents: 46926 diff changeset	52 from .node import hex
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	53 from .thirdparty import attr
51725 278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51717 diff changeset	54
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51717 diff changeset	55 # Force pytype to use the non-vendored package
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51717 diff changeset	56 if typing.TYPE_CHECKING:
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51717 diff changeset	57 # noinspection PyPackageRequirements
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51717 diff changeset	58 import attr
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51717 diff changeset	59
43087 66f2cc210a29 py3: manually import pycompat.setattr where it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 43085 diff changeset	60 from .pycompat import (
66f2cc210a29 py3: manually import pycompat.setattr where it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 43085 diff changeset	61 open,
66f2cc210a29 py3: manually import pycompat.setattr where it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 43085 diff changeset	62 )
39259 e00123f63410 util: make timedcm require the label (API) Augie Fackler <augie@google.com> parents: 39258 diff changeset	63 from hgdemandimport import tracing
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	64 from . import (
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	65 encoding,
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	66 error,
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	67 i18n,
32367 a9c71d578a1c osutil: switch to policy importer Yuya Nishihara <yuya@tcha.org> parents: 32306 diff changeset	68 policy,
28818 6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence timeless <timeless@mozdev.org> parents: 28497 diff changeset	69 pycompat,
51717 ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	70 typelib,
34467 192f7b126ed2 urllibcompat: move some adapters from pycompat to urllibcompat Augie Fackler <augie@google.com> parents: 34435 diff changeset	71 urllibcompat,
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	72 )
51934 fa7059f031a9 interfaces: introduce and use a protocol class for the `base85` module Matt Harbison <matt_harbison@yahoo.com> parents: 51927 diff changeset	73 from .interfaces import (
fa7059f031a9 interfaces: introduce and use a protocol class for the `base85` module Matt Harbison <matt_harbison@yahoo.com> parents: 51927 diff changeset	74 modules as intmod,
fa7059f031a9 interfaces: introduce and use a protocol class for the `base85` module Matt Harbison <matt_harbison@yahoo.com> parents: 51927 diff changeset	75 )
37083 f99d64e8a4e4 stringutil: move generic string helpers to new module Yuya Nishihara <yuya@tcha.org> parents: 37082 diff changeset	76 from .utils import (
42041 3e47d1ec9da5 util: extract compression code in `mercurial.utils.compression` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 41834 diff changeset	77 compression,
44060 a61287a95dc3 core: migrate uses of hashlib.sha1 to hashutil.sha1 Augie Fackler <augie@google.com> parents: 44035 diff changeset	78 hashutil,
37118 5be286db5fb5 procutil: move process/executable management functions to new module Yuya Nishihara <yuya@tcha.org> parents: 37117 diff changeset	79 procutil,
37083 f99d64e8a4e4 stringutil: move generic string helpers to new module Yuya Nishihara <yuya@tcha.org> parents: 37082 diff changeset	80 stringutil,
f99d64e8a4e4 stringutil: move generic string helpers to new module Yuya Nishihara <yuya@tcha.org> parents: 37082 diff changeset	81 )
3769 96095d9ff1f8 Add encoding detection Matt Mackall <mpm@selenic.com> parents: 3767 diff changeset	82
51282 9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	83 # keeps pyflakes happy
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	84 assert [
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	85 Iterable,
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	86 Iterator,
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	87 List,
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	88 Optional,
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	89 Tuple,
9d3721552b6c pytype: import typing directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	90 ]
46645 7711853110b9 typing: add some type annotations to mercurial/util.py Matt Harbison <matt_harbison@yahoo.com> parents: 46113 diff changeset	91
7711853110b9 typing: add some type annotations to mercurial/util.py Matt Harbison <matt_harbison@yahoo.com> parents: 46113 diff changeset	92
51934 fa7059f031a9 interfaces: introduce and use a protocol class for the `base85` module Matt Harbison <matt_harbison@yahoo.com> parents: 51927 diff changeset	93 base85: intmod.Base85 = policy.importmod('base85')
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	94 osutil = policy.importmod('osutil')
32367 a9c71d578a1c osutil: switch to policy importer Yuya Nishihara <yuya@tcha.org> parents: 32306 diff changeset	95
32201 4462a981e8df base85: proxy through util module Yuya Nishihara <yuya@tcha.org> parents: 32154 diff changeset	96 b85decode = base85.b85decode
4462a981e8df base85: proxy through util module Yuya Nishihara <yuya@tcha.org> parents: 32154 diff changeset	97 b85encode = base85.b85encode
4462a981e8df base85: proxy through util module Yuya Nishihara <yuya@tcha.org> parents: 32154 diff changeset	98
31934 12aca6770046 util: make cookielib module available Gregory Szorc <gregory.szorc@gmail.com> parents: 31878 diff changeset	99 cookielib = pycompat.cookielib
30471 00c9ac4ce816 util: rewrite pycompat imports to make pyflakes always happy Yuya Nishihara <yuya@tcha.org> parents: 30442 diff changeset	100 httplib = pycompat.httplib
37099 6ca5f825a0ca util: make safehasattr() a pycompat function Yuya Nishihara <yuya@tcha.org> parents: 37098 diff changeset	101 safehasattr = pycompat.safehasattr
30471 00c9ac4ce816 util: rewrite pycompat imports to make pyflakes always happy Yuya Nishihara <yuya@tcha.org> parents: 30442 diff changeset	102 socketserver = pycompat.socketserver
48873 5aafc3c5bdec py3: use io.BytesIO directly Gregory Szorc <gregory.szorc@gmail.com> parents: 48870 diff changeset	103 bytesio = io.BytesIO
36958 644a02f6b34f util: prefer "bytesio" to "stringio" Gregory Szorc <gregory.szorc@gmail.com> parents: 36946 diff changeset	104 # TODO deprecate stringio name, as it is a lie on Python 3.
644a02f6b34f util: prefer "bytesio" to "stringio" Gregory Szorc <gregory.szorc@gmail.com> parents: 36946 diff changeset	105 stringio = bytesio
30471 00c9ac4ce816 util: rewrite pycompat imports to make pyflakes always happy Yuya Nishihara <yuya@tcha.org> parents: 30442 diff changeset	106 xmlrpclib = pycompat.xmlrpclib
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28882 diff changeset	107
34467 192f7b126ed2 urllibcompat: move some adapters from pycompat to urllibcompat Augie Fackler <augie@google.com> parents: 34435 diff changeset	108 httpserver = urllibcompat.httpserver
192f7b126ed2 urllibcompat: move some adapters from pycompat to urllibcompat Augie Fackler <augie@google.com> parents: 34435 diff changeset	109 urlerr = urllibcompat.urlerr
192f7b126ed2 urllibcompat: move some adapters from pycompat to urllibcompat Augie Fackler <augie@google.com> parents: 34435 diff changeset	110 urlreq = urllibcompat.urlreq
192f7b126ed2 urllibcompat: move some adapters from pycompat to urllibcompat Augie Fackler <augie@google.com> parents: 34435 diff changeset	111
32572 377c74ef008d win32mbcs: avoid unintentional failure at colorization FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 31952 diff changeset	112 # workaround for win32mbcs
377c74ef008d win32mbcs: avoid unintentional failure at colorization FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 31952 diff changeset	113 _filenamebytestr = pycompat.bytestr
377c74ef008d win32mbcs: avoid unintentional failure at colorization FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 31952 diff changeset	114
34645 75979c8d4572 codemod: use pycompat.iswindows Jun Wu <quark@fb.com> parents: 34554 diff changeset	115 if pycompat.iswindows:
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	116 from . import windows as platform
14912 ec46a7da9f2c util: move windows and posix wildcard imports to begin of file Adrian Buehlmann <adrian@cadifra.com> parents: 14911 diff changeset	117 else:
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	118 from . import posix as platform
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	119
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	120 _ = i18n._
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	121
47622 bb917eea1605 windows: introduce a `util.abspath` to replace os.path.abspath Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47446 diff changeset	122 abspath = platform.abspath
29530 3239e2fdd2e2 chgserver: extract utility to bind unix domain socket to long path Yuya Nishihara <yuya@tcha.org> parents: 29455 diff changeset	123 bindunixsocket = platform.bindunixsocket
14927 2aa3e07b2f07 posix, windows: introduce cachestat Idan Kamara <idankk86@gmail.com> parents: 14926 diff changeset	124 cachestat = platform.cachestat
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	125 checkexec = platform.checkexec
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	126 checklink = platform.checklink
15011 5e44e4b3a0a3 util: move copymode into posix.py and windows.py Adrian Buehlmann <adrian@cadifra.com> parents: 15010 diff changeset	127 copymode = platform.copymode
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	128 expandglobs = platform.expandglobs
35513 c4caf530b1c7 util: add a function to show the mount point of the filesystem Matt Harbison <matt_harbison@yahoo.com> parents: 35511 diff changeset	129 getfsmountpoint = platform.getfsmountpoint
35509 beede158ea8a util: move getfstype() to the platform modules Matt Harbison <matt_harbison@yahoo.com> parents: 35460 diff changeset	130 getfstype = platform.getfstype
47079 5b3513177f2b util: avoid echoing the password to the console on Windows py3 (issue6446) Matt Harbison <matt_harbison@yahoo.com> parents: 47062 diff changeset	131 get_password = platform.get_password
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	132 groupmembers = platform.groupmembers
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	133 groupname = platform.groupname
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	134 isexec = platform.isexec
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	135 isowner = platform.isowner
32208 d74b0cff94a9 osutil: proxy through util (and platform) modules (API) Yuya Nishihara <yuya@tcha.org> parents: 32201 diff changeset	136 listdir = osutil.listdir
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	137 localpath = platform.localpath
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	138 lookupreg = platform.lookupreg
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	139 makedir = platform.makedir
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	140 nlinks = platform.nlinks
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	141 normpath = platform.normpath
15488 6eff984d8e76 dirstate: fix case-folding identity for traditional Unix Matt Mackall <mpm@selenic.com> parents: 15392 diff changeset	142 normcase = platform.normcase
24605 98744856b7d3 util: add normcase spec and fallback Siddharth Agarwal <sid0@fb.com> parents: 24439 diff changeset	143 normcasespec = platform.normcasespec
98744856b7d3 util: add normcase spec and fallback Siddharth Agarwal <sid0@fb.com> parents: 24439 diff changeset	144 normcasefallback = platform.normcasefallback
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	145 openhardlinks = platform.openhardlinks
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	146 oslink = platform.oslink
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	147 parsepatchoutput = platform.parsepatchoutput
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	148 pconvert = platform.pconvert
25420 c2ec81891502 util: add a simple poll utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25406 diff changeset	149 poll = platform.poll
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	150 posixfile = platform.posixfile
39904 5fe0b880200e py3: convert os.readlink() path to native strings on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 39777 diff changeset	151 readlink = platform.readlink
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	152 rename = platform.rename
24692 144883a8d0d4 util: add removedirs as platform depending function FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24635 diff changeset	153 removedirs = platform.removedirs
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	154 samedevice = platform.samedevice
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	155 samefile = platform.samefile
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	156 samestat = platform.samestat
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	157 setflags = platform.setflags
17560 9ee25d7b1aed util: implement a faster os.path.split for posix systems Bryan O'Sullivan <bryano@fb.com> parents: 17537 diff changeset	158 split = platform.split
18026 ddc0323db78b osutil: write a C implementation of statfiles for unix Bryan O'Sullivan <bryano@fb.com> parents: 18013 diff changeset	159 statfiles = getattr(osutil, 'statfiles', platform.statfiles)
18868 cafa447a7d3b util: add functions to check symlink/exec bits Bryan O'Sullivan <bryano@fb.com> parents: 18775 diff changeset	160 statisexec = platform.statisexec
cafa447a7d3b util: add functions to check symlink/exec bits Bryan O'Sullivan <bryano@fb.com> parents: 18775 diff changeset	161 statislink = platform.statislink
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	162 umask = platform.umask
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	163 unlink = platform.unlink
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	164 username = platform.username
14912 ec46a7da9f2c util: move windows and posix wildcard imports to begin of file Adrian Buehlmann <adrian@cadifra.com> parents: 14911 diff changeset	165
44629 d37975386798 chgserver: update the umask cache before each run Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	166
51883 1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51876 diff changeset	167 if typing.TYPE_CHECKING:
1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51876 diff changeset	168 _Tfilestat = TypeVar('_Tfilestat', bound='filestat')
1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51876 diff changeset	169
1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51876 diff changeset	170
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	171 def setumask(val: int) -> None:
47062 f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1 Kyle Lippincott <spectral@google.com> parents: 46926 diff changeset	172 '''updates the umask. used by chg server'''
44629 d37975386798 chgserver: update the umask cache before each run Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	173 if pycompat.iswindows:
d37975386798 chgserver: update the umask cache before each run Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	174 return
d37975386798 chgserver: update the umask cache before each run Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	175 os.umask(val)
d37975386798 chgserver: update the umask cache before each run Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	176 global umask
d37975386798 chgserver: update the umask cache before each run Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	177 platform.umask = umask = val & 0o777
d37975386798 chgserver: update the umask cache before each run Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	178
d37975386798 chgserver: update the umask cache before each run Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	179
42041 3e47d1ec9da5 util: extract compression code in `mercurial.utils.compression` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 41834 diff changeset	180 # small compat layer
3e47d1ec9da5 util: extract compression code in `mercurial.utils.compression` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 41834 diff changeset	181 compengines = compression.compengines
3e47d1ec9da5 util: extract compression code in `mercurial.utils.compression` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 41834 diff changeset	182 SERVERROLE = compression.SERVERROLE
3e47d1ec9da5 util: extract compression code in `mercurial.utils.compression` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 41834 diff changeset	183 CLIENTROLE = compression.CLIENTROLE
3e47d1ec9da5 util: extract compression code in `mercurial.utils.compression` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 41834 diff changeset	184
6470 ac0bcd951c2c python 2.6 compatibility: compatibility wrappers for hash functions Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 6339 diff changeset	185 # Python compatibility
3769 96095d9ff1f8 Add encoding detection Matt Mackall <mpm@selenic.com> parents: 3767 diff changeset	186
15656 4f5a78fa4917 util: clean up function ordering Matt Mackall <mpm@selenic.com> parents: 15611 diff changeset	187 _notset = object()
4f5a78fa4917 util: clean up function ordering Matt Mackall <mpm@selenic.com> parents: 15611 diff changeset	188
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	189
30745 c1b7b2285522 revlog: flag processor Remi Chaintron <remi@fb.com> parents: 30736 diff changeset	190 def bitsfrom(container):
c1b7b2285522 revlog: flag processor Remi Chaintron <remi@fb.com> parents: 30736 diff changeset	191 bits = 0
c1b7b2285522 revlog: flag processor Remi Chaintron <remi@fb.com> parents: 30736 diff changeset	192 for bit in container:
c1b7b2285522 revlog: flag processor Remi Chaintron <remi@fb.com> parents: 30736 diff changeset	193 bits \|= bit
c1b7b2285522 revlog: flag processor Remi Chaintron <remi@fb.com> parents: 30736 diff changeset	194 return bits
c1b7b2285522 revlog: flag processor Remi Chaintron <remi@fb.com> parents: 30736 diff changeset	195
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	196
31950 cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	197 # python 2.6 still have deprecation warning enabled by default. We do not want
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	198 # to display anything to standard user so detect if we are running test and
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	199 # only use python deprecation warning in this case.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	200 _dowarn = bool(encoding.environ.get(b'HGEMITWARNINGS'))
31950 cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	201 if _dowarn:
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	202 # explicitly unfilter our warning for python 2.7
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	203 #
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	204 # The option of setting PYTHONWARNINGS in the test runner was investigated.
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	205 # However, module name set through PYTHONWARNINGS was exactly matched, so
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	206 # we cannot set 'mercurial' and have it match eg: 'mercurial.scmutil'. This
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	207 # makes the whole PYTHONWARNINGS thing useless for our usecase.
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	208 warnings.filterwarnings('default', '', DeprecationWarning, 'mercurial')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	209 warnings.filterwarnings('default', '', DeprecationWarning, 'hgext')
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	210 warnings.filterwarnings('default', '', DeprecationWarning, 'hgext3rd')
48897 10839394f39b util: remove superfluous ispy3 test Gregory Szorc <gregory.szorc@gmail.com> parents: 48875 diff changeset	211 if _dowarn:
36588 281f66777ff0 py3: silence "bad escape" warning emitted by re.sub() Yuya Nishihara <yuya@tcha.org> parents: 36585 diff changeset	212 # silence warning emitted by passing user string to re.sub()
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	213 warnings.filterwarnings(
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	214 'ignore', 'bad escape', DeprecationWarning, 'mercurial'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	215 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	216 warnings.filterwarnings(
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	217 'ignore', 'invalid escape sequence', DeprecationWarning, 'mercurial'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	218 )
37455 9ecb7c471cfb py3: silence warning about deprecation of imp module Yuya Nishihara <yuya@tcha.org> parents: 37382 diff changeset	219 # TODO: reinvent imp.is_frozen()
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	220 warnings.filterwarnings(
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	221 'ignore',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	222 'the imp module is deprecated',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	223 DeprecationWarning,
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	224 'mercurial',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	225 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	226
31950 cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	227
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	228 def nouideprecwarn(msg, version, stacklevel=1):
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	229 """Issue an python native deprecation warning
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	230
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	231 This is a noop outside of tests, use 'ui.deprecwarn' when possible.
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	232 """
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	233 if _dowarn:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	234 msg += (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	235 b"\n(compatibility will be dropped after Mercurial-%s,"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	236 b" update your code.)"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	237 ) % version
36128 02ed94dd9fd6 util: call warnings.warn() with a sysstr in nouideprecwarn Augie Fackler <augie@google.com> parents: 36038 diff changeset	238 warnings.warn(pycompat.sysstr(msg), DeprecationWarning, stacklevel + 1)
44989 7af5c1f5b3a0 util: flush stderr explicitly after using warnings.warn() Pulkit Goyal <7895pulkit@gmail.com> parents: 44868 diff changeset	239 # on python 3 with chg, we will need to explicitly flush the output
7af5c1f5b3a0 util: flush stderr explicitly after using warnings.warn() Pulkit Goyal <7895pulkit@gmail.com> parents: 44868 diff changeset	240 sys.stderr.flush()
31950 cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	241
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	242
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	243 DIGESTS = {
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	244 b'md5': hashlib.md5,
44060 a61287a95dc3 core: migrate uses of hashlib.sha1 to hashutil.sha1 Augie Fackler <augie@google.com> parents: 44035 diff changeset	245 b'sha1': hashutil.sha1,
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	246 b'sha512': hashlib.sha512,
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	247 }
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	248 # List of digest types from strongest to weakest
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	249 DIGESTS_BY_STRENGTH = [b'sha512', b'sha1', b'md5']
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	250
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	251 for k in DIGESTS_BY_STRENGTH:
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	252 assert k in DIGESTS
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	253
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	254
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48940 diff changeset	255 class digester:
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	256 """helper to compute digests.
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	257
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	258 This helper can be used to compute one or more digests given their name.
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	259
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	260 >>> d = digester([b'md5', b'sha1'])
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	261 >>> d.update(b'foo')
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	262 >>> [k for k in sorted(d)]
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	263 ['md5', 'sha1']
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	264 >>> d[b'md5']
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	265 'acbd18db4cc2f85cedef654fccc4a4d8'
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	266 >>> d[b'sha1']
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	267 '0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33'
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	268 >>> digester.preferred([b'md5', b'sha1'])
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	269 'sha1'
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	270 """
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	271
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	272 def __init__(self, digests, s=b''):
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	273 self._hashes = {}
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	274 for k in digests:
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	275 if k not in DIGESTS:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	276 raise error.Abort(_(b'unknown digest type: %s') % k)
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	277 self._hashes[k] = DIGESTS[k]()
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	278 if s:
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	279 self.update(s)
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	280
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	281 def update(self, data):
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	282 for h in self._hashes.values():
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	283 h.update(data)
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	284
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	285 def __getitem__(self, key):
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	286 if key not in DIGESTS:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	287 raise error.Abort(_(b'unknown digest type: %s') % k)
46113 59fa3890d40a node: import symbols explicitly Joerg Sonnenberger <joerg@bec.de> parents: 45942 diff changeset	288 return hex(self._hashes[key].digest())
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	289
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	290 def __iter__(self):
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	291 return iter(self._hashes)
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	292
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	293 @staticmethod
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	294 def preferred(supported):
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	295 """returns the strongest digest type in both supported and DIGESTS."""
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	296
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	297 for k in DIGESTS_BY_STRENGTH:
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	298 if k in supported:
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	299 return k
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	300 return None
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	301
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	302
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48940 diff changeset	303 class digestchecker:
22963 56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	304 """file handle wrapper that additionally checks content against a given
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	305 size and digests.
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	306
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	307 d = digestchecker(fh, size, {'md5': '...'})
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	308
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	309 When multiple digests are given, all of them are validated.
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	310 """
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	311
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	312 def __init__(self, fh, size, digests):
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	313 self._fh = fh
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	314 self._size = size
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	315 self._got = 0
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	316 self._digests = dict(digests)
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	317 self._digester = digester(self._digests.keys())
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	318
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	319 def read(self, length=-1):
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	320 content = self._fh.read(length)
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	321 self._digester.update(content)
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	322 self._got += len(content)
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	323 return content
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	324
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	325 def validate(self):
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	326 if self._size != self._got:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	327 raise error.Abort(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	328 _(b'size mismatch: expected %d, got %d')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	329 % (self._size, self._got)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	330 )
22963 56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	331 for k, v in self._digests.items():
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	332 if v != self._digester[k]:
23076 c312ef382033 i18n: add hint to digest mismatch message Wagner Bruna <wbruna@softwareexpress.com.br> parents: 23030 diff changeset	333 # i18n: first parameter is a digest name
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	334 raise error.Abort(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	335 _(b'%s mismatch: expected %s, got %s')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	336 % (k, v, self._digester[k])
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	337 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	338
22963 56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	339
11565 7546d4a272c8 util: improved the check for the existence of the 'buffer' builtin Renato Cunha <renatoc@gmail.com> parents: 11469 diff changeset	340 try:
46645 7711853110b9 typing: add some type annotations to mercurial/util.py Matt Harbison <matt_harbison@yahoo.com> parents: 46113 diff changeset	341 buffer = buffer # pytype: disable=name-error
11565 7546d4a272c8 util: improved the check for the existence of the 'buffer' builtin Renato Cunha <renatoc@gmail.com> parents: 11469 diff changeset	342 except NameError:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	343
33549 9a2ee9591acc util: remove dead code which used to be for old python2 versions Alex Gaynor <agaynor@mozilla.com> parents: 33446 diff changeset	344 def buffer(sliceable, offset=0, length=None):
9a2ee9591acc util: remove dead code which used to be for old python2 versions Alex Gaynor <agaynor@mozilla.com> parents: 33446 diff changeset	345 if length is not None:
51846 f5c46c3518a5 util: make buffer readonly Rapha?l Gom?s <rgomes@octobus.net> parents: 51796 diff changeset	346 view = memoryview(sliceable)[offset : offset + length]
f5c46c3518a5 util: make buffer readonly Rapha?l Gom?s <rgomes@octobus.net> parents: 51796 diff changeset	347 else:
f5c46c3518a5 util: make buffer readonly Rapha?l Gom?s <rgomes@octobus.net> parents: 51796 diff changeset	348 view = memoryview(sliceable)[offset:]
f5c46c3518a5 util: make buffer readonly Rapha?l Gom?s <rgomes@octobus.net> parents: 51796 diff changeset	349 return view.toreadonly()
10756 cb681cc59a8d util: fake the builtin buffer if it's missing (jython) Ronny Pfannschmidt <Ronny.Pfannschmidt@gmx.de> parents: 10487 diff changeset	350
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	351
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	352 _chunksize = 4096
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	353
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	354
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48940 diff changeset	355 class bufferedinputpipe:
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	356 """a manually buffered input pipe
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	357
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	358 Python will not let us use buffered IO and lazy reading with 'polling' at
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	359 the same time. We cannot probe the buffer state and select will not detect
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	360 that data are ready to read if they are already buffered.
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	361
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	362 This class let us work around that by implementing its own buffering
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	363 (allowing efficient readline) while offering a way to know if the buffer is
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	364 empty from the output (allowing collaboration of the buffer with polling).
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	365
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	366 This class lives in the 'util' module because it makes use of the 'os'
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	367 module from the python stdlib.
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	368 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	369
36525 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	370 def __new__(cls, fh):
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	371 # If we receive a fileobjectproxy, we need to use a variation of this
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	372 # class that notifies observers about activity.
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	373 if isinstance(fh, fileobjectproxy):
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	374 cls = observedbufferedinputpipe
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	375
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	376 return super(bufferedinputpipe, cls).__new__(cls)
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	377
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	378 def __init__(self, input):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	379 self._input = input
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	380 self._buffer = []
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	381 self._eof = False
25672 050dc6eabc92 bufferedinputpipe: remove N^2 computation of buffer length (issue4735) Pierre-Yves David <pierre-yves.david@fb.com> parents: 25671 diff changeset	382 self._lenbuf = 0
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	383
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	384 @property
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	385 def hasbuffer(self):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	386 """True is any data is currently buffered
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	387
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	388 This will be used externally a pre-step for polling IO. If there is
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	389 already data then no polling should be set in place."""
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	390 return bool(self._buffer)
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	391
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	392 @property
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	393 def closed(self):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	394 return self._input.closed
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	395
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	396 def fileno(self):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	397 return self._input.fileno()
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	398
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	399 def close(self):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	400 return self._input.close()
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	401
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	402 def read(self, size):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	403 while (not self._eof) and (self._lenbuf < size):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	404 self._fillbuffer()
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	405 return self._frombuffer(size)
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	406
38713 27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38575 diff changeset	407 def unbufferedread(self, size):
27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38575 diff changeset	408 if not self._eof and self._lenbuf == 0:
27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38575 diff changeset	409 self._fillbuffer(max(size, _chunksize))
27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38575 diff changeset	410 return self._frombuffer(min(self._lenbuf, size))
27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38575 diff changeset	411
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	412 def readline(self, args, *kwargs):
40029 e2697acd9381 cleanup: some Yoda conditions, this patch removes Martin von Zweigbergk <martinvonz@google.com> parents: 39905 diff changeset	413 if len(self._buffer) > 1:
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	414 # this should not happen because both read and readline end with a
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	415 # _frombuffer call that collapse it.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	416 self._buffer = [b''.join(self._buffer)]
25672 050dc6eabc92 bufferedinputpipe: remove N^2 computation of buffer length (issue4735) Pierre-Yves David <pierre-yves.david@fb.com> parents: 25671 diff changeset	417 self._lenbuf = len(self._buffer[0])
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	418 lfi = -1
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	419 if self._buffer:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	420 lfi = self._buffer[-1].find(b'\n')
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	421 while (not self._eof) and lfi < 0:
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	422 self._fillbuffer()
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	423 if self._buffer:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	424 lfi = self._buffer[-1].find(b'\n')
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	425 size = lfi + 1
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	426 if lfi < 0: # end of file
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	427 size = self._lenbuf
40029 e2697acd9381 cleanup: some Yoda conditions, this patch removes Martin von Zweigbergk <martinvonz@google.com> parents: 39905 diff changeset	428 elif len(self._buffer) > 1:
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	429 # we need to take previous chunks into account
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	430 size += self._lenbuf - len(self._buffer[-1])
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	431 return self._frombuffer(size)
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	432
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	433 def _frombuffer(self, size):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	434 """return at most 'size' data from the buffer
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	435
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	436 The data are removed from the buffer."""
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	437 if size == 0 or not self._buffer:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	438 return b''
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	439 buf = self._buffer[0]
40029 e2697acd9381 cleanup: some Yoda conditions, this patch removes Martin von Zweigbergk <martinvonz@google.com> parents: 39905 diff changeset	440 if len(self._buffer) > 1:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	441 buf = b''.join(self._buffer)
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	442
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	443 data = buf[:size]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	444 buf = buf[len(data) :]
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	445 if buf:
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	446 self._buffer = [buf]
25672 050dc6eabc92 bufferedinputpipe: remove N^2 computation of buffer length (issue4735) Pierre-Yves David <pierre-yves.david@fb.com> parents: 25671 diff changeset	447 self._lenbuf = len(buf)
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	448 else:
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	449 self._buffer = []
25672 050dc6eabc92 bufferedinputpipe: remove N^2 computation of buffer length (issue4735) Pierre-Yves David <pierre-yves.david@fb.com> parents: 25671 diff changeset	450 self._lenbuf = 0
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	451 return data
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	452
38713 27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38575 diff changeset	453 def _fillbuffer(self, size=_chunksize):
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	454 """read data to the buffer"""
38713 27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38575 diff changeset	455 data = os.read(self._input.fileno(), size)
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	456 if not data:
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	457 self._eof = True
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	458 else:
25672 050dc6eabc92 bufferedinputpipe: remove N^2 computation of buffer length (issue4735) Pierre-Yves David <pierre-yves.david@fb.com> parents: 25671 diff changeset	459 self._lenbuf += len(data)
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	460 self._buffer.append(data)
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	461
36525 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	462 return data
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	463
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	464
51679 5f37c36f36b9 revlog: use mmap by default is pre-population is available Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51677 diff changeset	465 def has_mmap_populate():
51738 b619ba39d10a mmap: populate mapping in a background thread Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51727 diff changeset	466 return hasattr(osutil, "background_mmap_populate") or hasattr(
b619ba39d10a mmap: populate mapping in a background thread Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51727 diff changeset	467 mmap, 'MAP_POPULATE'
b619ba39d10a mmap: populate mapping in a background thread Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51727 diff changeset	468 )
51679 5f37c36f36b9 revlog: use mmap by default is pre-population is available Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51677 diff changeset	469
5f37c36f36b9 revlog: use mmap by default is pre-population is available Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51677 diff changeset	470
51677 522b4d729e89 mmap: populate the mapping by default Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51649 diff changeset	471 def mmapread(fp, size=None, pre_populate=True):
51649 ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51620 diff changeset	472 """Read a file content using mmap
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51620 diff changeset	473
51927 2d51b0cf707c util: minor copy editing of the documentation for `mmapread()` Matt Harbison <matt_harbison@yahoo.com> parents: 51926 diff changeset	474 The responsibility of checking the file system is mmap safe is the
2d51b0cf707c util: minor copy editing of the documentation for `mmapread()` Matt Harbison <matt_harbison@yahoo.com> parents: 51926 diff changeset	475 responsibility of the caller (see `vfs.is_mmap_safe`).
51649 ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51620 diff changeset	476
ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51620 diff changeset	477 In some case, a normal string might be returned.
51677 522b4d729e89 mmap: populate the mapping by default Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51649 diff changeset	478
522b4d729e89 mmap: populate the mapping by default Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51649 diff changeset	479 If `pre_populate` is True (the default), the mmapped data will be
522b4d729e89 mmap: populate the mapping by default Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51649 diff changeset	480 pre-populated in memory if the system support this option, this slow down
51927 2d51b0cf707c util: minor copy editing of the documentation for `mmapread()` Matt Harbison <matt_harbison@yahoo.com> parents: 51926 diff changeset	481 the initial mmapping but avoid potentially crippling page fault on later
51677 522b4d729e89 mmap: populate the mapping by default Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51649 diff changeset	482 access. If this is not the desired behavior, set `pre_populate` to False.
51649 ba205f944cb4 mmap: add a `is_mmap_safe` method to vfs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51620 diff changeset	483 """
44035 8ed8dfbeabb9 mmap: add a size argument to mmapread Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44021 diff changeset	484 if size == 0:
8ed8dfbeabb9 mmap: add a size argument to mmapread Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44021 diff changeset	485 # size of 0 to mmap.mmap() means "all data"
8ed8dfbeabb9 mmap: add a size argument to mmapread Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44021 diff changeset	486 # rather than "zero bytes", so special case that.
8ed8dfbeabb9 mmap: add a size argument to mmapread Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44021 diff changeset	487 return b''
8ed8dfbeabb9 mmap: add a size argument to mmapread Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44021 diff changeset	488 elif size is None:
8ed8dfbeabb9 mmap: add a size argument to mmapread Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44021 diff changeset	489 size = 0
48010 ae79611e3115 util: avoid a name-error warning in the `mmapread` exception handler Matt Harbison <matt_harbison@yahoo.com> parents: 47622 diff changeset	490 fd = getattr(fp, 'fileno', lambda: fp)()
51926 bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51885 diff changeset	491
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51885 diff changeset	492 if pycompat.iswindows:
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51885 diff changeset	493 _mmap = lambda fd, size: mmap.mmap(fd, size, access=mmap.ACCESS_READ)
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51885 diff changeset	494 else:
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51885 diff changeset	495 flags = mmap.MAP_PRIVATE
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51885 diff changeset	496 bg_populate = hasattr(osutil, "background_mmap_populate")
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51885 diff changeset	497
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51885 diff changeset	498 if pre_populate and not bg_populate:
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51885 diff changeset	499 flags \|= getattr(mmap, 'MAP_POPULATE', 0)
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51885 diff changeset	500
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51885 diff changeset	501 def _mmap(fd, size) -> mmap.mmap:
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51885 diff changeset	502 m = mmap.mmap(fd, size, flags=flags, prot=mmap.PROT_READ)
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51885 diff changeset	503 if pre_populate and bg_populate:
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51885 diff changeset	504 osutil.background_mmap_populate(m)
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51885 diff changeset	505 return m
bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51885 diff changeset	506
34295 3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34139 diff changeset	507 try:
51926 bc9ed92d4753 util: make `mmapread()` work on Windows again Matt Harbison <matt_harbison@yahoo.com> parents: 51885 diff changeset	508 return _mmap(fd, size)
34295 3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34139 diff changeset	509 except ValueError:
3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34139 diff changeset	510 # Empty files cannot be mmapped, but mmapread should still work. Check
3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34139 diff changeset	511 # if the file is empty, and if so, return an empty buffer.
3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34139 diff changeset	512 if os.fstat(fd).st_size == 0:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	513 return b''
34295 3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34139 diff changeset	514 raise
3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34139 diff changeset	515
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	516
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48940 diff changeset	517 class fileobjectproxy:
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	518 """A proxy around file objects that tells a watcher when events occur.
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	519
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	520 This type is intended to only be used for testing purposes. Think hard
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	521 before using it in important code.
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	522 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	523
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	524 __slots__ = (
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	525 '_orig',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	526 '_observer',
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	527 )
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	528
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	529 def __init__(self, fh, observer):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	530 object.__setattr__(self, '_orig', fh)
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	531 object.__setattr__(self, '_observer', observer)
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	532
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	533 def __getattribute__(self, name):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	534 ours = {
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	535 '_observer',
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	536 # IOBase
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	537 'close',
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	538 # closed if a property
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	539 'fileno',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	540 'flush',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	541 'isatty',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	542 'readable',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	543 'readline',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	544 'readlines',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	545 'seek',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	546 'seekable',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	547 'tell',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	548 'truncate',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	549 'writable',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	550 'writelines',
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	551 # RawIOBase
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	552 'read',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	553 'readall',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	554 'readinto',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	555 'write',
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	556 # BufferedIOBase
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	557 # raw is a property
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	558 'detach',
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	559 # read defined above
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	560 'read1',
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	561 # readinto defined above
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	562 # write defined above
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	563 }
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	564
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	565 # We only observe some methods.
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	566 if name in ours:
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	567 return object.__getattribute__(self, name)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	568
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	569 return getattr(object.__getattribute__(self, '_orig'), name)
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	570
36832 6bdea0efdab5 util: forward __bool__()/__nonzero__() on fileobjectproxy Matt Harbison <matt_harbison@yahoo.com> parents: 36793 diff changeset	571 def __nonzero__(self):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	572 return bool(object.__getattribute__(self, '_orig'))
36832 6bdea0efdab5 util: forward __bool__()/__nonzero__() on fileobjectproxy Matt Harbison <matt_harbison@yahoo.com> parents: 36793 diff changeset	573
6bdea0efdab5 util: forward __bool__()/__nonzero__() on fileobjectproxy Matt Harbison <matt_harbison@yahoo.com> parents: 36793 diff changeset	574 __bool__ = __nonzero__
6bdea0efdab5 util: forward __bool__()/__nonzero__() on fileobjectproxy Matt Harbison <matt_harbison@yahoo.com> parents: 36793 diff changeset	575
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	576 def __delattr__(self, name):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	577 return delattr(object.__getattribute__(self, '_orig'), name)
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	578
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	579 def __setattr__(self, name, value):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	580 return setattr(object.__getattribute__(self, '_orig'), name, value)
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	581
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	582 def __iter__(self):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	583 return object.__getattribute__(self, '_orig').__iter__()
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	584
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	585 def _observedcall(self, name, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	586 # Call the original object.
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	587 orig = object.__getattribute__(self, '_orig')
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	588 res = getattr(orig, name)(args, *kwargs)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	589
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	590 # Call a method on the observer of the same name with arguments
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	591 # so it can react, log, etc.
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	592 observer = object.__getattribute__(self, '_observer')
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	593 fn = getattr(observer, name, None)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	594 if fn:
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	595 fn(res, args, *kwargs)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	596
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	597 return res
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	598
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	599 def close(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	600 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	601 'close', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	602 )
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	603
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	604 def fileno(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	605 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	606 'fileno', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	607 )
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	608
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	609 def flush(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	610 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	611 'flush', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	612 )
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	613
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	614 def isatty(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	615 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	616 'isatty', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	617 )
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	618
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	619 def readable(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	620 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	621 'readable', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	622 )
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	623
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	624 def readline(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	625 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	626 'readline', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	627 )
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	628
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	629 def readlines(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	630 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	631 'readlines', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	632 )
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	633
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	634 def seek(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	635 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	636 'seek', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	637 )
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	638
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	639 def seekable(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	640 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	641 'seekable', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	642 )
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	643
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	644 def tell(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	645 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	646 'tell', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	647 )
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	648
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	649 def truncate(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	650 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	651 'truncate', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	652 )
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	653
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	654 def writable(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	655 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	656 'writable', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	657 )
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	658
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	659 def writelines(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	660 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	661 'writelines', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	662 )
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	663
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	664 def read(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	665 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	666 'read', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	667 )
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	668
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	669 def readall(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	670 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	671 'readall', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	672 )
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	673
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	674 def readinto(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	675 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	676 'readinto', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	677 )
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	678
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	679 def write(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	680 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	681 'write', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	682 )
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	683
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	684 def detach(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	685 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	686 'detach', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	687 )
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	688
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	689 def read1(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	690 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	691 'read1', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	692 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	693
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	694
36525 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	695 class observedbufferedinputpipe(bufferedinputpipe):
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	696 """A variation of bufferedinputpipe that is aware of fileobjectproxy.
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	697
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	698 ``bufferedinputpipe`` makes low-level calls to ``os.read()`` that
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	699 bypass ``fileobjectproxy``. Because of this, we need to make
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	700 ``bufferedinputpipe`` aware of these operations.
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	701
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	702 This variation of ``bufferedinputpipe`` can notify observers about
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	703 ``os.read()`` events. It also re-publishes other events, such as
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	704 ``read()`` and ``readline()``.
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	705 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	706
49814 1d1b244a91b6 util: fix the signature of observedbufferedinputpipe._fillbuffer() Matt Harbison <matt_harbison@yahoo.com> parents: 49813 diff changeset	707 def _fillbuffer(self, size=_chunksize):
1d1b244a91b6 util: fix the signature of observedbufferedinputpipe._fillbuffer() Matt Harbison <matt_harbison@yahoo.com> parents: 49813 diff changeset	708 res = super(observedbufferedinputpipe, self)._fillbuffer(size=size)
36525 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	709
43103 c95b2f40db7c py3: stop normalizing 2nd argument of attr() to unicode Gregory Szorc <gregory.szorc@gmail.com>* parents: 43090 diff changeset	710 fn = getattr(self._input._observer, 'osread', None)
36525 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	711 if fn:
49814 1d1b244a91b6 util: fix the signature of observedbufferedinputpipe._fillbuffer() Matt Harbison <matt_harbison@yahoo.com> parents: 49813 diff changeset	712 fn(res, size)
36525 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	713
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	714 return res
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	715
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	716 # We use different observer methods because the operation isn't
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	717 # performed on the actual file object but on us.
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	718 def read(self, size):
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	719 res = super(observedbufferedinputpipe, self).read(size)
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	720
43103 c95b2f40db7c py3: stop normalizing 2nd argument of attr() to unicode Gregory Szorc <gregory.szorc@gmail.com>* parents: 43090 diff changeset	721 fn = getattr(self._input._observer, 'bufferedread', None)
36525 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	722 if fn:
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	723 fn(res, size)
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	724
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	725 return res
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	726
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	727 def readline(self, args, *kwargs):
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	728 res = super(observedbufferedinputpipe, self).readline(args, *kwargs)
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	729
43103 c95b2f40db7c py3: stop normalizing 2nd argument of attr() to unicode Gregory Szorc <gregory.szorc@gmail.com>* parents: 43090 diff changeset	730 fn = getattr(self._input._observer, 'bufferedreadline', None)
36525 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	731 if fn:
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	732 fn(res)
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	733
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	734 return res
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	735
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	736
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	737 PROXIED_SOCKET_METHODS = {
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	738 'makefile',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	739 'recv',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	740 'recvfrom',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	741 'recvfrom_into',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	742 'recv_into',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	743 'send',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	744 'sendall',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	745 'sendto',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	746 'setblocking',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	747 'settimeout',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	748 'gettimeout',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	749 'setsockopt',
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	750 }
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	751
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	752
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48940 diff changeset	753 class socketproxy:
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	754 """A proxy around a socket that tells a watcher when events occur.
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	755
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	756 This is like ``fileobjectproxy`` except for sockets.
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	757
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	758 This type is intended to only be used for testing purposes. Think hard
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	759 before using it in important code.
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	760 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	761
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	762 __slots__ = (
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	763 '_orig',
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	764 '_observer',
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	765 )
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	766
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	767 def __init__(self, sock, observer):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	768 object.__setattr__(self, '_orig', sock)
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	769 object.__setattr__(self, '_observer', observer)
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	770
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	771 def __getattribute__(self, name):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	772 if name in PROXIED_SOCKET_METHODS:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	773 return object.__getattribute__(self, name)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	774
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	775 return getattr(object.__getattribute__(self, '_orig'), name)
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	776
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	777 def __delattr__(self, name):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	778 return delattr(object.__getattribute__(self, '_orig'), name)
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	779
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	780 def __setattr__(self, name, value):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	781 return setattr(object.__getattribute__(self, '_orig'), name, value)
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	782
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	783 def __nonzero__(self):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	784 return bool(object.__getattribute__(self, '_orig'))
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	785
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	786 __bool__ = __nonzero__
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	787
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	788 def _observedcall(self, name, args, *kwargs):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	789 # Call the original object.
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	790 orig = object.__getattribute__(self, '_orig')
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	791 res = getattr(orig, name)(args, *kwargs)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	792
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	793 # Call a method on the observer of the same name with arguments
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	794 # so it can react, log, etc.
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	795 observer = object.__getattribute__(self, '_observer')
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	796 fn = getattr(observer, name, None)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	797 if fn:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	798 fn(res, args, *kwargs)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	799
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	800 return res
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	801
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	802 def makefile(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	803 res = object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	804 'makefile', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	805 )
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	806
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	807 # The file object may be used for I/O. So we turn it into a
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	808 # proxy using our observer.
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	809 observer = object.__getattribute__(self, '_observer')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	810 return makeloggingfileobject(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	811 observer.fh,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	812 res,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	813 observer.name,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	814 reads=observer.reads,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	815 writes=observer.writes,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	816 logdata=observer.logdata,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	817 logdataapis=observer.logdataapis,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	818 )
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	819
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	820 def recv(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	821 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	822 'recv', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	823 )
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	824
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	825 def recvfrom(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	826 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	827 'recvfrom', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	828 )
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	829
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	830 def recvfrom_into(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	831 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	832 'recvfrom_into', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	833 )
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	834
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	835 def recv_into(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	836 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	837 'recv_info', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	838 )
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	839
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	840 def send(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	841 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	842 'send', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	843 )
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	844
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	845 def sendall(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	846 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	847 'sendall', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	848 )
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	849
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	850 def sendto(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	851 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	852 'sendto', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	853 )
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	854
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	855 def setblocking(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	856 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	857 'setblocking', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	858 )
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	859
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	860 def settimeout(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	861 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	862 'settimeout', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	863 )
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	864
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	865 def gettimeout(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	866 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	867 'gettimeout', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	868 )
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	869
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	870 def setsockopt(self, args, *kwargs):
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	871 return object.__getattribute__(self, '_observedcall')(
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	872 'setsockopt', args, *kwargs
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	873 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	874
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	875
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48940 diff changeset	876 class baseproxyobserver:
43920 4222b9d5d4fb util: move common proxyobserver attributes to the base class Matt Harbison <matt_harbison@yahoo.com> parents: 43907 diff changeset	877 def __init__(self, fh, name, logdata, logdataapis):
4222b9d5d4fb util: move common proxyobserver attributes to the base class Matt Harbison <matt_harbison@yahoo.com> parents: 43907 diff changeset	878 self.fh = fh
4222b9d5d4fb util: move common proxyobserver attributes to the base class Matt Harbison <matt_harbison@yahoo.com> parents: 43907 diff changeset	879 self.name = name
4222b9d5d4fb util: move common proxyobserver attributes to the base class Matt Harbison <matt_harbison@yahoo.com> parents: 43907 diff changeset	880 self.logdata = logdata
4222b9d5d4fb util: move common proxyobserver attributes to the base class Matt Harbison <matt_harbison@yahoo.com> parents: 43907 diff changeset	881 self.logdataapis = logdataapis
4222b9d5d4fb util: move common proxyobserver attributes to the base class Matt Harbison <matt_harbison@yahoo.com> parents: 43907 diff changeset	882
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	883 def _writedata(self, data):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	884 if not self.logdata:
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	885 if self.logdataapis:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	886 self.fh.write(b'\n')
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	887 self.fh.flush()
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	888 return
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	889
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	890 # Simple case writes all data on a single line.
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	891 if b'\n' not in data:
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	892 if self.logdataapis:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	893 self.fh.write(b': %s\n' % stringutil.escapestr(data))
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	894 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	895 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	896 b'%s> %s\n' % (self.name, stringutil.escapestr(data))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	897 )
36991 d683c7367989 wireproto: explicitly flush stdio to prevent stalls on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 36958 diff changeset	898 self.fh.flush()
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	899 return
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	900
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	901 # Data with newlines is written to multiple lines.
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	902 if self.logdataapis:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	903 self.fh.write(b':\n')
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	904
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	905 lines = data.splitlines(True)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	906 for line in lines:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	907 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	908 b'%s> %s\n' % (self.name, stringutil.escapestr(line))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	909 )
36991 d683c7367989 wireproto: explicitly flush stdio to prevent stalls on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 36958 diff changeset	910 self.fh.flush()
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	911
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	912
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	913 class fileobjectobserver(baseproxyobserver):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	914 """Logs file object activity."""
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	915
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	916 def __init__(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	917 self, fh, name, reads=True, writes=True, logdata=False, logdataapis=True
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	918 ):
43920 4222b9d5d4fb util: move common proxyobserver attributes to the base class Matt Harbison <matt_harbison@yahoo.com> parents: 43907 diff changeset	919 super(fileobjectobserver, self).__init__(fh, name, logdata, logdataapis)
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	920 self.reads = reads
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	921 self.writes = writes
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	922
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	923 def read(self, res, size=-1):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	924 if not self.reads:
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	925 return
36585 26a6b62919e2 util: work around Python 3 returning None at EOF instead of '' Augie Fackler <augie@google.com> parents: 36584 diff changeset	926 # Python 3 can return None from reads at EOF instead of empty strings.
26a6b62919e2 util: work around Python 3 returning None at EOF instead of '' Augie Fackler <augie@google.com> parents: 36584 diff changeset	927 if res is None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	928 res = b''
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	929
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	930 if size == -1 and res == b'':
38314 565074cc9ac6 tests: suppress read(-1) -> '' calls in fileobjectobserver Augie Fackler <augie@google.com> parents: 38164 diff changeset	931 # Suppress pointless read(-1) calls that return
565074cc9ac6 tests: suppress read(-1) -> '' calls in fileobjectobserver Augie Fackler <augie@google.com> parents: 38164 diff changeset	932 # nothing. These happen _a lot_ on Python 3, and there
565074cc9ac6 tests: suppress read(-1) -> '' calls in fileobjectobserver Augie Fackler <augie@google.com> parents: 38164 diff changeset	933 # doesn't seem to be a better workaround to have matching
565074cc9ac6 tests: suppress read(-1) -> '' calls in fileobjectobserver Augie Fackler <augie@google.com> parents: 38164 diff changeset	934 # Python 2 and 3 behavior. :(
565074cc9ac6 tests: suppress read(-1) -> '' calls in fileobjectobserver Augie Fackler <augie@google.com> parents: 38164 diff changeset	935 return
565074cc9ac6 tests: suppress read(-1) -> '' calls in fileobjectobserver Augie Fackler <augie@google.com> parents: 38164 diff changeset	936
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	937 if self.logdataapis:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	938 self.fh.write(b'%s> read(%d) -> %d' % (self.name, size, len(res)))
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	939
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	940 self._writedata(res)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	941
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	942 def readline(self, res, limit=-1):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	943 if not self.reads:
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	944 return
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	945
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	946 if self.logdataapis:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	947 self.fh.write(b'%s> readline() -> %d' % (self.name, len(res)))
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	948
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	949 self._writedata(res)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	950
36630 29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36629 diff changeset	951 def readinto(self, res, dest):
29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36629 diff changeset	952 if not self.reads:
29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36629 diff changeset	953 return
29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36629 diff changeset	954
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	955 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	956 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	957 b'%s> readinto(%d) -> %r' % (self.name, len(dest), res)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	958 )
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	959
36630 29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36629 diff changeset	960 data = dest[0:res] if res is not None else b''
41384 b141b5243b37 util: cast memoryview to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 41289 diff changeset	961
b141b5243b37 util: cast memoryview to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 41289 diff changeset	962 # _writedata() uses "in" operator and is confused by memoryview because
b141b5243b37 util: cast memoryview to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 41289 diff changeset	963 # characters are ints on Python 3.
b141b5243b37 util: cast memoryview to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 41289 diff changeset	964 if isinstance(data, memoryview):
b141b5243b37 util: cast memoryview to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 41289 diff changeset	965 data = data.tobytes()
b141b5243b37 util: cast memoryview to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 41289 diff changeset	966
36630 29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36629 diff changeset	967 self._writedata(data)
29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36629 diff changeset	968
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	969 def write(self, res, data):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	970 if not self.writes:
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	971 return
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	972
36631 8395fddde46c util: report integer result from write() Gregory Szorc <gregory.szorc@gmail.com> parents: 36630 diff changeset	973 # Python 2 returns None from some write() calls. Python 3 (reasonably)
8395fddde46c util: report integer result from write() Gregory Szorc <gregory.szorc@gmail.com> parents: 36630 diff changeset	974 # returns the integer bytes written.
8395fddde46c util: report integer result from write() Gregory Szorc <gregory.szorc@gmail.com> parents: 36630 diff changeset	975 if res is None and data:
8395fddde46c util: report integer result from write() Gregory Szorc <gregory.szorc@gmail.com> parents: 36630 diff changeset	976 res = len(data)
8395fddde46c util: report integer result from write() Gregory Szorc <gregory.szorc@gmail.com> parents: 36630 diff changeset	977
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	978 if self.logdataapis:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	979 self.fh.write(b'%s> write(%d) -> %r' % (self.name, len(data), res))
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	980
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	981 self._writedata(data)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	982
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	983 def flush(self, res):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	984 if not self.writes:
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	985 return
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	986
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	987 self.fh.write(b'%s> flush() -> %r\n' % (self.name, res))
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	988
36525 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	989 # For observedbufferedinputpipe.
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	990 def bufferedread(self, res, size):
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	991 if not self.reads:
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	992 return
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	993
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	994 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	995 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	996 b'%s> bufferedread(%d) -> %d' % (self.name, size, len(res))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	997 )
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	998
36525 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	999 self._writedata(res)
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	1000
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	1001 def bufferedreadline(self, res):
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	1002 if not self.reads:
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	1003 return
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	1004
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	1005 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1006 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1007 b'%s> bufferedreadline() -> %d' % (self.name, len(res))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1008 )
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	1009
36525 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	1010 self._writedata(res)
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36524 diff changeset	1011
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1012
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1013 def makeloggingfileobject(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1014 logh, fh, name, reads=True, writes=True, logdata=False, logdataapis=True
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1015 ):
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	1016 """Turn a file object into a logging file object."""
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	1017
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1018 observer = fileobjectobserver(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1019 logh,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1020 name,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1021 reads=reads,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1022 writes=writes,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1023 logdata=logdata,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1024 logdataapis=logdataapis,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1025 )
36524 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	1026 return fileobjectproxy(fh, observer)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36432 diff changeset	1027
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1028
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1029 class socketobserver(baseproxyobserver):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1030 """Logs socket activity."""
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1031
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1032 def __init__(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1033 self,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1034 fh,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1035 name,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1036 reads=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1037 writes=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1038 states=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1039 logdata=False,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1040 logdataapis=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1041 ):
43920 4222b9d5d4fb util: move common proxyobserver attributes to the base class Matt Harbison <matt_harbison@yahoo.com> parents: 43907 diff changeset	1042 super(socketobserver, self).__init__(fh, name, logdata, logdataapis)
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1043 self.reads = reads
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1044 self.writes = writes
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1045 self.states = states
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1046
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1047 def makefile(self, res, mode=None, bufsize=None):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1048 if not self.states:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1049 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1050
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1051 self.fh.write(b'%s> makefile(%r, %r)\n' % (self.name, mode, bufsize))
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1052
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1053 def recv(self, res, size, flags=0):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1054 if not self.reads:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1055 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1056
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	1057 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1058 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1059 b'%s> recv(%d, %d) -> %d' % (self.name, size, flags, len(res))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1060 )
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1061 self._writedata(res)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1062
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1063 def recvfrom(self, res, size, flags=0):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1064 if not self.reads:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1065 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1066
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	1067 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1068 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1069 b'%s> recvfrom(%d, %d) -> %d'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1070 % (self.name, size, flags, len(res[0]))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1071 )
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	1072
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1073 self._writedata(res[0])
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1074
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1075 def recvfrom_into(self, res, buf, size, flags=0):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1076 if not self.reads:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1077 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1078
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	1079 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1080 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1081 b'%s> recvfrom_into(%d, %d) -> %d'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1082 % (self.name, size, flags, res[0])
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1083 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1084
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1085 self._writedata(buf[0 : res[0]])
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1086
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1087 def recv_into(self, res, buf, size=0, flags=0):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1088 if not self.reads:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1089 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1090
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	1091 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1092 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1093 b'%s> recv_into(%d, %d) -> %d' % (self.name, size, flags, res)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1094 )
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	1095
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1096 self._writedata(buf[0:res])
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1097
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1098 def send(self, res, data, flags=0):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1099 if not self.writes:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1100 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1101
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1102 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1103 b'%s> send(%d, %d) -> %d' % (self.name, len(data), flags, len(res))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1104 )
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1105 self._writedata(data)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1106
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1107 def sendall(self, res, data, flags=0):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1108 if not self.writes:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1109 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1110
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	1111 if self.logdataapis:
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	1112 # Returns None on success. So don't bother reporting return value.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1113 self.fh.write(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1114 b'%s> sendall(%d, %d)' % (self.name, len(data), flags)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1115 )
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	1116
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1117 self._writedata(data)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1118
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1119 def sendto(self, res, data, flagsoraddress, address=None):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1120 if not self.writes:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1121 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1122
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1123 if address:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1124 flags = flagsoraddress
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1125 else:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1126 flags = 0
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1127
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	1128 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1129 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1130 b'%s> sendto(%d, %d, %r) -> %d'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1131 % (self.name, len(data), flags, address, res)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1132 )
37044 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37010 diff changeset	1133
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1134 self._writedata(data)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1135
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1136 def setblocking(self, res, flag):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1137 if not self.states:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1138 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1139
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1140 self.fh.write(b'%s> setblocking(%r)\n' % (self.name, flag))
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1141
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1142 def settimeout(self, res, value):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1143 if not self.states:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1144 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1145
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1146 self.fh.write(b'%s> settimeout(%r)\n' % (self.name, value))
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1147
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1148 def gettimeout(self, res):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1149 if not self.states:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1150 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1151
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1152 self.fh.write(b'%s> gettimeout() -> %f\n' % (self.name, res))
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1153
39060 ed8160e4fea0 util: fix signature of setsockopt in socket observer Augie Fackler <augie@google.com> parents: 38812 diff changeset	1154 def setsockopt(self, res, level, optname, value):
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1155 if not self.states:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1156 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1157
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1158 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1159 b'%s> setsockopt(%r, %r, %r) -> %r\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1160 % (self.name, level, optname, value, res)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1161 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1162
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1163
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1164 def makeloggingsocket(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1165 logh,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1166 fh,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1167 name,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1168 reads=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1169 writes=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1170 states=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1171 logdata=False,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1172 logdataapis=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1173 ):
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1174 """Turn a socket into a logging socket."""
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1175
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1176 observer = socketobserver(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1177 logh,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1178 name,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1179 reads=reads,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1180 writes=writes,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1181 states=states,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1182 logdata=logdata,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1183 logdataapis=logdataapis,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1184 )
37010 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1185 return socketproxy(fh, observer)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36991 diff changeset	1186
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1187
7632 9626819b2e3d refactor version code Matt Mackall <mpm@selenic.com> parents: 7559 diff changeset	1188 def version():
9626819b2e3d refactor version code Matt Mackall <mpm@selenic.com> parents: 7559 diff changeset	1189 """Return version information if available."""
9626819b2e3d refactor version code Matt Mackall <mpm@selenic.com> parents: 7559 diff changeset	1190 try:
51727 92845af308b4 typing: narrow the scope of some recent disabled import warnings Matt Harbison <matt_harbison@yahoo.com> parents: 51725 diff changeset	1191 from . import __version__ # pytype: disable=import-error
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1192
7632 9626819b2e3d refactor version code Matt Mackall <mpm@selenic.com> parents: 7559 diff changeset	1193 return __version__.version
9626819b2e3d refactor version code Matt Mackall <mpm@selenic.com> parents: 7559 diff changeset	1194 except ImportError:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1195 return b'unknown'
7632 9626819b2e3d refactor version code Matt Mackall <mpm@selenic.com> parents: 7559 diff changeset	1196
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1197
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1198 def versiontuple(v=None, n=4):
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1199 """Parses a Mercurial version string into an N-tuple.
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1200
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1201 The version string to be parsed is specified with the ``v`` argument.
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1202 If it isn't defined, the current Mercurial version string will be parsed.
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1203
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1204 ``n`` can be 2, 3, or 4. Here is how some version strings map to
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1205 returned values:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1206
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	1207 >>> v = b'3.6.1+190-df9b73d2d444'
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1208 >>> versiontuple(v, 2)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1209 (3, 6)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1210 >>> versiontuple(v, 3)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1211 (3, 6, 1)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1212 >>> versiontuple(v, 4)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1213 (3, 6, 1, '190-df9b73d2d444')
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1214
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	1215 >>> versiontuple(b'3.6.1+190-df9b73d2d444+20151118')
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1216 (3, 6, 1, '190-df9b73d2d444+20151118')
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1217
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	1218 >>> v = b'3.6'
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1219 >>> versiontuple(v, 2)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1220 (3, 6)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1221 >>> versiontuple(v, 3)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1222 (3, 6, None)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1223 >>> versiontuple(v, 4)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1224 (3, 6, None, None)
29613 616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1225
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	1226 >>> v = b'3.9-rc'
29613 616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1227 >>> versiontuple(v, 2)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1228 (3, 9)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1229 >>> versiontuple(v, 3)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1230 (3, 9, None)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1231 >>> versiontuple(v, 4)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1232 (3, 9, None, 'rc')
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1233
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	1234 >>> v = b'3.9-rc+2-02a8fea4289b'
29613 616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1235 >>> versiontuple(v, 2)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1236 (3, 9)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1237 >>> versiontuple(v, 3)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1238 (3, 9, None)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1239 >>> versiontuple(v, 4)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1240 (3, 9, None, 'rc+2-02a8fea4289b')
37801 5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1241
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1242 >>> versiontuple(b'4.6rc0')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1243 (4, 6, None, 'rc0')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1244 >>> versiontuple(b'4.6rc0+12-425d55e54f98')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1245 (4, 6, None, 'rc0+12-425d55e54f98')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1246 >>> versiontuple(b'.1.2.3')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1247 (None, None, None, '.1.2.3')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1248 >>> versiontuple(b'12.34..5')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1249 (12, 34, None, '..5')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1250 >>> versiontuple(b'1.2.3.4.5.6')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1251 (1, 2, 3, '.4.5.6')
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1252 """
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1253 if not v:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1254 v = version()
44021 6d3b67a837a6 cleanup: drop redundant character escapes from `[]` character sets Matt Harbison <matt_harbison@yahoo.com> parents: 44018 diff changeset	1255 m = remod.match(br'(\d+(?:\.\d+){,2})[+-]?(.*)', v)
37801 5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1256 if not m:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1257 vparts, extra = b'', v
37801 5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1258 elif m.group(2):
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1259 vparts, extra = m.groups()
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1260 else:
37801 5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1261 vparts, extra = m.group(1), None
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1262
43882 a21a6dad4b38 typing: add an assertion to util.versiontuple Matt Harbison <matt_harbison@yahoo.com> parents: 43881 diff changeset	1263 assert vparts is not None # help pytype
a21a6dad4b38 typing: add an assertion to util.versiontuple Matt Harbison <matt_harbison@yahoo.com> parents: 43881 diff changeset	1264
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1265 vints = []
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1266 for i in vparts.split(b'.'):
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1267 try:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1268 vints.append(int(i))
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1269 except ValueError:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1270 break
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1271 # (3, 6) -> (3, 6, None)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1272 while len(vints) < 3:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1273 vints.append(None)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1274
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1275 if n == 2:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1276 return (vints[0], vints[1])
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1277 if n == 3:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1278 return (vints[0], vints[1], vints[2])
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1279 if n == 4:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1280 return (vints[0], vints[1], vints[2], extra)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1281
48013 376d08ae904f util: eliminate the possibility of returning None from `versiontuple()` Matt Harbison <matt_harbison@yahoo.com> parents: 48010 diff changeset	1282 raise error.ProgrammingError(b"invalid version part request: %d" % n)
376d08ae904f util: eliminate the possibility of returning None from `versiontuple()` Matt Harbison <matt_harbison@yahoo.com> parents: 48010 diff changeset	1283
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1284
3145 e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1285 def cachefunc(func):
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1286 '''cache the result of function calls'''
3147 97420a49188d add comments in cachefunc Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 3145 diff changeset	1287 # XXX doesn't handle keywords args
28832 f5ff10f6fa6b util: use __code__ (available since py2.6) timeless <timeless@mozdev.org> parents: 28826 diff changeset	1288 if func.__code__.co_argcount == 0:
43880 eff050dbb703 util: rename a variable to avoid confusing pytype Matt Harbison <matt_harbison@yahoo.com> parents: 43879 diff changeset	1289 listcache = []
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1290
20835 0e8417131a29 util: add the code path to "cachefunc()" for the function taking no arguments FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20679 diff changeset	1291 def f():
43880 eff050dbb703 util: rename a variable to avoid confusing pytype Matt Harbison <matt_harbison@yahoo.com> parents: 43879 diff changeset	1292 if len(listcache) == 0:
eff050dbb703 util: rename a variable to avoid confusing pytype Matt Harbison <matt_harbison@yahoo.com> parents: 43879 diff changeset	1293 listcache.append(func())
eff050dbb703 util: rename a variable to avoid confusing pytype Matt Harbison <matt_harbison@yahoo.com> parents: 43879 diff changeset	1294 return listcache[0]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1295
20835 0e8417131a29 util: add the code path to "cachefunc()" for the function taking no arguments FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20679 diff changeset	1296 return f
3145 e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1297 cache = {}
28832 f5ff10f6fa6b util: use __code__ (available since py2.6) timeless <timeless@mozdev.org> parents: 28826 diff changeset	1298 if func.__code__.co_argcount == 1:
3147 97420a49188d add comments in cachefunc Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 3145 diff changeset	1299 # we gain a small amount of time because
97420a49188d add comments in cachefunc Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 3145 diff changeset	1300 # we don't need to pack/unpack the list
3145 e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1301 def f(arg):
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1302 if arg not in cache:
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1303 cache[arg] = func(arg)
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1304 return cache[arg]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1305
3145 e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1306 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1307
3145 e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1308 def f(*args):
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1309 if args not in cache:
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1310 cache[args] = func(*args)
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1311 return cache[args]
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1312
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1313 return f
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1314
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1315
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48940 diff changeset	1316 class cow:
34357 c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1317 """helper class to make copy-on-write easier
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1318
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1319 Call preparewrite before doing any writes.
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1320 """
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1321
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1322 def preparewrite(self):
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1323 """call this before writes, return self or a copied new object"""
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1324 if getattr(self, '_copied', 0):
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1325 self._copied -= 1
46901 51841b23670b typing: make minor adjustments to mercurial/util.py to pass pytype checking Matt Harbison <matt_harbison@yahoo.com> parents: 46900 diff changeset	1326 # Function cow.__init__ expects 1 arg(s), got 2 [wrong-arg-count]
51841b23670b typing: make minor adjustments to mercurial/util.py to pass pytype checking Matt Harbison <matt_harbison@yahoo.com> parents: 46900 diff changeset	1327 return self.__class__(self) # pytype: disable=wrong-arg-count
34357 c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1328 return self
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1329
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1330 def copy(self):
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1331 """always do a cheap copy"""
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1332 self._copied = getattr(self, '_copied', 0) + 1
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1333 return self
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1334
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1335
32300 bd0fd3ff9916 util: rewrite sortdict using Python 2.7's OrderedDict Martin von Zweigbergk <martinvonz@google.com> parents: 32291 diff changeset	1336 class sortdict(collections.OrderedDict):
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	1337 """a simple sorted dictionary
32306 2d19664e257d util: drop unneeded override, sortdict.copy() Yuya Nishihara <yuya@tcha.org> parents: 32300 diff changeset	1338
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	1339 >>> d1 = sortdict([(b'a', 0), (b'b', 1)])
32306 2d19664e257d util: drop unneeded override, sortdict.copy() Yuya Nishihara <yuya@tcha.org> parents: 32300 diff changeset	1340 >>> d2 = d1.copy()
50750 a2df74853f8d tests: fix sortdict doctest with Python 3.12 Mads Kiilerich <mads@kiilerich.com> parents: 50686 diff changeset	1341 >>> list(d2.items())
a2df74853f8d tests: fix sortdict doctest with Python 3.12 Mads Kiilerich <mads@kiilerich.com> parents: 50686 diff changeset	1342 [('a', 0), ('b', 1)]
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	1343 >>> d2.update([(b'a', 2)])
34139 be00af4a1ac5 doctest: coerce dict.keys() to list Yuya Nishihara <yuya@tcha.org> parents: 34137 diff changeset	1344 >>> list(d2.keys()) # should still be in last-set order
32306 2d19664e257d util: drop unneeded override, sortdict.copy() Yuya Nishihara <yuya@tcha.org> parents: 32300 diff changeset	1345 ['b', 'a']
43907 68af0228fedc util: implement sortdict.insert() Martin von Zweigbergk <martinvonz@google.com> parents: 43883 diff changeset	1346 >>> d1.insert(1, b'a.5', 0.5)
50750 a2df74853f8d tests: fix sortdict doctest with Python 3.12 Mads Kiilerich <mads@kiilerich.com> parents: 50686 diff changeset	1347 >>> list(d1.items())
a2df74853f8d tests: fix sortdict doctest with Python 3.12 Mads Kiilerich <mads@kiilerich.com> parents: 50686 diff changeset	1348 [('a', 0), ('a.5', 0.5), ('b', 1)]
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	1349 """
32306 2d19664e257d util: drop unneeded override, sortdict.copy() Yuya Nishihara <yuya@tcha.org> parents: 32300 diff changeset	1350
32300 bd0fd3ff9916 util: rewrite sortdict using Python 2.7's OrderedDict Martin von Zweigbergk <martinvonz@google.com> parents: 32291 diff changeset	1351 def __setitem__(self, key, value):
21813 c2262004c2e2 config: move config.sortdict class into util Angel Ezquerra <angel.ezquerra@gmail.com> parents: 21046 diff changeset	1352 if key in self:
32300 bd0fd3ff9916 util: rewrite sortdict using Python 2.7's OrderedDict Martin von Zweigbergk <martinvonz@google.com> parents: 32291 diff changeset	1353 del self[key]
bd0fd3ff9916 util: rewrite sortdict using Python 2.7's OrderedDict Martin von Zweigbergk <martinvonz@google.com> parents: 32291 diff changeset	1354 super(sortdict, self).__setitem__(key, value)
21813 c2262004c2e2 config: move config.sortdict class into util Angel Ezquerra <angel.ezquerra@gmail.com> parents: 21046 diff changeset	1355
33626 524b13fc711f util: fix sortdict.update() to call __setitem__() on PyPy (issue5639) Yuya Nishihara <yuya@tcha.org> parents: 33549 diff changeset	1356 if pycompat.ispypy:
524b13fc711f util: fix sortdict.update() to call __setitem__() on PyPy (issue5639) Yuya Nishihara <yuya@tcha.org> parents: 33549 diff changeset	1357 # __setitem__() isn't called as of PyPy 5.8.0
46900 64400d05db1e util: fix the signature for the pypy override of sortdict.update() Matt Harbison <matt_harbison@yahoo.com> parents: 46827 diff changeset	1358 def update(self, src, **f):
33626 524b13fc711f util: fix sortdict.update() to call __setitem__() on PyPy (issue5639) Yuya Nishihara <yuya@tcha.org> parents: 33549 diff changeset	1359 if isinstance(src, dict):
48913 f254fc73d956 global: bulk replace simple pycompat.iteritems(x) with x.items() Gregory Szorc <gregory.szorc@gmail.com> parents: 48897 diff changeset	1360 src = src.items()
33626 524b13fc711f util: fix sortdict.update() to call __setitem__() on PyPy (issue5639) Yuya Nishihara <yuya@tcha.org> parents: 33549 diff changeset	1361 for k, v in src:
524b13fc711f util: fix sortdict.update() to call __setitem__() on PyPy (issue5639) Yuya Nishihara <yuya@tcha.org> parents: 33549 diff changeset	1362 self[k] = v
46900 64400d05db1e util: fix the signature for the pypy override of sortdict.update() Matt Harbison <matt_harbison@yahoo.com> parents: 46827 diff changeset	1363 for k in f:
64400d05db1e util: fix the signature for the pypy override of sortdict.update() Matt Harbison <matt_harbison@yahoo.com> parents: 46827 diff changeset	1364 self[k] = f[k]
33626 524b13fc711f util: fix sortdict.update() to call __setitem__() on PyPy (issue5639) Yuya Nishihara <yuya@tcha.org> parents: 33549 diff changeset	1365
43907 68af0228fedc util: implement sortdict.insert() Martin von Zweigbergk <martinvonz@google.com> parents: 43883 diff changeset	1366 def insert(self, position, key, value):
51699 ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51698 diff changeset	1367 for i, (k, v) in enumerate(list(self.items())):
43907 68af0228fedc util: implement sortdict.insert() Martin von Zweigbergk <martinvonz@google.com> parents: 43883 diff changeset	1368 if i == position:
68af0228fedc util: implement sortdict.insert() Martin von Zweigbergk <martinvonz@google.com> parents: 43883 diff changeset	1369 self[key] = value
68af0228fedc util: implement sortdict.insert() Martin von Zweigbergk <martinvonz@google.com> parents: 43883 diff changeset	1370 if i >= position:
68af0228fedc util: implement sortdict.insert() Martin von Zweigbergk <martinvonz@google.com> parents: 43883 diff changeset	1371 del self[k]
68af0228fedc util: implement sortdict.insert() Martin von Zweigbergk <martinvonz@google.com> parents: 43883 diff changeset	1372 self[k] = v
68af0228fedc util: implement sortdict.insert() Martin von Zweigbergk <martinvonz@google.com> parents: 43883 diff changeset	1373
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1374
34357 c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1375 class cowdict(cow, dict):
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1376 """copy-on-write dict
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1377
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1378 Be sure to call d = d.preparewrite() before writing to d.
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1379
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1380 >>> a = cowdict()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1381 >>> a is a.preparewrite()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1382 True
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1383 >>> b = a.copy()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1384 >>> b is a
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1385 True
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1386 >>> c = b.copy()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1387 >>> c is a
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1388 True
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1389 >>> a = a.preparewrite()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1390 >>> b is a
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1391 False
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1392 >>> a is a.preparewrite()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1393 True
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1394 >>> c = c.preparewrite()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1395 >>> b is c
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1396 False
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1397 >>> b is b.preparewrite()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1398 True
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1399 """
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1400
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1401
34357 c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1402 class cowsortdict(cow, sortdict):
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1403 """copy-on-write sortdict
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1404
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1405 Be sure to call d = d.preparewrite() before writing to d.
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1406 """
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34295 diff changeset	1407
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1408
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48940 diff changeset	1409 class transactional: # pytype: disable=ignored-metaclass
33793 bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1410 """Base class for making a transactional type into a context manager."""
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1411
33793 bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1412 __metaclass__ = abc.ABCMeta
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1413
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1414 @abc.abstractmethod
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1415 def close(self):
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1416 """Successfully closes the transaction."""
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1417
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1418 @abc.abstractmethod
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1419 def release(self):
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1420 """Marks the end of the transaction.
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1421
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1422 If the transaction has not been closed, it will be aborted.
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1423 """
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1424
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1425 def __enter__(self):
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1426 return self
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1427
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1428 def __exit__(self, exc_type, exc_val, exc_tb):
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1429 try:
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1430 if exc_type is None:
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1431 self.close()
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1432 finally:
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1433 self.release()
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33737 diff changeset	1434
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1435
33446 fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1436 @contextlib.contextmanager
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1437 def acceptintervention(tr=None):
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1438 """A context manager that closes the transaction on InterventionRequired
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1439
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1440 If no transaction was provided, this simply runs the body and returns
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1441 """
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1442 if not tr:
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1443 yield
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1444 return
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1445 try:
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1446 yield
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1447 tr.close()
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1448 except error.InterventionRequired:
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1449 tr.close()
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1450 raise
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1451 finally:
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1452 tr.release()
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1453
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1454
33619 609606d21765 rebase: use one dirstateguard for when using rebase.singletransaction Durham Goode <durham@fb.com> parents: 33549 diff changeset	1455 @contextlib.contextmanager
45146 a0791bfd9cfa util: enhance `nullcontextmanager` to be able to return __enter__ result Manuel Jacob <me@manueljacob.de> parents: 45061 diff changeset	1456 def nullcontextmanager(enter_result=None):
a0791bfd9cfa util: enhance `nullcontextmanager` to be able to return __enter__ result Manuel Jacob <me@manueljacob.de> parents: 45061 diff changeset	1457 yield enter_result
33619 609606d21765 rebase: use one dirstateguard for when using rebase.singletransaction Durham Goode <durham@fb.com> parents: 33549 diff changeset	1458
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1459
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48940 diff changeset	1460 class _lrucachenode:
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1461 """A node in a doubly linked list.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1462
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1463 Holds a reference to nodes on either side as well as a key-value
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1464 pair for the dictionary entry.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1465 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1466
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43494 diff changeset	1467 __slots__ = ('next', 'prev', 'key', 'value', 'cost')
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1468
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1469 def __init__(self):
46901 51841b23670b typing: make minor adjustments to mercurial/util.py to pass pytype checking Matt Harbison <matt_harbison@yahoo.com> parents: 46900 diff changeset	1470 self.next = self
51841b23670b typing: make minor adjustments to mercurial/util.py to pass pytype checking Matt Harbison <matt_harbison@yahoo.com> parents: 46900 diff changeset	1471 self.prev = self
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1472
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1473 self.key = _notset
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1474 self.value = None
39567 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1475 self.cost = 0
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1476
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1477 def markempty(self):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1478 """Mark the node as emptied."""
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1479 self.key = _notset
39567 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1480 self.value = None
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1481 self.cost = 0
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1482
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1483
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48940 diff changeset	1484 class lrucachedict:
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1485 """Dict that caches most recent accesses and sets.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1486
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1487 The dict consists of an actual backing dict - indexed by original
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1488 key - and a doubly linked circular list defining the order of entries in
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1489 the cache.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1490
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1491 The head node is the newest entry in the cache. If the cache is full,
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1492 we recycle head.prev and make it the new head. Cache accesses result in
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1493 the node being moved to before the existing head and being marked as the
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1494 new head node.
39567 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1495
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1496 Items in the cache can be inserted with an optional "cost" value. This is
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1497 simply an integer that is specified by the caller. The cache can be queried
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1498 for the total cost of all items presently in the cache.
39568 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1499
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1500 The cache can also define a maximum cost. If a cache insertion would
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1501 cause the total cost of the cache to go beyond the maximum cost limit,
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1502 nodes will be evicted to make room for the new code. This can be used
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1503 to e.g. set a max memory limit and associate an estimated bytes size
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1504 cost to each item in the cache. By default, no maximum cost is enforced.
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1505 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1506
39568 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1507 def __init__(self, max, maxcost=0):
18603 2251b3184e6e util: add an LRU cache dict Siddharth Agarwal <sid0@fb.com> parents: 18537 diff changeset	1508 self._cache = {}
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1509
46901 51841b23670b typing: make minor adjustments to mercurial/util.py to pass pytype checking Matt Harbison <matt_harbison@yahoo.com> parents: 46900 diff changeset	1510 self._head = _lrucachenode()
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1511 self._size = 1
39564 5d75a3c16193 util: make capacity a public attribute on lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39563 diff changeset	1512 self.capacity = max
39567 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1513 self.totalcost = 0
39568 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1514 self.maxcost = maxcost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1515
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1516 def __len__(self):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1517 return len(self._cache)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1518
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1519 def __contains__(self, k):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1520 return k in self._cache
18603 2251b3184e6e util: add an LRU cache dict Siddharth Agarwal <sid0@fb.com> parents: 18537 diff changeset	1521
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1522 def __iter__(self):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1523 # We don't have to iterate in cache order, but why not.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1524 n = self._head
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1525 for i in range(len(self._cache)):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1526 yield n.key
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1527 n = n.next
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1528
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1529 def __getitem__(self, k):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1530 node = self._cache[k]
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1531 self._movetohead(node)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1532 return node.value
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1533
39567 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1534 def insert(self, k, v, cost=0):
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1535 """Insert a new item in the cache with optional cost value."""
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1536 node = self._cache.get(k)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1537 # Replace existing value and mark as newest.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1538 if node is not None:
39567 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1539 self.totalcost -= node.cost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1540 node.value = v
39567 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1541 node.cost = cost
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1542 self.totalcost += cost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1543 self._movetohead(node)
39568 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1544
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1545 if self.maxcost:
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1546 self._enforcecostlimit()
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1547
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1548 return
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1549
39564 5d75a3c16193 util: make capacity a public attribute on lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39563 diff changeset	1550 if self._size < self.capacity:
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1551 node = self._addcapacity()
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1552 else:
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1553 # Grab the last/oldest item.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1554 node = self._head.prev
18603 2251b3184e6e util: add an LRU cache dict Siddharth Agarwal <sid0@fb.com> parents: 18537 diff changeset	1555
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1556 # At capacity. Kill the old entry.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1557 if node.key is not _notset:
39567 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1558 self.totalcost -= node.cost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1559 del self._cache[node.key]
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1560
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1561 node.key = k
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1562 node.value = v
39567 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1563 node.cost = cost
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1564 self.totalcost += cost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1565 self._cache[k] = node
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1566 # And mark it as newest entry. No need to adjust order since it
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1567 # is already self._head.prev.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1568 self._head = node
18603 2251b3184e6e util: add an LRU cache dict Siddharth Agarwal <sid0@fb.com> parents: 18537 diff changeset	1569
39568 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1570 if self.maxcost:
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1571 self._enforcecostlimit()
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1572
39567 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1573 def __setitem__(self, k, v):
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1574 self.insert(k, v)
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1575
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1576 def __delitem__(self, k):
40880 7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40879 diff changeset	1577 self.pop(k)
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40879 diff changeset	1578
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40879 diff changeset	1579 def pop(self, k, default=_notset):
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40879 diff changeset	1580 try:
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40879 diff changeset	1581 node = self._cache.pop(k)
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40879 diff changeset	1582 except KeyError:
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40879 diff changeset	1583 if default is _notset:
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40879 diff changeset	1584 raise
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40879 diff changeset	1585 return default
43881 b5655f337bd7 typing: add a couple of assertions to lrucachedict to help pytype Matt Harbison <matt_harbison@yahoo.com> parents: 43880 diff changeset	1586
40880 7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40879 diff changeset	1587 value = node.value
39567 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1588 self.totalcost -= node.cost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1589 node.markempty()
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1590
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1591 # Temporarily mark as newest item before re-adjusting head to make
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1592 # this node the oldest item.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1593 self._movetohead(node)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1594 self._head = node.next
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1595
40880 7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40879 diff changeset	1596 return value
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40879 diff changeset	1597
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1598 # Additional dict methods.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1599
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1600 def get(self, k, default=None):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1601 try:
39571 8f2c0d1b454c util: update lrucachedict order during get() Gregory Szorc <gregory.szorc@gmail.com> parents: 39570 diff changeset	1602 return self.__getitem__(k)
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1603 except KeyError:
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1604 return default
18603 2251b3184e6e util: add an LRU cache dict Siddharth Agarwal <sid0@fb.com> parents: 18537 diff changeset	1605
40879 0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40689 diff changeset	1606 def peek(self, k, default=_notset):
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40689 diff changeset	1607 """Get the specified item without moving it to the head
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40689 diff changeset	1608
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40689 diff changeset	1609 Unlike get(), this doesn't mutate the internal state. But be aware
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40689 diff changeset	1610 that it doesn't mean peek() is thread safe.
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40689 diff changeset	1611 """
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40689 diff changeset	1612 try:
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40689 diff changeset	1613 node = self._cache[k]
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40689 diff changeset	1614 return node.value
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40689 diff changeset	1615 except KeyError:
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40689 diff changeset	1616 if default is _notset:
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40689 diff changeset	1617 raise
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40689 diff changeset	1618 return default
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40689 diff changeset	1619
19710 887ffa22fd0d lrucachedict: implement clear() Siddharth Agarwal <sid0@fb.com> parents: 19461 diff changeset	1620 def clear(self):
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1621 n = self._head
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1622 while n.key is not _notset:
39567 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1623 self.totalcost -= n.cost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1624 n.markempty()
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1625 n = n.next
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1626
19710 887ffa22fd0d lrucachedict: implement clear() Siddharth Agarwal <sid0@fb.com> parents: 19461 diff changeset	1627 self._cache.clear()
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1628
39568 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1629 def copy(self, capacity=None, maxcost=0):
39565 2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39564 diff changeset	1630 """Create a new cache as a copy of the current one.
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39564 diff changeset	1631
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39564 diff changeset	1632 By default, the new cache has the same capacity as the existing one.
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39564 diff changeset	1633 But, the cache capacity can be changed as part of performing the
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39564 diff changeset	1634 copy.
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39564 diff changeset	1635
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39564 diff changeset	1636 Items in the copy have an insertion/access order matching this
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39564 diff changeset	1637 instance.
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39564 diff changeset	1638 """
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39564 diff changeset	1639
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39564 diff changeset	1640 capacity = capacity or self.capacity
39568 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1641 maxcost = maxcost or self.maxcost
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1642 result = lrucachedict(capacity, maxcost=maxcost)
39563 b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	1643
b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	1644 # We copy entries by iterating in oldest-to-newest order so the copy
b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	1645 # has the correct ordering.
b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	1646
b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	1647 # Find the first non-empty entry.
27576 6cd3044985c2 lrucachedict: add copy method Eric Sumner <ericsumner@fb.com> parents: 27391 diff changeset	1648 n = self._head.prev
39563 b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	1649 while n.key is _notset and n is not self._head:
b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	1650 n = n.prev
b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	1651
39565 2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39564 diff changeset	1652 # We could potentially skip the first N items when decreasing capacity.
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39564 diff changeset	1653 # But let's keep it simple unless it is a performance problem.
27576 6cd3044985c2 lrucachedict: add copy method Eric Sumner <ericsumner@fb.com> parents: 27391 diff changeset	1654 for i in range(len(self._cache)):
39567 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1655 result.insert(n.key, n.value, cost=n.cost)
27576 6cd3044985c2 lrucachedict: add copy method Eric Sumner <ericsumner@fb.com> parents: 27391 diff changeset	1656 n = n.prev
39563 b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39259 diff changeset	1657
27576 6cd3044985c2 lrucachedict: add copy method Eric Sumner <ericsumner@fb.com> parents: 27391 diff changeset	1658 return result
6cd3044985c2 lrucachedict: add copy method Eric Sumner <ericsumner@fb.com> parents: 27391 diff changeset	1659
39566 bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1660 def popoldest(self):
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1661 """Remove the oldest item from the cache.
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1662
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1663 Returns the (key, value) describing the removed cache entry.
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1664 """
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1665 if not self._cache:
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1666 return
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1667
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1668 # Walk the linked list backwards starting at tail node until we hit
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1669 # a non-empty node.
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1670 n = self._head.prev
46901 51841b23670b typing: make minor adjustments to mercurial/util.py to pass pytype checking Matt Harbison <matt_harbison@yahoo.com> parents: 46900 diff changeset	1671
39566 bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1672 while n.key is _notset:
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1673 n = n.prev
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1674
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1675 key, value = n.key, n.value
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1676
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1677 # And remove it from the cache and mark it as empty.
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1678 del self._cache[n.key]
39567 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39566 diff changeset	1679 self.totalcost -= n.cost
39566 bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1680 n.markempty()
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1681
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1682 return key, value
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39565 diff changeset	1683
51295 c8a2fdf5ca37 pytype: add the couple annotations for pytype to understands the lrunode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51291 diff changeset	1684 def _movetohead(self, node: _lrucachenode):
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1685 """Mark a node as the newest, making it the new head.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1686
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1687 When a node is accessed, it becomes the freshest entry in the LRU
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1688 list, which is denoted by self._head.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1689
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1690 Visually, let's make ``N`` the new head node (* denotes head):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1691
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1692 previous/oldest <-> head <-> next/next newest
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1693
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1694 ----<->--- A* ---<->-----
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1695 \| \|
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1696 E <-> D <-> N <-> C <-> B
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1697
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1698 To:
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1699
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1700 ----<->--- N* ---<->-----
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1701 \| \|
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1702 E <-> D <-> C <-> B <-> A
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1703
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1704 This requires the following moves:
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1705
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1706 C.next = D (node.prev.next = node.next)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1707 D.prev = C (node.next.prev = node.prev)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1708 E.next = N (head.prev.next = node)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1709 N.prev = E (node.prev = head.prev)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1710 N.next = A (node.next = head)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1711 A.prev = N (head.prev = node)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1712 """
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1713 head = self._head
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1714 # C.next = D
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1715 node.prev.next = node.next
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1716 # D.prev = C
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1717 node.next.prev = node.prev
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1718 # N.prev = E
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1719 node.prev = head.prev
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1720 # N.next = A
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1721 # It is tempting to do just "head" here, however if node is
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1722 # adjacent to head, this will do bad things.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1723 node.next = head.prev.next
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1724 # E.next = N
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1725 node.next.prev = node
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1726 # A.prev = N
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1727 node.prev.next = node
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1728
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1729 self._head = node
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1730
51295 c8a2fdf5ca37 pytype: add the couple annotations for pytype to understands the lrunode Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51291 diff changeset	1731 def _addcapacity(self) -> _lrucachenode:
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1732 """Add a node to the circular linked list.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1733
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1734 The new node is inserted before the head node.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1735 """
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1736 head = self._head
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1737 node = _lrucachenode()
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1738 head.prev.next = node
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1739 node.prev = head.prev
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1740 node.next = head
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1741 head.prev = node
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1742 self._size += 1
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1743 return node
19710 887ffa22fd0d lrucachedict: implement clear() Siddharth Agarwal <sid0@fb.com> parents: 19461 diff changeset	1744
39568 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1745 def _enforcecostlimit(self):
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1746 # This should run after an insertion. It should only be called if total
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1747 # cost limits are being enforced.
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1748 # The most recently inserted node is never evicted.
39569 cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39568 diff changeset	1749 if len(self) <= 1 or self.totalcost <= self.maxcost:
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39568 diff changeset	1750 return
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39568 diff changeset	1751
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39568 diff changeset	1752 # This is logically equivalent to calling popoldest() until we
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39568 diff changeset	1753 # free up enough cost. We don't do that since popoldest() needs
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39568 diff changeset	1754 # to walk the linked list and doing this in a loop would be
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39568 diff changeset	1755 # quadratic. So we find the first non-empty node and then
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39568 diff changeset	1756 # walk nodes until we free up enough capacity.
39570 f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39569 diff changeset	1757 #
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39569 diff changeset	1758 # If we only removed the minimum number of nodes to free enough
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39569 diff changeset	1759 # cost at insert time, chances are high that the next insert would
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39569 diff changeset	1760 # also require pruning. This would effectively constitute quadratic
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39569 diff changeset	1761 # behavior for insert-heavy workloads. To mitigate this, we set a
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39569 diff changeset	1762 # target cost that is a percentage of the max cost. This will tend
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39569 diff changeset	1763 # to free more nodes when the high water mark is reached, which
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39569 diff changeset	1764 # lowers the chances of needing to prune on the subsequent insert.
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39569 diff changeset	1765 targetcost = int(self.maxcost * 0.75)
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39569 diff changeset	1766
39569 cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39568 diff changeset	1767 n = self._head.prev
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39568 diff changeset	1768 while n.key is _notset:
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39568 diff changeset	1769 n = n.prev
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39568 diff changeset	1770
39570 f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39569 diff changeset	1771 while len(self) > 1 and self.totalcost > targetcost:
39569 cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39568 diff changeset	1772 del self._cache[n.key]
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39568 diff changeset	1773 self.totalcost -= n.cost
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39568 diff changeset	1774 n.markempty()
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39568 diff changeset	1775 n = n.prev
39568 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39567 diff changeset	1776
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1777
9097 431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1778 def lrucachefunc(func):
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1779 '''cache most recent results of function calls'''
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1780 cache = {}
25113 0ca8410ea345 util: drop alias for collections.deque Martin von Zweigbergk <martinvonz@google.com> parents: 25112 diff changeset	1781 order = collections.deque()
28832 f5ff10f6fa6b util: use __code__ (available since py2.6) timeless <timeless@mozdev.org> parents: 28826 diff changeset	1782 if func.__code__.co_argcount == 1:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1783
9097 431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1784 def f(arg):
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1785 if arg not in cache:
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1786 if len(cache) > 20:
16803 107a3270a24a cleanup: use the deque type where appropriate Bryan O'Sullivan <bryano@fb.com> parents: 16769 diff changeset	1787 del cache[order.popleft()]
9097 431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1788 cache[arg] = func(arg)
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1789 else:
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1790 order.remove(arg)
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1791 order.append(arg)
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1792 return cache[arg]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1793
9097 431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1794 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1795
9097 431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1796 def f(*args):
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1797 if args not in cache:
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1798 if len(cache) > 20:
16803 107a3270a24a cleanup: use the deque type where appropriate Bryan O'Sullivan <bryano@fb.com> parents: 16769 diff changeset	1799 del cache[order.popleft()]
9097 431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1800 cache[args] = func(*args)
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1801 else:
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1802 order.remove(args)
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1803 order.append(args)
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1804 return cache[args]
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1805
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1806 return f
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1807
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1808
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48940 diff changeset	1809 class propertycache:
8207 dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1810 def __init__(self, func):
dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1811 self.func = func
dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1812 self.name = func.__name__
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1813
8207 dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1814 def __get__(self, obj, type=None):
dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1815 result = self.func(obj)
18013 98c867ac1330 clfilter: add a propertycache that must be unfiltered Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 17962 diff changeset	1816 self.cachevalue(obj, result)
8207 dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1817 return result
dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1818
18013 98c867ac1330 clfilter: add a propertycache that must be unfiltered Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 17962 diff changeset	1819 def cachevalue(self, obj, value):
19951 d51c4d85ec23 spelling: random spell checker fixes Mads Kiilerich <madski@unity3d.com> parents: 19852 diff changeset	1820 # __dict__ assignment required to bypass __setattr__ (eg: repoview)
19845 a1237a4b437d repoview: make propertycache.setcache compatible with repoview Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 19461 diff changeset	1821 obj.__dict__[self.name] = value
18013 98c867ac1330 clfilter: add a propertycache that must be unfiltered Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 17962 diff changeset	1822
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1823
35014 be6aa0cff8ea util: add util.clearcachedproperty Mark Thomas <mbthomas@fb.com> parents: 34647 diff changeset	1824 def clearcachedproperty(obj, prop):
be6aa0cff8ea util: add util.clearcachedproperty Mark Thomas <mbthomas@fb.com> parents: 34647 diff changeset	1825 '''clear a cached property value, if one has been set'''
40689 475921a3028c py3: cast attribute name to sysstr in clearcachedproperty() Yuya Nishihara <yuya@tcha.org> parents: 40251 diff changeset	1826 prop = pycompat.sysstr(prop)
35014 be6aa0cff8ea util: add util.clearcachedproperty Mark Thomas <mbthomas@fb.com> parents: 34647 diff changeset	1827 if prop in obj.__dict__:
be6aa0cff8ea util: add util.clearcachedproperty Mark Thomas <mbthomas@fb.com> parents: 34647 diff changeset	1828 del obj.__dict__[prop]
be6aa0cff8ea util: add util.clearcachedproperty Mark Thomas <mbthomas@fb.com> parents: 34647 diff changeset	1829
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1830
7396 526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1831 def increasingchunks(source, min=1024, max=65536):
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	1832 """return no less than min bytes per chunk while data remains,
89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	1833 doubling min after each chunk until it reaches max"""
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1834
7396 526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1835 def log2(x):
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1836 if not x:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1837 return 0
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1838 i = 0
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1839 while x:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1840 x >>= 1
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1841 i += 1
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1842 return i - 1
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1843
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1844 buf = []
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1845 blen = 0
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1846 for chunk in source:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1847 buf.append(chunk)
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1848 blen += len(chunk)
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1849 if blen >= min:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1850 if min < max:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1851 min = min << 1
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1852 nmin = 1 << log2(blen)
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1853 if nmin > min:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1854 min = nmin
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1855 if min > max:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1856 min = max
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1857 yield b''.join(buf)
7396 526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1858 blen = 0
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1859 buf = []
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1860 if buf:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1861 yield b''.join(buf)
7396 526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1862
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1863
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1864 def always(fn):
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1865 return True
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1866
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1867
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1868 def never(fn):
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1869 return False
724 1c0c413cccdd Get add and locate to use new repo and dirstate walk code. Bryan O'Sullivan <bos@serpentine.com> parents: 705 diff changeset	1870
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1871
51546 a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	1872 def nogc(func=None) -> Any:
23495 b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1873 """disable garbage collector
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1874
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1875 Python's garbage collector triggers a GC each time a certain number of
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1876 container objects (the number being defined by gc.get_threshold()) are
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1877 allocated even when marked not to be tracked by the collector. Tracking has
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1878 no effect on when GCs are triggered, only on what objects the GC looks
23543 4dd8a6a1240d spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23495 diff changeset	1879 into. As a workaround, disable GC while building complex (huge)
23495 b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1880 containers.
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1881
33799 05264fc9d8d6 util: make nogc effective for CPython Jun Wu <quark@fb.com> parents: 33793 diff changeset	1882 This garbage collector issue have been fixed in 2.7. But it still affect
05264fc9d8d6 util: make nogc effective for CPython Jun Wu <quark@fb.com> parents: 33793 diff changeset	1883 CPython's performance.
23495 b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1884 """
51546 a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	1885 if func is None:
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	1886 return _nogc_context()
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	1887 else:
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	1888 return _nogc_decorator(func)
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	1889
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	1890
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	1891 @contextlib.contextmanager
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	1892 def _nogc_context():
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	1893 gcenabled = gc.isenabled()
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	1894 gc.disable()
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	1895 try:
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	1896 yield
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	1897 finally:
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	1898 if gcenabled:
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	1899 gc.enable()
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	1900
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	1901
a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	1902 def _nogc_decorator(func):
23495 b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1903 def wrapper(args, *kwargs):
51546 a452807df09b nocg: make the utility work are both a decorator and context manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	1904 with _nogc_context():
23495 b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1905 return func(args, *kwargs)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1906
23495 b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1907 return wrapper
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1908
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1909
33799 05264fc9d8d6 util: make nogc effective for CPython Jun Wu <quark@fb.com> parents: 33793 diff changeset	1910 if pycompat.ispypy:
05264fc9d8d6 util: make nogc effective for CPython Jun Wu <quark@fb.com> parents: 33793 diff changeset	1911 # PyPy runs slower with gc disabled
05264fc9d8d6 util: make nogc effective for CPython Jun Wu <quark@fb.com> parents: 33793 diff changeset	1912 nogc = lambda x: x
05264fc9d8d6 util: make nogc effective for CPython Jun Wu <quark@fb.com> parents: 33793 diff changeset	1913
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1914
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	1915 def pathto(root: bytes, n1: bytes, n2: bytes) -> bytes:
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	1916 """return the relative path from one place to another.
4229 24c22a3f2ef8 pass repo.root to util.pathto() in preparation for the next patch Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4129 diff changeset	1917 root should use os.sep to separate directories
3669 48768b1ab23c fix util.pathto Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 3629 diff changeset	1918 n1 should use os.sep to separate directories
48768b1ab23c fix util.pathto Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 3629 diff changeset	1919 n2 should use "/" to separate directories
48768b1ab23c fix util.pathto Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 3629 diff changeset	1920 returns an os.sep-separated path.
4229 24c22a3f2ef8 pass repo.root to util.pathto() in preparation for the next patch Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4129 diff changeset	1921
24c22a3f2ef8 pass repo.root to util.pathto() in preparation for the next patch Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4129 diff changeset	1922 If n1 is a relative path, it's assumed it's
24c22a3f2ef8 pass repo.root to util.pathto() in preparation for the next patch Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4129 diff changeset	1923 relative to root.
24c22a3f2ef8 pass repo.root to util.pathto() in preparation for the next patch Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4129 diff changeset	1924 n2 should always be relative to root.
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	1925 """
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1926 if not n1:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1927 return localpath(n2)
4230 c93562fb12cc Fix handling of paths when run outside the repo. Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4229 diff changeset	1928 if os.path.isabs(n1):
c93562fb12cc Fix handling of paths when run outside the repo. Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4229 diff changeset	1929 if os.path.splitdrive(root)[0] != os.path.splitdrive(n1)[0]:
c93562fb12cc Fix handling of paths when run outside the repo. Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4229 diff changeset	1930 return os.path.join(root, localpath(n2))
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1931 n2 = b'/'.join((pconvert(root), n2))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1932 a, b = splitpath(n1), n2.split(b'/')
1541 bf4e7ef08741 fixed some stuff pychecker shows, marked unclear/wrong stuff with XXX twaldmann@thinkmo.de parents: 1528 diff changeset	1933 a.reverse()
bf4e7ef08741 fixed some stuff pychecker shows, marked unclear/wrong stuff with XXX twaldmann@thinkmo.de parents: 1528 diff changeset	1934 b.reverse()
884 087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types. Bryan O'Sullivan <bos@serpentine.com> parents: 878 diff changeset	1935 while a and b and a[-1] == b[-1]:
1541 bf4e7ef08741 fixed some stuff pychecker shows, marked unclear/wrong stuff with XXX twaldmann@thinkmo.de parents: 1528 diff changeset	1936 a.pop()
bf4e7ef08741 fixed some stuff pychecker shows, marked unclear/wrong stuff with XXX twaldmann@thinkmo.de parents: 1528 diff changeset	1937 b.pop()
884 087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types. Bryan O'Sullivan <bos@serpentine.com> parents: 878 diff changeset	1938 b.reverse()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1939 return pycompat.ossep.join(([b'..'] * len(a)) + b) or b'.'
884 087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types. Bryan O'Sullivan <bos@serpentine.com> parents: 878 diff changeset	1940
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1941
44436 09f3e003fc2a phabricator: avoid a stacktrace when command arguments are missing Matt Harbison <matt_harbison@yahoo.com> parents: 44060 diff changeset	1942 def checksignature(func, depth=1):
7388 5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1943 '''wrap a function with code to check for calling errors'''
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1944
7388 5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1945 def check(args, *kwargs):
5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1946 try:
5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1947 return func(args, *kwargs)
5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1948 except TypeError:
44436 09f3e003fc2a phabricator: avoid a stacktrace when command arguments are missing Matt Harbison <matt_harbison@yahoo.com> parents: 44060 diff changeset	1949 if len(traceback.extract_tb(sys.exc_info()[2])) == depth:
7646 e62a456b8dc5 error: move SignatureError Matt Mackall <mpm@selenic.com> parents: 7644 diff changeset	1950 raise error.SignatureError
7388 5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1951 raise
5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1952
5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1953 return check
5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1954
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1955
31575 e506e461c7a9 util: disable hardlink for copyfile if fstype is outside a whitelist Jun Wu <quark@fb.com> parents: 31573 diff changeset	1956 # a whilelist of known filesystems where hardlink works reliably
32291 bd872f64a8ba cleanup: use set literals Martin von Zweigbergk <martinvonz@google.com> parents: 32290 diff changeset	1957 _hardlinkfswhitelist = {
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1958 b'apfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1959 b'btrfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1960 b'ext2',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1961 b'ext3',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1962 b'ext4',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1963 b'hfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1964 b'jfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1965 b'NTFS',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1966 b'reiserfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1967 b'tmpfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1968 b'ufs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1969 b'xfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1970 b'zfs',
32291 bd872f64a8ba cleanup: use set literals Martin von Zweigbergk <martinvonz@google.com> parents: 32290 diff changeset	1971 }
31575 e506e461c7a9 util: disable hardlink for copyfile if fstype is outside a whitelist Jun Wu <quark@fb.com> parents: 31573 diff changeset	1972
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	1973
47400 9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47233 diff changeset	1974 def copyfile(
47445 d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47417 diff changeset	1975 src,
d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47417 diff changeset	1976 dest,
d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47417 diff changeset	1977 hardlink=False,
d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47417 diff changeset	1978 copystat=False,
d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47417 diff changeset	1979 checkambig=False,
d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47417 diff changeset	1980 nb_bytes=None,
d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47417 diff changeset	1981 no_hardlink_cb=None,
47446 09ff5d532a25 copyfiles: add a way to relax the file system checking for hardlink Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47445 diff changeset	1982 check_fs_hardlink=True,
47400 9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47233 diff changeset	1983 ):
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	1984 """copy a file, preserving mode and optionally other stat info like
29367 4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1985 atime/mtime
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1986
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1987 checkambig argument is used with filestat, and is useful only if
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1988 destination file is guarded by any lock (e.g. repo.lock or
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1989 repo.wlock).
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1990
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1991 copystat and checkambig should be exclusive.
47400 9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47233 diff changeset	1992
9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47233 diff changeset	1993 nb_bytes: if set only copy the first `nb_bytes` of the source file.
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	1994 """
29204 ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	1995 assert not (copystat and checkambig)
ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	1996 oldstat = None
18326 614f769e6aa7 util: copyfile: remove dest before copying Mads Kiilerich <mads@kiilerich.com> parents: 18026 diff changeset	1997 if os.path.lexists(dest):
29204 ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	1998 if checkambig:
32772 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32749 diff changeset	1999 oldstat = checkambig and filestat.frompath(dest)
18326 614f769e6aa7 util: copyfile: remove dest before copying Mads Kiilerich <mads@kiilerich.com> parents: 18026 diff changeset	2000 unlink(dest)
47446 09ff5d532a25 copyfiles: add a way to relax the file system checking for hardlink Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47445 diff changeset	2001 if hardlink and check_fs_hardlink:
31575 e506e461c7a9 util: disable hardlink for copyfile if fstype is outside a whitelist Jun Wu <quark@fb.com> parents: 31573 diff changeset	2002 # Hardlinks are problematic on CIFS (issue4546), do not allow hardlinks
e506e461c7a9 util: disable hardlink for copyfile if fstype is outside a whitelist Jun Wu <quark@fb.com> parents: 31573 diff changeset	2003 # unless we are confident that dest is on a whitelisted filesystem.
31678 1ed57a7dd904 statfs: make getfstype() raise OSError Yuya Nishihara <yuya@tcha.org> parents: 31662 diff changeset	2004 try:
1ed57a7dd904 statfs: make getfstype() raise OSError Yuya Nishihara <yuya@tcha.org> parents: 31662 diff changeset	2005 fstype = getfstype(os.path.dirname(dest))
1ed57a7dd904 statfs: make getfstype() raise OSError Yuya Nishihara <yuya@tcha.org> parents: 31662 diff changeset	2006 except OSError:
1ed57a7dd904 statfs: make getfstype() raise OSError Yuya Nishihara <yuya@tcha.org> parents: 31662 diff changeset	2007 fstype = None
31575 e506e461c7a9 util: disable hardlink for copyfile if fstype is outside a whitelist Jun Wu <quark@fb.com> parents: 31573 diff changeset	2008 if fstype not in _hardlinkfswhitelist:
47445 d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47417 diff changeset	2009 if no_hardlink_cb is not None:
d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47417 diff changeset	2010 no_hardlink_cb()
31575 e506e461c7a9 util: disable hardlink for copyfile if fstype is outside a whitelist Jun Wu <quark@fb.com> parents: 31573 diff changeset	2011 hardlink = False
31577 e7a02e9ad162 util: enable hardlink for copyfile Jun Wu <quark@fb.com> parents: 31575 diff changeset	2012 if hardlink:
23899 4e451d1359de copyfile: allow optional hardlinking Pierre-Yves David <pierre-yves.david@fb.com> parents: 23864 diff changeset	2013 try:
4e451d1359de copyfile: allow optional hardlinking Pierre-Yves David <pierre-yves.david@fb.com> parents: 23864 diff changeset	2014 oslink(src, dest)
47400 9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47233 diff changeset	2015 if nb_bytes is not None:
9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47233 diff changeset	2016 m = "the `nb_bytes` argument is incompatible with `hardlink`"
9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47233 diff changeset	2017 raise error.ProgrammingError(m)
23899 4e451d1359de copyfile: allow optional hardlinking Pierre-Yves David <pierre-yves.david@fb.com> parents: 23864 diff changeset	2018 return
47445 d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47417 diff changeset	2019 except (IOError, OSError) as exc:
d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47417 diff changeset	2020 if exc.errno != errno.EEXIST and no_hardlink_cb is not None:
d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47417 diff changeset	2021 no_hardlink_cb()
d756fc11cfb9 copyfile: add a option callback for failed hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47417 diff changeset	2022 # fall back to normal copy
4271 1eaa8d90c689 fix util.copyfile to deal with symlinks Eric St-Jean <esj@wwd.ca> parents: 4256 diff changeset	2023 if os.path.islink(src):
1eaa8d90c689 fix util.copyfile to deal with symlinks Eric St-Jean <esj@wwd.ca> parents: 4256 diff changeset	2024 os.symlink(os.readlink(src), dest)
27369 c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	2025 # copytime is ignored for symlinks, but in general copytime isn't needed
c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	2026 # for them anyway
47400 9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47233 diff changeset	2027 if nb_bytes is not None:
9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47233 diff changeset	2028 m = "cannot use `nb_bytes` on a symlink"
9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47233 diff changeset	2029 raise error.ProgrammingError(m)
4271 1eaa8d90c689 fix util.copyfile to deal with symlinks Eric St-Jean <esj@wwd.ca> parents: 4256 diff changeset	2030 else:
1eaa8d90c689 fix util.copyfile to deal with symlinks Eric St-Jean <esj@wwd.ca> parents: 4256 diff changeset	2031 try:
1eaa8d90c689 fix util.copyfile to deal with symlinks Eric St-Jean <esj@wwd.ca> parents: 4256 diff changeset	2032 shutil.copyfile(src, dest)
27369 c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	2033 if copystat:
c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	2034 # copystat also copies mode
c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	2035 shutil.copystat(src, dest)
c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	2036 else:
c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	2037 shutil.copymode(src, dest)
29204 ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	2038 if oldstat and oldstat.stat:
32772 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32749 diff changeset	2039 newstat = filestat.frompath(dest)
29204 ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	2040 if newstat.isambig(oldstat):
ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	2041 # stat of copied file is ambiguous to original one
36781 ffa3026d4196 cleanup: use stat_result[stat.ST_MTIME] instead of stat_result.st_mtime Augie Fackler <augie@google.com> parents: 36724 diff changeset	2042 advanced = (
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2043 oldstat.stat[stat.ST_MTIME] + 1
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2044 ) & 0x7FFFFFFF
29204 ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	2045 os.utime(dest, (advanced, advanced))
47400 9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47233 diff changeset	2046 # We could do something smarter using `copy_file_range` call or similar
9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47233 diff changeset	2047 if nb_bytes is not None:
9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47233 diff changeset	2048 with open(dest, mode='r+') as f:
9b841267253c util: add `nb_bytes` argument to `copyfile` to partially copy a file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47233 diff changeset	2049 f.truncate(nb_bytes)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25420 diff changeset	2050 except shutil.Error as inst:
43724 5be128f669d4 util: convert an exception to bytes when passing to Abort() Matt Harbison <matt_harbison@yahoo.com> parents: 43117 diff changeset	2051 raise error.Abort(stringutil.forcebytestr(inst))
3629 4cfb72bcb978 util: add copyfile function Matt Mackall <mpm@selenic.com> parents: 3568 diff changeset	2052
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2053
38380 63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38314 diff changeset	2054 def copyfiles(src, dst, hardlink=None, progress=None):
24439 2ddfac2f163e util: add progress callback support to copyfiles Augie Fackler <augie@google.com> parents: 24236 diff changeset	2055 """Copy a directory tree using hardlinks if possible."""
2ddfac2f163e util: add progress callback support to copyfiles Augie Fackler <augie@google.com> parents: 24236 diff changeset	2056 num = 0
1241 3b4f05ff3130 Add support for cloning with hardlinks on windows. Stephen Darnell parents: 1207 diff changeset	2057
38380 63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38314 diff changeset	2058 def settopic():
63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38314 diff changeset	2059 if progress:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2060 progress.topic = _(b'linking') if hardlink else _(b'copying')
698 df78d8ccac4c Use python function instead of external 'cp' command when cloning repos. Thomas Arendsen Hein <thomas@intevation.de> parents: 667 diff changeset	2061
1207 a7b8812973d9 Rewrite copytree as copyfiles mpm@selenic.com parents: 1200 diff changeset	2062 if os.path.isdir(src):
31719 456efd1b51fd hardlink: duplicate hardlink detection for copying files and directories Jun Wu <quark@fb.com> parents: 31718 diff changeset	2063 if hardlink is None:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2064 hardlink = (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2065 os.stat(src).st_dev == os.stat(os.path.dirname(dst)).st_dev
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2066 )
38380 63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38314 diff changeset	2067 settopic()
1207 a7b8812973d9 Rewrite copytree as copyfiles mpm@selenic.com parents: 1200 diff changeset	2068 os.mkdir(dst)
32208 d74b0cff94a9 osutil: proxy through util (and platform) modules (API) Yuya Nishihara <yuya@tcha.org> parents: 32201 diff changeset	2069 for name, kind in listdir(src):
1207 a7b8812973d9 Rewrite copytree as copyfiles mpm@selenic.com parents: 1200 diff changeset	2070 srcname = os.path.join(src, name)
a7b8812973d9 Rewrite copytree as copyfiles mpm@selenic.com parents: 1200 diff changeset	2071 dstname = os.path.join(dst, name)
38380 63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38314 diff changeset	2072 hardlink, n = copyfiles(srcname, dstname, hardlink, progress)
11251 c61442f6d106 clone: print number of linked/copied files on --debug Adrian Buehlmann <adrian@cadifra.com> parents: 11232 diff changeset	2073 num += n
1207 a7b8812973d9 Rewrite copytree as copyfiles mpm@selenic.com parents: 1200 diff changeset	2074 else:
31719 456efd1b51fd hardlink: duplicate hardlink detection for copying files and directories Jun Wu <quark@fb.com> parents: 31718 diff changeset	2075 if hardlink is None:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2076 hardlink = (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2077 os.stat(os.path.dirname(src)).st_dev
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2078 == os.stat(os.path.dirname(dst)).st_dev
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2079 )
38380 63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38314 diff changeset	2080 settopic()
31719 456efd1b51fd hardlink: duplicate hardlink detection for copying files and directories Jun Wu <quark@fb.com> parents: 31718 diff changeset	2081
1241 3b4f05ff3130 Add support for cloning with hardlinks on windows. Stephen Darnell parents: 1207 diff changeset	2082 if hardlink:
3b4f05ff3130 Add support for cloning with hardlinks on windows. Stephen Darnell parents: 1207 diff changeset	2083 try:
14235 b9e1b041744f rename util.os_link to oslink Adrian Buehlmann <adrian@cadifra.com> parents: 14234 diff changeset	2084 oslink(src, dst)
47417 9ea525216edb copyfiles: deal with existing file when hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47400 diff changeset	2085 except (IOError, OSError) as exc:
9ea525216edb copyfiles: deal with existing file when hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47400 diff changeset	2086 if exc.errno != errno.EEXIST:
9ea525216edb copyfiles: deal with existing file when hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47400 diff changeset	2087 hardlink = False
9ea525216edb copyfiles: deal with existing file when hardlinking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47400 diff changeset	2088 # XXX maybe try to relink if the file exist ?
1591 5a3229cf1492 do not copy atime and mtime in util.copyfiles Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 1585 diff changeset	2089 shutil.copy(src, dst)
1241 3b4f05ff3130 Add support for cloning with hardlinks on windows. Stephen Darnell parents: 1207 diff changeset	2090 else:
1591 5a3229cf1492 do not copy atime and mtime in util.copyfiles Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 1585 diff changeset	2091 shutil.copy(src, dst)
11251 c61442f6d106 clone: print number of linked/copied files on --debug Adrian Buehlmann <adrian@cadifra.com> parents: 11232 diff changeset	2092 num += 1
38380 63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38314 diff changeset	2093 if progress:
63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38314 diff changeset	2094 progress.increment()
698 df78d8ccac4c Use python function instead of external 'cp' command when cloning repos. Thomas Arendsen Hein <thomas@intevation.de> parents: 667 diff changeset	2095
11251 c61442f6d106 clone: print number of linked/copied files on --debug Adrian Buehlmann <adrian@cadifra.com> parents: 11232 diff changeset	2096 return hardlink, num
11254 640d419725d0 util.copyfiles: don't try os_link() again if it failed before Adrian Buehlmann <adrian@cadifra.com> parents: 11010 diff changeset	2097
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2098
34052 ca6a3852daf0 util: use set for reserved Windows filenames Gregory Szorc <gregory.szorc@gmail.com> parents: 34002 diff changeset	2099 _winreservednames = {
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2100 b'con',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2101 b'prn',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2102 b'aux',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2103 b'nul',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2104 b'com1',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2105 b'com2',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2106 b'com3',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2107 b'com4',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2108 b'com5',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2109 b'com6',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2110 b'com7',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2111 b'com8',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2112 b'com9',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2113 b'lpt1',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2114 b'lpt2',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2115 b'lpt3',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2116 b'lpt4',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2117 b'lpt5',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2118 b'lpt6',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2119 b'lpt7',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2120 b'lpt8',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2121 b'lpt9',
34052 ca6a3852daf0 util: use set for reserved Windows filenames Gregory Szorc <gregory.szorc@gmail.com> parents: 34002 diff changeset	2122 }
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2123 _winreservedchars = b':*?"<>\|'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2124
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2125
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	2126 def checkwinfilename(path: bytes) -> Optional[bytes]:
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2127 r"""Check that the base-relative path is a valid filename on Windows.
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2128 Returns None if the path is ok, or a UI string describing the problem.
98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2129
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	2130 >>> checkwinfilename(b"just/a/normal/path")
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	2131 >>> checkwinfilename(b"foo/bar/con.xml")
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2132 "filename contains 'con', which is reserved on Windows"
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	2133 >>> checkwinfilename(b"foo/con.xml/bar")
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2134 "filename contains 'con', which is reserved on Windows"
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	2135 >>> checkwinfilename(b"foo/bar/xml.con")
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	2136 >>> checkwinfilename(b"foo/bar/AUX/bla.txt")
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2137 "filename contains 'AUX', which is reserved on Windows"
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	2138 >>> checkwinfilename(b"foo/bar/bla:.txt")
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2139 "filename contains ':', which is reserved on Windows"
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	2140 >>> checkwinfilename(b"foo/bar/b\07la.txt")
20000 0849d280663e util: warn when adding paths ending with \ Mads Kiilerich <madski@unity3d.com> parents: 19951 diff changeset	2141 "filename contains '\\x07', which is invalid on Windows"
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	2142 >>> checkwinfilename(b"foo/bar/bla ")
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2143 "filename ends with ' ', which is not allowed on Windows"
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	2144 >>> checkwinfilename(b"../bar")
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	2145 >>> checkwinfilename(b"foo\\")
20000 0849d280663e util: warn when adding paths ending with \ Mads Kiilerich <madski@unity3d.com> parents: 19951 diff changeset	2146 "filename ends with '\\', which is invalid on Windows"
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	2147 >>> checkwinfilename(b"foo\\/bar")
20000 0849d280663e util: warn when adding paths ending with \ Mads Kiilerich <madski@unity3d.com> parents: 19951 diff changeset	2148 "directory name ends with '\\', which is invalid on Windows"
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2149 """
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2150 if path.endswith(b'\\'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2151 return _(b"filename ends with '\\', which is invalid on Windows")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2152 if b'\\/' in path:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2153 return _(b"directory name ends with '\\', which is invalid on Windows")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2154 for n in path.replace(b'\\', b'/').split(b'/'):
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2155 if not n:
98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2156 continue
32572 377c74ef008d win32mbcs: avoid unintentional failure at colorization FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 31952 diff changeset	2157 for c in _filenamebytestr(n):
14262 23cd7eeff678 util: rename _windows_reserved_filenames and _windows_reserved_chars Adrian Buehlmann <adrian@cadifra.com> parents: 14250 diff changeset	2158 if c in _winreservedchars:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2159 return (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2160 _(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2161 b"filename contains '%s', which is reserved "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2162 b"on Windows"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2163 )
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2164 % c
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2165 )
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2166 if ord(c) <= 31:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2167 return _(
43117 8ff1ecfadcd1 cleanup: join string literals that are already on one line Martin von Zweigbergk <martinvonz@google.com> parents: 43106 diff changeset	2168 b"filename contains '%s', which is invalid on Windows"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2169 ) % stringutil.escapestr(c)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2170 base = n.split(b'.')[0]
14262 23cd7eeff678 util: rename _windows_reserved_filenames and _windows_reserved_chars Adrian Buehlmann <adrian@cadifra.com> parents: 14250 diff changeset	2171 if base and base.lower() in _winreservednames:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2172 return (
43117 8ff1ecfadcd1 cleanup: join string literals that are already on one line Martin von Zweigbergk <martinvonz@google.com> parents: 43106 diff changeset	2173 _(b"filename contains '%s', which is reserved on Windows")
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2174 % base
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2175 )
34361 7508a7dc95c1 py3: replace bytes[n] with slicing in checkwinfilename() Yuya Nishihara <yuya@tcha.org> parents: 34360 diff changeset	2176 t = n[-1:]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2177 if t in b'. ' and n not in b'..':
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2178 return (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2179 _(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2180 b"filename ends with '%s', which is not allowed "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2181 b"on Windows"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2182 )
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2183 % t
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2184 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2185
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2186
44018 f9d29e1d3354 util: avoid referencing `time.clock()` on Windows when missing (issue6238) Matt Harbison <matt_harbison@yahoo.com> parents: 43920 diff changeset	2187 timer = getattr(time, "perf_counter", None)
f9d29e1d3354 util: avoid referencing `time.clock()` on Windows when missing (issue6238) Matt Harbison <matt_harbison@yahoo.com> parents: 43920 diff changeset	2188
34645 75979c8d4572 codemod: use pycompat.iswindows Jun Wu <quark@fb.com> parents: 34554 diff changeset	2189 if pycompat.iswindows:
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2190 checkosfilename = checkwinfilename
44018 f9d29e1d3354 util: avoid referencing `time.clock()` on Windows when missing (issue6238) Matt Harbison <matt_harbison@yahoo.com> parents: 43920 diff changeset	2191 if not timer:
51291 7d3b92e8df13 pytype: ignore attribute error for time.clock Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51284 diff changeset	2192 timer = time.clock # pytype: disable=module-attr
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2193 else:
43883 09bcbeacedc7 typing: suppress a warning that mercurial.windows.checkosfilename is missing Matt Harbison <matt_harbison@yahoo.com> parents: 43882 diff changeset	2194 # mercurial.windows doesn't have platform.checkosfilename
09bcbeacedc7 typing: suppress a warning that mercurial.windows.checkosfilename is missing Matt Harbison <matt_harbison@yahoo.com> parents: 43882 diff changeset	2195 checkosfilename = platform.checkosfilename # pytype: disable=module-attr
44018 f9d29e1d3354 util: avoid referencing `time.clock()` on Windows when missing (issue6238) Matt Harbison <matt_harbison@yahoo.com> parents: 43920 diff changeset	2196 if not timer:
f9d29e1d3354 util: avoid referencing `time.clock()` on Windows when missing (issue6238) Matt Harbison <matt_harbison@yahoo.com> parents: 43920 diff changeset	2197 timer = time.time
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2198
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2199
51885 cbd01bf33802 typing: add type annotations to `mercurial.util.makelock()` Matt Harbison <matt_harbison@yahoo.com> parents: 51884 diff changeset	2200 def makelock(info: bytes, pathname: bytes) -> None:
36701 d77c3b023393 lock: block signal interrupt while making a lock file Yuya Nishihara <yuya@tcha.org> parents: 36699 diff changeset	2201 """Create a lock file atomically if possible
d77c3b023393 lock: block signal interrupt while making a lock file Yuya Nishihara <yuya@tcha.org> parents: 36699 diff changeset	2202
d77c3b023393 lock: block signal interrupt while making a lock file Yuya Nishihara <yuya@tcha.org> parents: 36699 diff changeset	2203 This may leave a stale lock file if symlink isn't supported and signal
d77c3b023393 lock: block signal interrupt while making a lock file Yuya Nishihara <yuya@tcha.org> parents: 36699 diff changeset	2204 interrupt is enabled.
d77c3b023393 lock: block signal interrupt while making a lock file Yuya Nishihara <yuya@tcha.org> parents: 36699 diff changeset	2205 """
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2206 try:
e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2207 return os.symlink(info, pathname)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25420 diff changeset	2208 except OSError as why:
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2209 if why.errno == errno.EEXIST:
e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2210 raise
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2211 except AttributeError: # no symlink in os
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2212 pass
e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2213
36783 1fbbb8e83392 py3: read/write plain lock file in binary mode Yuya Nishihara <yuya@tcha.org> parents: 36782 diff changeset	2214 flags = os.O_CREAT \| os.O_WRONLY \| os.O_EXCL \| getattr(os, 'O_BINARY', 0)
1fbbb8e83392 py3: read/write plain lock file in binary mode Yuya Nishihara <yuya@tcha.org> parents: 36782 diff changeset	2215 ld = os.open(pathname, flags)
51884 f833ad92ee44 util: avoid a leaked file descriptor in `util.makelock()` exceptional case Matt Harbison <matt_harbison@yahoo.com> parents: 51883 diff changeset	2216 try:
f833ad92ee44 util: avoid a leaked file descriptor in `util.makelock()` exceptional case Matt Harbison <matt_harbison@yahoo.com> parents: 51883 diff changeset	2217 os.write(ld, info)
f833ad92ee44 util: avoid a leaked file descriptor in `util.makelock()` exceptional case Matt Harbison <matt_harbison@yahoo.com> parents: 51883 diff changeset	2218 finally:
f833ad92ee44 util: avoid a leaked file descriptor in `util.makelock()` exceptional case Matt Harbison <matt_harbison@yahoo.com> parents: 51883 diff changeset	2219 os.close(ld)
704 5ca319a641e1 Make makelock and readlock work on filesystems without symlink support. Thomas Arendsen Hein <thomas@intevation.de> parents: 698 diff changeset	2220
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2221
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	2222 def readlock(pathname: bytes) -> bytes:
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2223 try:
39904 5fe0b880200e py3: convert os.readlink() path to native strings on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 39777 diff changeset	2224 return readlink(pathname)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25420 diff changeset	2225 except OSError as why:
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2226 if why.errno not in (errno.EINVAL, errno.ENOSYS):
e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2227 raise
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2228 except AttributeError: # no symlink in os
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2229 pass
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2230 with posixfile(pathname, b'rb') as fp:
39905 4017968f0a1d util: use a context manager in readlock() Matt Harbison <matt_harbison@yahoo.com> parents: 39904 diff changeset	2231 return fp.read()
704 5ca319a641e1 Make makelock and readlock work on filesystems without symlink support. Thomas Arendsen Hein <thomas@intevation.de> parents: 698 diff changeset	2232
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2233
2176 9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2234 def fstat(fp):
9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2235 '''stat file object that may not have fileno method.'''
9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2236 try:
9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2237 return os.fstat(fp.fileno())
9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2238 except AttributeError:
9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2239 return os.stat(fp.name)
9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2240
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2241
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2242 # File system features
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2243
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2244
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	2245 def fscasesensitive(path: bytes) -> bool:
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2246 """
18911 451eb1c27c1b util: improve doc for checkcase Mads Kiilerich <mads@kiilerich.com> parents: 18868 diff changeset	2247 Return true if the given path is on a case-sensitive filesystem
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2248
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2249 Requires a path (like /foo/.hg) ending with a foldable final
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2250 directory component.
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2251 """
24902 986a5c23b1c1 util.checkcase: don't abort on broken symlinks Siddharth Agarwal <sid0@fb.com> parents: 24692 diff changeset	2252 s1 = os.lstat(path)
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2253 d, b = os.path.split(path)
15667 eacfd851cb9e icasefs: consider as case sensitive if there is no counterevidence, for safety FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15609 diff changeset	2254 b2 = b.upper()
eacfd851cb9e icasefs: consider as case sensitive if there is no counterevidence, for safety FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15609 diff changeset	2255 if b == b2:
eacfd851cb9e icasefs: consider as case sensitive if there is no counterevidence, for safety FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15609 diff changeset	2256 b2 = b.lower()
eacfd851cb9e icasefs: consider as case sensitive if there is no counterevidence, for safety FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15609 diff changeset	2257 if b == b2:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2258 return True # no evidence against case sensitivity
15667 eacfd851cb9e icasefs: consider as case sensitive if there is no counterevidence, for safety FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15609 diff changeset	2259 p2 = os.path.join(d, b2)
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2260 try:
24902 986a5c23b1c1 util.checkcase: don't abort on broken symlinks Siddharth Agarwal <sid0@fb.com> parents: 24692 diff changeset	2261 s2 = os.lstat(p2)
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2262 if s2 == s1:
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2263 return False
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2264 return True
14004 97ed99d1f419 eliminate various naked except clauses Idan Kamara <idankk86@gmail.com> parents: 13985 diff changeset	2265 except OSError:
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2266 return True
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2267
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2268
46802 112826b59476 re2: feed unicode string to re2 module when necessary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46521 diff changeset	2269 _re2_input = lambda x: x
51607 6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	2270 # google-re2 will need to be tell to not output error on its own
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	2271 _re2_options = None
16943 8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2272 try:
43879 40bd667491a7 pytype: suppress the import-error in util.py when importing re2 Matt Harbison <matt_harbison@yahoo.com> parents: 43754 diff changeset	2273 import re2 # pytype: disable=import-error
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2274
16943 8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2275 _re2 = None
8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2276 except ImportError:
8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2277 _re2 = False
8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2278
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2279
50686 a45460e235a2 re2: fix reporting of availability in `hg debuginstall` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50685 diff changeset	2280 def has_re2():
a45460e235a2 re2: fix reporting of availability in `hg debuginstall` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50685 diff changeset	2281 """return True is re2 is available, False otherwise"""
a45460e235a2 re2: fix reporting of availability in `hg debuginstall` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50685 diff changeset	2282 if _re2 is None:
a45460e235a2 re2: fix reporting of availability in `hg debuginstall` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50685 diff changeset	2283 _re._checkre2()
a45460e235a2 re2: fix reporting of availability in `hg debuginstall` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50685 diff changeset	2284 return _re2
a45460e235a2 re2: fix reporting of availability in `hg debuginstall` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50685 diff changeset	2285
a45460e235a2 re2: fix reporting of availability in `hg debuginstall` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50685 diff changeset	2286
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48940 diff changeset	2287 class _re:
50685 293e1763982e re: make _checkre2 a static method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50684 diff changeset	2288 @staticmethod
293e1763982e re: make _checkre2 a static method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50684 diff changeset	2289 def _checkre2():
21913 50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2290 global _re2
46802 112826b59476 re2: feed unicode string to re2 module when necessary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46521 diff changeset	2291 global _re2_input
51607 6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	2292 global _re2_options
50684 82cf392c99f6 re2: exit `_checkre2` early if calling it is useless Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49861 diff changeset	2293 if _re2 is not None:
82cf392c99f6 re2: exit `_checkre2` early if calling it is useless Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49861 diff changeset	2294 # we already have the answer
82cf392c99f6 re2: exit `_checkre2` early if calling it is useless Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49861 diff changeset	2295 return
46926 9c3e84569071 re2: adjust local variable assignment scope Matt Harbison <matt_harbison@yahoo.com> parents: 46907 diff changeset	2296
9c3e84569071 re2: adjust local variable assignment scope Matt Harbison <matt_harbison@yahoo.com> parents: 46907 diff changeset	2297 check_pattern = br'\[([^\[]+)\]'
9c3e84569071 re2: adjust local variable assignment scope Matt Harbison <matt_harbison@yahoo.com> parents: 46907 diff changeset	2298 check_input = b'[ui]'
21913 50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2299 try:
50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2300 # check if match works, see issue3964
46802 112826b59476 re2: feed unicode string to re2 module when necessary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46521 diff changeset	2301 _re2 = bool(re2.match(check_pattern, check_input))
21913 50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2302 except ImportError:
50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2303 _re2 = False
46802 112826b59476 re2: feed unicode string to re2 module when necessary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46521 diff changeset	2304 except TypeError:
112826b59476 re2: feed unicode string to re2 module when necessary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46521 diff changeset	2305 # the `pyre-2` project provides a re2 module that accept bytes
112826b59476 re2: feed unicode string to re2 module when necessary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46521 diff changeset	2306 # the `fb-re2` project provides a re2 module that acccept sysstr
112826b59476 re2: feed unicode string to re2 module when necessary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46521 diff changeset	2307 check_pattern = pycompat.sysstr(check_pattern)
112826b59476 re2: feed unicode string to re2 module when necessary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46521 diff changeset	2308 check_input = pycompat.sysstr(check_input)
112826b59476 re2: feed unicode string to re2 module when necessary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46521 diff changeset	2309 _re2 = bool(re2.match(check_pattern, check_input))
112826b59476 re2: feed unicode string to re2 module when necessary Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46521 diff changeset	2310 _re2_input = pycompat.sysstr
51607 6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	2311 try:
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	2312 quiet = re2.Options()
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	2313 quiet.log_errors = False
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	2314 _re2_options = quiet
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	2315 except AttributeError:
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	2316 pass
21913 50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2317
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2318 def compile(self, pat, flags=0):
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2319 """Compile a regular expression, using re2 if possible
16943 8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2320
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2321 For best performance, use only re2-compatible regexp features. The
cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2322 only flags from the re module that are re2-compatible are
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2323 IGNORECASE and MULTILINE."""
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2324 if _re2 is None:
21913 50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2325 self._checkre2()
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2326 if _re2 and (flags & ~(remod.IGNORECASE \| remod.MULTILINE)) == 0:
cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2327 if flags & remod.IGNORECASE:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2328 pat = b'(?i)' + pat
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2329 if flags & remod.MULTILINE:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2330 pat = b'(?m)' + pat
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2331 try:
51607 6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	2332 input_regex = _re2_input(pat)
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	2333 if _re2_options is not None:
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	2334 compiled = re2.compile(input_regex, options=_re2_options)
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	2335 else:
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	2336 compiled = re2.compile(input_regex)
6c39edd1d348 re2: make errors quiet Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51427 diff changeset	2337 return compiled
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2338 except re2.error:
cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2339 pass
cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2340 return remod.compile(pat, flags)
cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2341
21914 10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2342 @propertycache
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2343 def escape(self):
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2344 """Return the version of escape corresponding to self.compile.
21914 10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2345
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2346 This is imperfect because whether re2 or re is used for a particular
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2347 function depends on the flags, etc, but it's the best we can do.
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2348 """
21914 10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2349 global _re2
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2350 if _re2 is None:
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2351 self._checkre2()
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2352 if _re2:
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2353 return re2.escape
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2354 else:
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2355 return remod.escape
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2356
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2357
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2358 re = _re()
16943 8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2359
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2360 _fspathcache = {}
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2361
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2362
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	2363 def fspath(name: bytes, root: bytes) -> bytes:
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2364 """Get name in the case stored in the filesystem
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2365
15710 f63e40047372 icasefs: avoid path-absoluteness/existance check in util.fspath() for efficiency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15709 diff changeset	2366 The name should be relative to root, and be normcase-ed for efficiency.
f63e40047372 icasefs: avoid path-absoluteness/existance check in util.fspath() for efficiency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15709 diff changeset	2367
f63e40047372 icasefs: avoid path-absoluteness/existance check in util.fspath() for efficiency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15709 diff changeset	2368 Note that this function is unnecessary, and should not be
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2369 called, for case-sensitive filesystems (simply because it's expensive).
15670 d6c19cfa03ce icasefs: avoid normcase()-ing in util.fspath() for efficiency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15669 diff changeset	2370
15710 f63e40047372 icasefs: avoid path-absoluteness/existance check in util.fspath() for efficiency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15709 diff changeset	2371 The root should be normcase-ed, too.
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2372 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2373
23097 30124c40d11f util.fspath: use a dict rather than a linear scan for lookups Siddharth Agarwal <sid0@fb.com> parents: 23076 diff changeset	2374 def _makefspathcacheentry(dir):
44452 9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things Augie Fackler <augie@google.com> parents: 44436 diff changeset	2375 return {normcase(n): n for n in os.listdir(dir)}
15709 a1f4bd47d18e icasefs: retry directory scan once for already invalidated cache FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15674 diff changeset	2376
30613 1112ff99d965 py3: replace os.sep with pycompat.ossep (part 1 of 4) Pulkit Goyal <7895pulkit@gmail.com> parents: 30536 diff changeset	2377 seps = pycompat.ossep
30625 bcf4a975f93d py3: replace os.altsep with pycompat.altsep Pulkit Goyal <7895pulkit@gmail.com> parents: 30613 diff changeset	2378 if pycompat.osaltsep:
bcf4a975f93d py3: replace os.altsep with pycompat.altsep Pulkit Goyal <7895pulkit@gmail.com> parents: 30613 diff changeset	2379 seps = seps + pycompat.osaltsep
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2380 # Protect backslashes. This gets silly very quickly.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2381 seps.replace(b'\\', b'\\\\')
31496 670166e0fcaa util: use bytes re on bytes input in fspath Augie Fackler <augie@google.com> parents: 31495 diff changeset	2382 pattern = remod.compile(br'([^%s]+)\|([%s]+)' % (seps, seps))
15669 390bcd01775a icasefs: use util.normcase() instead of lower() or os.path.normcase in fspath FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15667 diff changeset	2383 dir = os.path.normpath(root)
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2384 result = []
33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2385 for part, sep in pattern.findall(name):
33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2386 if sep:
33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2387 result.append(sep)
33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2388 continue
33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2389
15719 1dd60426b061 icasefs: follow standard cache look up pattern FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15718 diff changeset	2390 if dir not in _fspathcache:
23097 30124c40d11f util.fspath: use a dict rather than a linear scan for lookups Siddharth Agarwal <sid0@fb.com> parents: 23076 diff changeset	2391 _fspathcache[dir] = _makefspathcacheentry(dir)
15719 1dd60426b061 icasefs: follow standard cache look up pattern FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15718 diff changeset	2392 contents = _fspathcache[dir]
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2393
23097 30124c40d11f util.fspath: use a dict rather than a linear scan for lookups Siddharth Agarwal <sid0@fb.com> parents: 23076 diff changeset	2394 found = contents.get(part)
15709 a1f4bd47d18e icasefs: retry directory scan once for already invalidated cache FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15674 diff changeset	2395 if not found:
15720 3bcfea777efc icasefs: rewrite comment to explain situtation precisely FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15719 diff changeset	2396 # retry "once per directory" per "dirstate.walk" which
3bcfea777efc icasefs: rewrite comment to explain situtation precisely FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15719 diff changeset	2397 # may take place for each patches of "hg qpush", for example
23097 30124c40d11f util.fspath: use a dict rather than a linear scan for lookups Siddharth Agarwal <sid0@fb.com> parents: 23076 diff changeset	2398 _fspathcache[dir] = contents = _makefspathcacheentry(dir)
30124c40d11f util.fspath: use a dict rather than a linear scan for lookups Siddharth Agarwal <sid0@fb.com> parents: 23076 diff changeset	2399 found = contents.get(part)
15709 a1f4bd47d18e icasefs: retry directory scan once for already invalidated cache FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15674 diff changeset	2400
a1f4bd47d18e icasefs: retry directory scan once for already invalidated cache FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15674 diff changeset	2401 result.append(found or part)
15669 390bcd01775a icasefs: use util.normcase() instead of lower() or os.path.normcase in fspath FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15667 diff changeset	2402 dir = os.path.join(dir, part)
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2403
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2404 return b''.join(result)
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2405
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2406
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	2407 def checknlink(testfile: bytes) -> bool:
12938 bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2408 '''check whether hardlink count reporting works properly'''
bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2409
13204 5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2410 # testfile may be open, so we need a separate file for checking to
5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2411 # work around issue2543 (or testfile may get lost on Samba shares)
34084 6c5cdb02f2f9 checknlink: rename file object from 'fd' to 'fp' Jun Wu <quark@fb.com> parents: 34079 diff changeset	2412 f1, f2, fp = None, None, None
12938 bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2413 try:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2414 fd, f1 = pycompat.mkstemp(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2415 prefix=b'.%s-' % os.path.basename(testfile),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2416 suffix=b'1~',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2417 dir=os.path.dirname(testfile),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2418 )
34079 1104718fb090 checknlink: use a random temp file name for checking Jun Wu <quark@fb.com> parents: 34072 diff changeset	2419 os.close(fd)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2420 f2 = b'%s2~' % f1[:-2]
34079 1104718fb090 checknlink: use a random temp file name for checking Jun Wu <quark@fb.com> parents: 34072 diff changeset	2421
25088 754df8e932d3 util: use try/except/finally Matt Mackall <mpm@selenic.com> parents: 24902 diff changeset	2422 oslink(f1, f2)
12938 bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2423 # nlinks() may behave differently for files on Windows shares if
bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2424 # the file is open.
34084 6c5cdb02f2f9 checknlink: rename file object from 'fd' to 'fp' Jun Wu <quark@fb.com> parents: 34079 diff changeset	2425 fp = posixfile(f2)
13204 5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2426 return nlinks(f2) > 1
25088 754df8e932d3 util: use try/except/finally Matt Mackall <mpm@selenic.com> parents: 24902 diff changeset	2427 except OSError:
754df8e932d3 util: use try/except/finally Matt Mackall <mpm@selenic.com> parents: 24902 diff changeset	2428 return False
12938 bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2429 finally:
34084 6c5cdb02f2f9 checknlink: rename file object from 'fd' to 'fp' Jun Wu <quark@fb.com> parents: 34079 diff changeset	2430 if fp is not None:
6c5cdb02f2f9 checknlink: rename file object from 'fd' to 'fp' Jun Wu <quark@fb.com> parents: 34079 diff changeset	2431 fp.close()
13204 5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2432 for f in (f1, f2):
5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2433 try:
34079 1104718fb090 checknlink: use a random temp file name for checking Jun Wu <quark@fb.com> parents: 34072 diff changeset	2434 if f is not None:
1104718fb090 checknlink: use a random temp file name for checking Jun Wu <quark@fb.com> parents: 34072 diff changeset	2435 os.unlink(f)
13204 5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2436 except OSError:
5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2437 pass
12938 bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2438
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2439
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	2440 def endswithsep(path: bytes) -> bool:
5843 83c354c4d529 Add endswithsep() and use it instead of using os.sep and os.altsep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5802 diff changeset	2441 '''Check path ends with os.sep or os.altsep.'''
46645 7711853110b9 typing: add some type annotations to mercurial/util.py Matt Harbison <matt_harbison@yahoo.com> parents: 46113 diff changeset	2442 return bool( # help pytype
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2443 path.endswith(pycompat.ossep)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2444 or pycompat.osaltsep
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2445 and path.endswith(pycompat.osaltsep)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2446 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2447
5843 83c354c4d529 Add endswithsep() and use it instead of using os.sep and os.altsep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5802 diff changeset	2448
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	2449 def splitpath(path: bytes) -> List[bytes]:
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2450 """Split path by os.sep.
5844 07d8eb78dd68 Add util.splitpath() and use it instead of using os.sep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5843 diff changeset	2451 Note that this function does not use os.altsep because this is
07d8eb78dd68 Add util.splitpath() and use it instead of using os.sep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5843 diff changeset	2452 an alternative of simple "xxx.split(os.sep)".
07d8eb78dd68 Add util.splitpath() and use it instead of using os.sep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5843 diff changeset	2453 It is recommended to use os.path.normpath() before using this
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2454 function if need."""
30613 1112ff99d965 py3: replace os.sep with pycompat.ossep (part 1 of 4) Pulkit Goyal <7895pulkit@gmail.com> parents: 30536 diff changeset	2455 return path.split(pycompat.ossep)
5844 07d8eb78dd68 Add util.splitpath() and use it instead of using os.sep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5843 diff changeset	2456
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2457
51796 62806be5cbda typing: add hints to `mercurial.util.mktempcopy()` Matt Harbison <matt_harbison@yahoo.com> parents: 51738 diff changeset	2458 def mktempcopy(
62806be5cbda typing: add hints to `mercurial.util.mktempcopy()` Matt Harbison <matt_harbison@yahoo.com> parents: 51738 diff changeset	2459 name: bytes,
62806be5cbda typing: add hints to `mercurial.util.mktempcopy()` Matt Harbison <matt_harbison@yahoo.com> parents: 51738 diff changeset	2460 emptyok: bool = False,
62806be5cbda typing: add hints to `mercurial.util.mktempcopy()` Matt Harbison <matt_harbison@yahoo.com> parents: 51738 diff changeset	2461 createmode: Optional[int] = None,
62806be5cbda typing: add hints to `mercurial.util.mktempcopy()` Matt Harbison <matt_harbison@yahoo.com> parents: 51738 diff changeset	2462 enforcewritable: bool = False,
62806be5cbda typing: add hints to `mercurial.util.mktempcopy()` Matt Harbison <matt_harbison@yahoo.com> parents: 51738 diff changeset	2463 ) -> bytes:
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2464 """Create a temporary file with the same contents from name
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2465
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2466 The permission bits are copied from the original file.
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2467
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2468 If the temporary file is going to be truncated immediately, you
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2469 can use emptyok=True as an optimization.
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2470
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2471 Returns the name of the temporary file.
2176 9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2472 """
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2473 d, fn = os.path.split(name)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2474 fd, temp = pycompat.mkstemp(prefix=b'.%s-' % fn, suffix=b'~', dir=d)
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2475 os.close(fd)
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2476 # Temporary files are created with mode 0600, which is usually not
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2477 # what we want. If the original file already exists, just copy
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2478 # its mode. Otherwise, manually obey umask.
41289 593f6359681d update: fix edge-case with update.atomic-file and read-only files Boris Feld <boris.feld@octobus.net> parents: 40880 diff changeset	2479 copymode(name, temp, createmode, enforcewritable)
593f6359681d update: fix edge-case with update.atomic-file and read-only files Boris Feld <boris.feld@octobus.net> parents: 40880 diff changeset	2480
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2481 if emptyok:
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2482 return temp
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2483 try:
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2484 try:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2485 ifp = posixfile(name, b"rb")
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25420 diff changeset	2486 except IOError as inst:
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2487 if inst.errno == errno.ENOENT:
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2488 return temp
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2489 if not getattr(inst, 'filename', None):
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2490 inst.filename = name
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2491 raise
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2492 ofp = posixfile(temp, b"wb")
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2493 for chunk in filechunkiter(ifp):
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2494 ofp.write(chunk)
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2495 ifp.close()
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2496 ofp.close()
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2497 except: # re-raises
34435 5326e4ef1dab style: never put multiple statements on one line Alex Gaynor <agaynor@mozilla.com> parents: 34362 diff changeset	2498 try:
5326e4ef1dab style: never put multiple statements on one line Alex Gaynor <agaynor@mozilla.com> parents: 34362 diff changeset	2499 os.unlink(temp)
5326e4ef1dab style: never put multiple statements on one line Alex Gaynor <agaynor@mozilla.com> parents: 34362 diff changeset	2500 except OSError:
5326e4ef1dab style: never put multiple statements on one line Alex Gaynor <agaynor@mozilla.com> parents: 34362 diff changeset	2501 pass
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2502 raise
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2503 return temp
2176 9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2504
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2505
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48940 diff changeset	2506 class filestat:
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2507 """help to exactly detect change of a file
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2508
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2509 'stat' attribute is result of 'os.stat()' if specified 'path'
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2510 exists. Otherwise, it is None. This can avoid preparative
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2511 'exists()' examination on client side of this class.
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2512 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2513
51883 1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51876 diff changeset	2514 def __init__(self, stat: Optional[os.stat_result]) -> None:
32772 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32749 diff changeset	2515 self.stat = stat
7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32749 diff changeset	2516
7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32749 diff changeset	2517 @classmethod
51883 1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51876 diff changeset	2518 def frompath(cls: Type[_Tfilestat], path: bytes) -> _Tfilestat:
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2519 try:
32772 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32749 diff changeset	2520 stat = os.stat(path)
49306 2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT Manuel Jacob <me@manueljacob.de> parents: 49275 diff changeset	2521 except FileNotFoundError:
32772 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32749 diff changeset	2522 stat = None
7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32749 diff changeset	2523 return cls(stat)
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2524
32816 1b25c648d5b7 fsmonitor: don't write out state if identity has changed (issue5581) Siddharth Agarwal <sid0@fb.com> parents: 32772 diff changeset	2525 @classmethod
51883 1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51876 diff changeset	2526 def fromfp(cls: Type[_Tfilestat], fp: BinaryIO) -> _Tfilestat:
32816 1b25c648d5b7 fsmonitor: don't write out state if identity has changed (issue5581) Siddharth Agarwal <sid0@fb.com> parents: 32772 diff changeset	2527 stat = os.fstat(fp.fileno())
1b25c648d5b7 fsmonitor: don't write out state if identity has changed (issue5581) Siddharth Agarwal <sid0@fb.com> parents: 32772 diff changeset	2528 return cls(stat)
1b25c648d5b7 fsmonitor: don't write out state if identity has changed (issue5581) Siddharth Agarwal <sid0@fb.com> parents: 32772 diff changeset	2529
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2530 __hash__ = object.__hash__
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2531
51883 1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51876 diff changeset	2532 def __eq__(self, old) -> bool:
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2533 try:
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2534 # if ambiguity between stat of new and old file is
30332 318a24b52eeb spelling: fixes of non-dictionary words Mads Kiilerich <madski@unity3d.com> parents: 30329 diff changeset	2535 # avoided, comparison of size, ctime and mtime is enough
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2536 # to exactly detect change of a file regardless of platform
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2537 return (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2538 self.stat.st_size == old.stat.st_size
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2539 and self.stat[stat.ST_CTIME] == old.stat[stat.ST_CTIME]
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2540 and self.stat[stat.ST_MTIME] == old.stat[stat.ST_MTIME]
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2541 )
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2542 except AttributeError:
32749 b5524fd9a4e3 util: make filestat.__eq__ return True if both of self and old have None stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32746 diff changeset	2543 pass
b5524fd9a4e3 util: make filestat.__eq__ return True if both of self and old have None stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32746 diff changeset	2544 try:
b5524fd9a4e3 util: make filestat.__eq__ return True if both of self and old have None stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32746 diff changeset	2545 return self.stat is None and old.stat is None
b5524fd9a4e3 util: make filestat.__eq__ return True if both of self and old have None stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32746 diff changeset	2546 except AttributeError:
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2547 return False
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2548
51883 1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51876 diff changeset	2549 def isambig(self, old: _Tfilestat) -> bool:
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2550 """Examine whether new (= self) stat is ambiguous against old one
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2551
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2552 "S[N]" below means stat of a file at N-th change:
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2553
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2554 - S[n-1].ctime < S[n].ctime: can detect change of a file
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2555 - S[n-1].ctime == S[n].ctime
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2556 - S[n-1].ctime < S[n].mtime: means natural advancing (*1)
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2557 - S[n-1].ctime == S[n].mtime: is ambiguous (*2)
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2558 - S[n-1].ctime > S[n].mtime: never occurs naturally (don't care)
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2559 - S[n-1].ctime > S[n].ctime: never occurs naturally (don't care)
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2560
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2561 Case (*2) above means that a file was changed twice or more at
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2562 same time in sec (= S[n-1].ctime), and comparison of timestamp
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2563 is ambiguous.
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2564
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2565 Base idea to avoid such ambiguity is "advance mtime 1 sec, if
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2566 timestamp is ambiguous".
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2567
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2568 But advancing mtime only in case (*2) doesn't work as
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2569 expected, because naturally advanced S[n].mtime in case (*1)
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2570 might be equal to manually advanced S[n-1 or earlier].mtime.
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2571
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2572 Therefore, all "S[n-1].ctime == S[n].ctime" cases should be
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2573 treated as ambiguous regardless of mtime, to avoid overlooking
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2574 by confliction between such mtime.
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2575
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2576 Advancing mtime "if isambig(oldstat)" ensures "S[n-1].mtime !=
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2577 S[n].mtime", even if size of a file isn't changed.
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2578 """
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2579 try:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2580 return self.stat[stat.ST_CTIME] == old.stat[stat.ST_CTIME]
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2581 except AttributeError:
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2582 return False
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2583
51883 1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51876 diff changeset	2584 def avoidambig(self, path: bytes, old: _Tfilestat) -> bool:
30319 b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2585 """Change file stat of specified path to avoid ambiguity
b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2586
b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2587 'old' should be previous filestat of 'path'.
b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2588
b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2589 This skips avoiding ambiguity, if a process doesn't have
32746 77f354ae1123 util: make filestat.avoidambig() return whether ambiguity is avoided or not FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32652 diff changeset	2590 appropriate privileges for 'path'. This returns False in this
77f354ae1123 util: make filestat.avoidambig() return whether ambiguity is avoided or not FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32652 diff changeset	2591 case.
77f354ae1123 util: make filestat.avoidambig() return whether ambiguity is avoided or not FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32652 diff changeset	2592
77f354ae1123 util: make filestat.avoidambig() return whether ambiguity is avoided or not FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32652 diff changeset	2593 Otherwise, this returns True, as "ambiguity is avoided".
30319 b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2594 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2595 advanced = (old.stat[stat.ST_MTIME] + 1) & 0x7FFFFFFF
30319 b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2596 try:
b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2597 os.utime(path, (advanced, advanced))
49308 d2adebe35635 py3: catch PermissionError instead of checking errno == EPERM Manuel Jacob <me@manueljacob.de> parents: 49306 diff changeset	2598 except PermissionError:
d2adebe35635 py3: catch PermissionError instead of checking errno == EPERM Manuel Jacob <me@manueljacob.de> parents: 49306 diff changeset	2599 # utime() on the file created by another user causes EPERM,
d2adebe35635 py3: catch PermissionError instead of checking errno == EPERM Manuel Jacob <me@manueljacob.de> parents: 49306 diff changeset	2600 # if a process doesn't have appropriate privileges
d2adebe35635 py3: catch PermissionError instead of checking errno == EPERM Manuel Jacob <me@manueljacob.de> parents: 49306 diff changeset	2601 return False
32746 77f354ae1123 util: make filestat.avoidambig() return whether ambiguity is avoided or not FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32652 diff changeset	2602 return True
30319 b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2603
51883 1d95a87813ad typing: add type annotations to the `mercurial.util.filestat` class Matt Harbison <matt_harbison@yahoo.com> parents: 51876 diff changeset	2604 def __ne__(self, other) -> bool:
29298 82f6193ff2bc util: add __ne__ to filestat class for consistency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29261 diff changeset	2605 return not self == other
82f6193ff2bc util: add __ne__ to filestat class for consistency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29261 diff changeset	2606
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2607
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48940 diff changeset	2608 class atomictempfile:
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2609 """writable file object that atomically updates a file
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2610
14008 da65edcac72a atomictempfile: rewrite docstring to clarify rename() vs. close(). Greg Ward <greg@gerg.ca> parents: 14007 diff changeset	2611 All writes will go to a temporary copy of the original file. Call
15057 774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2612 close() when you are done writing, and atomictempfile will rename
774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2613 the temporary copy to the original name, making the changes
774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2614 visible. If the object is destroyed without being closed, all your
774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2615 writes are discarded.
29367 4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	2616
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	2617 checkambig argument of constructor is used with filestat, and is
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	2618 useful only if target file is guarded by any lock (e.g. repo.lock
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	2619 or repo.wlock).
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	2620 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2621
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2622 def __init__(self, name, mode=b'w+b', createmode=None, checkambig=False):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2623 self.__name = name # permanent name
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2624 self._tempname = mktempcopy(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2625 name,
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2626 emptyok=(b'w' in mode),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2627 createmode=createmode,
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2628 enforcewritable=(b'w' in mode),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2629 )
41289 593f6359681d update: fix edge-case with update.atomic-file and read-only files Boris Feld <boris.feld@octobus.net> parents: 40880 diff changeset	2630
14007 d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2631 self._fp = posixfile(self._tempname, mode)
29201 a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2632 self._checkambig = checkambig
8327 aa25be1c2889 atomictempfile: delegate to posixfile instead of inheriting from it Bryan O'Sullivan <bos@serpentine.com> parents: 8312 diff changeset	2633
14007 d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2634 # delegated methods
29393 50269a4dce61 atomictempfile: add read to the supported file operations Martijn Pieters <mjpieters@fb.com> parents: 29367 diff changeset	2635 self.read = self._fp.read
14007 d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2636 self.write = self._fp.write
49572 c4f07a011714 util: implement `writelines()` on atomictempfile Matt Harbison <matt_harbison@yahoo.com> parents: 49308 diff changeset	2637 self.writelines = self._fp.writelines
17237 e73128535105 util: delegate seek and tell methods of atomictempfile Bryan O'Sullivan <bryano@fb.com> parents: 17203 diff changeset	2638 self.seek = self._fp.seek
e73128535105 util: delegate seek and tell methods of atomictempfile Bryan O'Sullivan <bryano@fb.com> parents: 17203 diff changeset	2639 self.tell = self._fp.tell
14007 d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2640 self.fileno = self._fp.fileno
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2641
15057 774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2642 def close(self):
8785 7a9151bc5b37 atomictempfile: fix exception in __del__ if mktempcopy fails (self._fp is None) Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8778 diff changeset	2643 if not self._fp.closed:
8327 aa25be1c2889 atomictempfile: delegate to posixfile instead of inheriting from it Bryan O'Sullivan <bos@serpentine.com> parents: 8312 diff changeset	2644 self._fp.close()
29201 a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2645 filename = localpath(self.__name)
32772 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32749 diff changeset	2646 oldstat = self._checkambig and filestat.frompath(filename)
29201 a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2647 if oldstat and oldstat.stat:
a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2648 rename(self._tempname, filename)
32772 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32749 diff changeset	2649 newstat = filestat.frompath(filename)
29201 a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2650 if newstat.isambig(oldstat):
a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2651 # stat of changed file is ambiguous to original one
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2652 advanced = (oldstat.stat[stat.ST_MTIME] + 1) & 0x7FFFFFFF
29201 a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2653 os.utime(filename, (advanced, advanced))
a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2654 else:
a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2655 rename(self._tempname, filename)
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2656
15057 774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2657 def discard(self):
8785 7a9151bc5b37 atomictempfile: fix exception in __del__ if mktempcopy fails (self._fp is None) Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8778 diff changeset	2658 if not self._fp.closed:
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2659 try:
14007 d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2660 os.unlink(self._tempname)
d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2661 except OSError:
d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2662 pass
8327 aa25be1c2889 atomictempfile: delegate to posixfile instead of inheriting from it Bryan O'Sullivan <bos@serpentine.com> parents: 8312 diff changeset	2663 self._fp.close()
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2664
13098 f7d6750dcd01 util: make atomicfiles closable Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 13072 diff changeset	2665 def __del__(self):
50925 d718eddf01d9 safehasattr: drop usage in favor of hasattr Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50750 diff changeset	2666 if hasattr(self, '_fp'): # constructor actually did something
15057 774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2667 self.discard()
13098 f7d6750dcd01 util: make atomicfiles closable Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 13072 diff changeset	2668
29394 6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2669 def __enter__(self):
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2670 return self
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2671
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2672 def __exit__(self, exctype, excvalue, traceback):
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2673 if exctype is not None:
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2674 self.discard()
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2675 else:
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2676 self.close()
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2677
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2678
49190 4ff4e23de7df clone: use better names for temp files Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 48995 diff changeset	2679 def tryrmdir(f):
4ff4e23de7df clone: use better names for temp files Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 48995 diff changeset	2680 try:
4ff4e23de7df clone: use better names for temp files Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 48995 diff changeset	2681 removedirs(f)
4ff4e23de7df clone: use better names for temp files Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 48995 diff changeset	2682 except OSError as e:
4ff4e23de7df clone: use better names for temp files Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 48995 diff changeset	2683 if e.errno != errno.ENOENT and e.errno != errno.ENOTEMPTY:
4ff4e23de7df clone: use better names for temp files Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 48995 diff changeset	2684 raise
4ff4e23de7df clone: use better names for temp files Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 48995 diff changeset	2685
4ff4e23de7df clone: use better names for temp files Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 48995 diff changeset	2686
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	2687 def unlinkpath(
f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	2688 f: bytes, ignoremissing: bool = False, rmdir: bool = True
f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	2689 ) -> None:
31539 52361c4f4dac util: unify unlinkpath Ryan McElroy <rmcelroy@fb.com> parents: 31533 diff changeset	2690 """unlink and remove the directory if it is empty"""
31541 bd9daafbf87c util: use tryunlink in unlinkpath Ryan McElroy <rmcelroy@fb.com> parents: 31540 diff changeset	2691 if ignoremissing:
bd9daafbf87c util: use tryunlink in unlinkpath Ryan McElroy <rmcelroy@fb.com> parents: 31540 diff changeset	2692 tryunlink(f)
bd9daafbf87c util: use tryunlink in unlinkpath Ryan McElroy <rmcelroy@fb.com> parents: 31540 diff changeset	2693 else:
31539 52361c4f4dac util: unify unlinkpath Ryan McElroy <rmcelroy@fb.com> parents: 31533 diff changeset	2694 unlink(f)
38493 da2a7d8354b2 unlinkpath: make empty directory removal optional (issue5901) (issue5826) Kyle Lippincott <spectral@google.com> parents: 38380 diff changeset	2695 if rmdir:
da2a7d8354b2 unlinkpath: make empty directory removal optional (issue5901) (issue5826) Kyle Lippincott <spectral@google.com> parents: 38380 diff changeset	2696 # try removing directories that might now be empty
da2a7d8354b2 unlinkpath: make empty directory removal optional (issue5901) (issue5826) Kyle Lippincott <spectral@google.com> parents: 38380 diff changeset	2697 try:
da2a7d8354b2 unlinkpath: make empty directory removal optional (issue5901) (issue5826) Kyle Lippincott <spectral@google.com> parents: 38380 diff changeset	2698 removedirs(os.path.dirname(f))
da2a7d8354b2 unlinkpath: make empty directory removal optional (issue5901) (issue5826) Kyle Lippincott <spectral@google.com> parents: 38380 diff changeset	2699 except OSError:
da2a7d8354b2 unlinkpath: make empty directory removal optional (issue5901) (issue5826) Kyle Lippincott <spectral@google.com> parents: 38380 diff changeset	2700 pass
31539 52361c4f4dac util: unify unlinkpath Ryan McElroy <rmcelroy@fb.com> parents: 31533 diff changeset	2701
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2702
51427 187c5769a629 vfs: have tryunlink tell what it did Georges Racinet <georges.racinet@octobus.net> parents: 51295 diff changeset	2703 def tryunlink(f: bytes) -> bool:
187c5769a629 vfs: have tryunlink tell what it did Georges Racinet <georges.racinet@octobus.net> parents: 51295 diff changeset	2704 """Attempt to remove a file, ignoring FileNotFoundError.
187c5769a629 vfs: have tryunlink tell what it did Georges Racinet <georges.racinet@octobus.net> parents: 51295 diff changeset	2705
187c5769a629 vfs: have tryunlink tell what it did Georges Racinet <georges.racinet@octobus.net> parents: 51295 diff changeset	2706 Returns False in case the file did not exit, True otherwise
187c5769a629 vfs: have tryunlink tell what it did Georges Racinet <georges.racinet@octobus.net> parents: 51295 diff changeset	2707 """
31540 6d5b77abf306 util: add tryunlink function Ryan McElroy <rmcelroy@fb.com> parents: 31539 diff changeset	2708 try:
6d5b77abf306 util: add tryunlink function Ryan McElroy <rmcelroy@fb.com> parents: 31539 diff changeset	2709 unlink(f)
51427 187c5769a629 vfs: have tryunlink tell what it did Georges Racinet <georges.racinet@octobus.net> parents: 51295 diff changeset	2710 return True
49306 2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT Manuel Jacob <me@manueljacob.de> parents: 49275 diff changeset	2711 except FileNotFoundError:
51427 187c5769a629 vfs: have tryunlink tell what it did Georges Racinet <georges.racinet@octobus.net> parents: 51295 diff changeset	2712 return False
31540 6d5b77abf306 util: add tryunlink function Ryan McElroy <rmcelroy@fb.com> parents: 31539 diff changeset	2713
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2714
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	2715 def makedirs(
f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	2716 name: bytes, mode: Optional[int] = None, notindexed: bool = False
f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	2717 ) -> None:
29017 07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2718 """recursive directory creation with parent mode inheritance
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2719
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2720 Newly created directories are marked as "not to be indexed by
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2721 the content indexing service", if ``notindexed`` is specified
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2722 for "write" mode access.
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2723 """
6062 3c3b126e5619 Make files in .hg inherit the permissions from .hg/store Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 6007 diff changeset	2724 try:
18938 e22107cff6bf util: add notindexed optional parameter to makedirs function Angel Ezquerra <angel.ezquerra@gmail.com> parents: 18930 diff changeset	2725 makedir(name, notindexed)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25420 diff changeset	2726 except OSError as err:
6062 3c3b126e5619 Make files in .hg inherit the permissions from .hg/store Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 6007 diff changeset	2727 if err.errno == errno.EEXIST:
3c3b126e5619 Make files in .hg inherit the permissions from .hg/store Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 6007 diff changeset	2728 return
15058 81f33be0ea79 util: postpone and reorder parent calculation in makedirs Adrian Buehlmann <adrian@cadifra.com> parents: 15057 diff changeset	2729 if err.errno != errno.ENOENT or not name:
81f33be0ea79 util: postpone and reorder parent calculation in makedirs Adrian Buehlmann <adrian@cadifra.com> parents: 15057 diff changeset	2730 raise
47622 bb917eea1605 windows: introduce a `util.abspath` to replace os.path.abspath Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47446 diff changeset	2731 parent = os.path.dirname(abspath(name))
15058 81f33be0ea79 util: postpone and reorder parent calculation in makedirs Adrian Buehlmann <adrian@cadifra.com> parents: 15057 diff changeset	2732 if parent == name:
6062 3c3b126e5619 Make files in .hg inherit the permissions from .hg/store Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 6007 diff changeset	2733 raise
18938 e22107cff6bf util: add notindexed optional parameter to makedirs function Angel Ezquerra <angel.ezquerra@gmail.com> parents: 18930 diff changeset	2734 makedirs(parent, mode, notindexed)
29017 07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2735 try:
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2736 makedir(name, notindexed)
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2737 except OSError as err:
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2738 # Catch EEXIST to handle races
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2739 if err.errno == errno.EEXIST:
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2740 return
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2741 raise
18678 423eee0b0b14 util: make ensuredirs safer against races Bryan O'Sullivan <bryano@fb.com> parents: 18668 diff changeset	2742 if mode is not None:
423eee0b0b14 util: make ensuredirs safer against races Bryan O'Sullivan <bryano@fb.com> parents: 18668 diff changeset	2743 os.chmod(name, mode)
18668 4034b8d551b1 scmutil: create directories in a race-safe way during update Bryan O'Sullivan <bryano@fb.com> parents: 18614 diff changeset	2744
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2745
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	2746 def readfile(path: bytes) -> bytes:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2747 with open(path, b'rb') as fp:
14100 3e9e02a41dfb util: really drop size from readfile Matt Mackall <mpm@selenic.com> parents: 14099 diff changeset	2748 return fp.read()
14099 0824a0a3cefc util: add readfile() & writefile() helper functions Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 14077 diff changeset	2749
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2750
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	2751 def writefile(path: bytes, text: bytes) -> None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2752 with open(path, b'wb') as fp:
14167 0e4753807c93 util & scmutil: adapt read/write helpers as request by mpm Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 14147 diff changeset	2753 fp.write(text)
0e4753807c93 util & scmutil: adapt read/write helpers as request by mpm Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 14147 diff changeset	2754
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2755
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	2756 def appendfile(path: bytes, text: bytes) -> None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2757 with open(path, b'ab') as fp:
14099 0824a0a3cefc util: add readfile() & writefile() helper functions Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 14077 diff changeset	2758 fp.write(text)
0824a0a3cefc util: add readfile() & writefile() helper functions Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 14077 diff changeset	2759
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2760
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48940 diff changeset	2761 class chunkbuffer:
1199 78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an Eric Hopper <hopper@omnifarious.org> parents: 1169 diff changeset	2762 """Allow arbitrary sized chunks of data to be efficiently read from an
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an Eric Hopper <hopper@omnifarious.org> parents: 1169 diff changeset	2763 iterator over chunks of arbitrary size."""
1200 333de1d53846 Minor cleanups. Bryan O'Sullivan <bos@serpentine.com> parents: 1199 diff changeset	2764
5446 fa836e050c50 chunkbuffer: removed unused method and arg Matt Mackall <mpm@selenic.com> parents: 5420 diff changeset	2765 def __init__(self, in_iter):
32123 5f53c267e362 util: remove doc of long gone 'targetsize' argument Martin von Zweigbergk <martinvonz@google.com> parents: 31952 diff changeset	2766 """in_iter is the iterator that's iterating over the input chunks."""
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2767
11670 1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2768 def splitbig(chunks):
1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2769 for chunk in chunks:
51699 ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51698 diff changeset	2770 if len(chunk) > 2**20:
11670 1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2771 pos = 0
1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2772 while pos < len(chunk):
51699 ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51698 diff changeset	2773 end = pos + 2**18
11670 1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2774 yield chunk[pos:end]
1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2775 pos = end
1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2776 else:
1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2777 yield chunk
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2778
11670 1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2779 self.iter = splitbig(in_iter)
25113 0ca8410ea345 util: drop alias for collections.deque Martin von Zweigbergk <martinvonz@google.com> parents: 25112 diff changeset	2780 self._queue = collections.deque()
26480 6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2781 self._chunkoffset = 0
1200 333de1d53846 Minor cleanups. Bryan O'Sullivan <bos@serpentine.com> parents: 1199 diff changeset	2782
21018 c848bfd02366 util: support None size in chunkbuffer.read() Pierre-Yves David <pierre-yves.david@fb.com> parents: 20835 diff changeset	2783 def read(self, l=None):
1200 333de1d53846 Minor cleanups. Bryan O'Sullivan <bos@serpentine.com> parents: 1199 diff changeset	2784 """Read L bytes of data from the iterator of chunks of data.
21018 c848bfd02366 util: support None size in chunkbuffer.read() Pierre-Yves David <pierre-yves.david@fb.com> parents: 20835 diff changeset	2785 Returns less than L bytes if the iterator runs dry.
c848bfd02366 util: support None size in chunkbuffer.read() Pierre-Yves David <pierre-yves.david@fb.com> parents: 20835 diff changeset	2786
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23097 diff changeset	2787 If size parameter is omitted, read everything"""
26478 a3f7e5461dbd util.chunkbuffer: special case reading everything Gregory Szorc <gregory.szorc@gmail.com> parents: 26450 diff changeset	2788 if l is None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2789 return b''.join(self.iter)
26478 a3f7e5461dbd util.chunkbuffer: special case reading everything Gregory Szorc <gregory.szorc@gmail.com> parents: 26450 diff changeset	2790
11758 a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2791 left = l
17962 4c29668ca316 util: make chunkbuffer non-quadratic on Windows Matt Mackall <mpm@selenic.com> parents: 17560 diff changeset	2792 buf = []
16873 37e081609828 util: simplify queue management in chunkbuffer Bryan O'Sullivan <bryano@fb.com> parents: 16834 diff changeset	2793 queue = self._queue
26478 a3f7e5461dbd util.chunkbuffer: special case reading everything Gregory Szorc <gregory.szorc@gmail.com> parents: 26450 diff changeset	2794 while left > 0:
11758 a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2795 # refill the queue
a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2796 if not queue:
51699 ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51698 diff changeset	2797 target = 2**18
11758 a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2798 for chunk in self.iter:
a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2799 queue.append(chunk)
a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2800 target -= len(chunk)
a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2801 if target <= 0:
a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2802 break
a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2803 if not queue:
1199 78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an Eric Hopper <hopper@omnifarious.org> parents: 1169 diff changeset	2804 break
11758 a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2805
26480 6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2806 # The easy way to do this would be to queue.popleft(), modify the
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2807 # chunk (if necessary), then queue.appendleft(). However, for cases
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2808 # where we read partial chunk content, this incurs 2 dequeue
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2809 # mutations and creates a new str for the remaining chunk in the
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2810 # queue. Our code below avoids this overhead.
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2811
26479 46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2812 chunk = queue[0]
46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2813 chunkl = len(chunk)
26480 6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2814 offset = self._chunkoffset
26479 46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2815
46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2816 # Use full chunk.
26480 6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2817 if offset == 0 and left >= chunkl:
26479 46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2818 left -= chunkl
46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2819 queue.popleft()
46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2820 buf.append(chunk)
26480 6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2821 # self._chunkoffset remains at 0.
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2822 continue
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2823
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2824 chunkremaining = chunkl - offset
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2825
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2826 # Use all of unconsumed part of chunk.
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2827 if left >= chunkremaining:
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2828 left -= chunkremaining
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2829 queue.popleft()
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2830 # offset == 0 is enabled by block above, so this won't merely
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2831 # copy via ``chunk[0:]``.
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2832 buf.append(chunk[offset:])
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2833 self._chunkoffset = 0
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2834
26479 46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2835 # Partial chunk needed.
46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2836 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2837 buf.append(chunk[offset : offset + left])
26480 6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2838 self._chunkoffset += left
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2839 left -= chunkremaining
11758 a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2840
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2841 return b''.join(buf)
11758 a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2842
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2843
30181 7356e6b1f5b8 util: increase filechunkiter size to 128k Mads Kiilerich <madski@unity3d.com> parents: 30087 diff changeset	2844 def filechunkiter(f, size=131072, limit=None):
2462 d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2845 """Create a generator that produces the data in the file size
30181 7356e6b1f5b8 util: increase filechunkiter size to 128k Mads Kiilerich <madski@unity3d.com> parents: 30087 diff changeset	2846 (default 131072) bytes at a time, up to optional limit (default is
2462 d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2847 to read all data). Chunks may be less than size bytes if the
d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2848 chunk is the last chunk in the file, or the file is a socket or
d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2849 some other type of file that sometimes reads less data than is
d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2850 requested."""
d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2851 assert size >= 0
d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2852 assert limit is None or limit >= 0
d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2853 while True:
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2854 if limit is None:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2855 nbytes = size
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2856 else:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2857 nbytes = min(limit, size)
2462 d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2858 s = nbytes and f.read(nbytes)
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2859 if not s:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2860 break
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2861 if limit:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2862 limit -= len(s)
1199 78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an Eric Hopper <hopper@omnifarious.org> parents: 1169 diff changeset	2863 yield s
1320 5f277e73778f Fix up representation of dates in hgweb. Bryan O'Sullivan <bos@serpentine.com> parents: 1312 diff changeset	2864
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2865
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48940 diff changeset	2866 class cappedreader:
36362 01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2867 """A file object proxy that allows reading up to N bytes.
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2868
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2869 Given a source file object, instances of this type allow reading up to
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2870 N bytes from that source file object. Attempts to read past the allowed
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2871 limit are treated as EOF.
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2872
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2873 It is assumed that I/O is not performed on the original file object
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2874 in addition to I/O that is performed by this instance. If there is,
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2875 state tracking will get out of sync and unexpected results will ensue.
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2876 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2877
36362 01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2878 def __init__(self, fh, limit):
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2879 """Allow reading up to <limit> bytes from <fh>."""
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2880 self._fh = fh
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2881 self._left = limit
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2882
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2883 def read(self, n=-1):
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2884 if not self._left:
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2885 return b''
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2886
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2887 if n < 0:
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2888 n = self._left
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2889
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2890 data = self._fh.read(min(n, self._left))
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2891 self._left -= len(data)
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2892 assert self._left >= 0
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2893
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2894 return data
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36266 diff changeset	2895
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37044 diff changeset	2896 def readinto(self, b):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37044 diff changeset	2897 res = self.read(len(b))
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37044 diff changeset	2898 if res is None:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37044 diff changeset	2899 return None
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37044 diff changeset	2900
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2901 b[0 : len(res)] = res
37052 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37044 diff changeset	2902 return len(res)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37044 diff changeset	2903
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2904
18735 716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2905 def unitcountfn(*unittable):
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2906 '''return a function that renders a readable count of some quantity'''
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2907
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2908 def go(count):
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2909 for multiplier, divisor, format in unittable:
31946 f3b80537a70d util: fix human-readable printing of negative byte counts G?bor Stefanik <gabor.stefanik@nng.com> parents: 31934 diff changeset	2910 if abs(count) >= divisor * multiplier:
18735 716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2911 return format % (count / float(divisor))
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2912 return unittable[-1][2] % count
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2913
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2914 return go
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2915
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2916
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	2917 def processlinerange(fromline: int, toline: int) -> Tuple[int, int]:
31662 080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31598 diff changeset	2918 """Check that linerange <fromline>:<toline> makes sense and return a
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31598 diff changeset	2919 0-based range.
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31598 diff changeset	2920
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31598 diff changeset	2921 >>> processlinerange(10, 20)
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31598 diff changeset	2922 (9, 20)
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31598 diff changeset	2923 >>> processlinerange(2, 1)
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31598 diff changeset	2924 Traceback (most recent call last):
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31598 diff changeset	2925 ...
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31598 diff changeset	2926 ParseError: line range must be positive
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31598 diff changeset	2927 >>> processlinerange(0, 5)
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31598 diff changeset	2928 Traceback (most recent call last):
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31598 diff changeset	2929 ...
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31598 diff changeset	2930 ParseError: fromline must be strictly positive
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31598 diff changeset	2931 """
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31598 diff changeset	2932 if toline - fromline < 0:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2933 raise error.ParseError(_(b"line range must be positive"))
31662 080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31598 diff changeset	2934 if fromline < 1:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2935 raise error.ParseError(_(b"fromline must be strictly positive"))
31662 080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31598 diff changeset	2936 return fromline - 1, toline
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31598 diff changeset	2937
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2938
18735 716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2939 bytecount = unitcountfn(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2940 (100, 1 << 30, _(b'%.0f GB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2941 (10, 1 << 30, _(b'%.1f GB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2942 (1, 1 << 30, _(b'%.2f GB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2943 (100, 1 << 20, _(b'%.0f MB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2944 (10, 1 << 20, _(b'%.1f MB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2945 (1, 1 << 20, _(b'%.2f MB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2946 (100, 1 << 10, _(b'%.0f KB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2947 (10, 1 << 10, _(b'%.1f KB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2948 (1, 1 << 10, _(b'%.2f KB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2949 (1, 1, _(b'%.0f bytes')),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2950 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2951
16397 f0f7f3fab315 util: create bytecount array just once Matt Mackall <mpm@selenic.com> parents: 16383 diff changeset	2952
51717 ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	2953 class transformingwriter(typelib.BinaryIO_Proxy):
36837 472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36835 diff changeset	2954 """Writable file wrapper to transform data by function"""
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36835 diff changeset	2955
51717 ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	2956 def __init__(self, fp: BinaryIO, encode: Callable[[bytes], bytes]) -> None:
36837 472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36835 diff changeset	2957 self._fp = fp
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36835 diff changeset	2958 self._encode = encode
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36835 diff changeset	2959
51717 ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	2960 def close(self) -> None:
36837 472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36835 diff changeset	2961 self._fp.close()
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36835 diff changeset	2962
51717 ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	2963 def flush(self) -> None:
36837 472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36835 diff changeset	2964 self._fp.flush()
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36835 diff changeset	2965
51717 ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	2966 def write(self, data: bytes) -> int:
36837 472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36835 diff changeset	2967 return self._fp.write(self._encode(data))
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36835 diff changeset	2968
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2969
31776 fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31720 diff changeset	2970 # Matches a single EOL which can either be a CRLF where repeated CR
fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31720 diff changeset	2971 # are removed or a LF. We do not care about old Macintosh files, so a
fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31720 diff changeset	2972 # stray CR is an error.
fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31720 diff changeset	2973 _eolre = remod.compile(br'\r*\n')
fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31720 diff changeset	2974
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2975
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	2976 def tolf(s: bytes) -> bytes:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2977 return _eolre.sub(b'\n', s)
31776 fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31720 diff changeset	2978
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2979
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	2980 def tocrlf(s: bytes) -> bytes:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2981 return _eolre.sub(b'\r\n', s)
31776 fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31720 diff changeset	2982
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2983
51717 ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	2984 def _crlfwriter(fp: typelib.BinaryIO_Proxy) -> typelib.BinaryIO_Proxy:
36837 472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36835 diff changeset	2985 return transformingwriter(fp, tocrlf)
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36835 diff changeset	2986
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	2987
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2988 if pycompat.oslinesep == b'\r\n':
31777 6a5b69b0abec util: add helper to convert between LF and native EOL Yuya Nishihara <yuya@tcha.org> parents: 31776 diff changeset	2989 tonativeeol = tocrlf
6a5b69b0abec util: add helper to convert between LF and native EOL Yuya Nishihara <yuya@tcha.org> parents: 31776 diff changeset	2990 fromnativeeol = tolf
36837 472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36835 diff changeset	2991 nativeeolwriter = _crlfwriter
31777 6a5b69b0abec util: add helper to convert between LF and native EOL Yuya Nishihara <yuya@tcha.org> parents: 31776 diff changeset	2992 else:
6a5b69b0abec util: add helper to convert between LF and native EOL Yuya Nishihara <yuya@tcha.org> parents: 31776 diff changeset	2993 tonativeeol = pycompat.identity
6a5b69b0abec util: add helper to convert between LF and native EOL Yuya Nishihara <yuya@tcha.org> parents: 31776 diff changeset	2994 fromnativeeol = pycompat.identity
36837 472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36835 diff changeset	2995 nativeeolwriter = pycompat.identity
31777 6a5b69b0abec util: add helper to convert between LF and native EOL Yuya Nishihara <yuya@tcha.org> parents: 31776 diff changeset	2996
51717 ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	2997 if typing.TYPE_CHECKING:
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	2998 # Replace the various overloads that come along with aliasing other methods
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	2999 # with the narrow definition that we care about in the type checking phase
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	3000 # only. This ensures that both Windows and POSIX see only the definition
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	3001 # that is actually available.
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	3002
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	3003 def tonativeeol(s: bytes) -> bytes:
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	3004 raise NotImplementedError
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	3005
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	3006 def fromnativeeol(s: bytes) -> bytes:
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	3007 raise NotImplementedError
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	3008
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	3009 def nativeeolwriter(fp: typelib.BinaryIO_Proxy) -> typelib.BinaryIO_Proxy:
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	3010 raise NotImplementedError
ed28085827ec typing: explicitly type some `mercurial.util` eol code to avoid @overload Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	3011
48940 2974cdda819b util: remove iterfile() variant for buggy EINTR handling Gregory Szorc <gregory.szorc@gmail.com> parents: 48913 diff changeset	3012
2974cdda819b util: remove iterfile() variant for buggy EINTR handling Gregory Szorc <gregory.szorc@gmail.com> parents: 48913 diff changeset	3013 # TODO delete since workaround variant for Python 2 no longer needed.
2974cdda819b util: remove iterfile() variant for buggy EINTR handling Gregory Szorc <gregory.szorc@gmail.com> parents: 48913 diff changeset	3014 def iterfile(fp):
2974cdda819b util: remove iterfile() variant for buggy EINTR handling Gregory Szorc <gregory.szorc@gmail.com> parents: 48913 diff changeset	3015 return fp
30395 10514a92860e util: add iterfile to workaround a fileobj.__iter__ issue with EINTR Jun Wu <quark@fb.com> parents: 30359 diff changeset	3016
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3017
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	3018 def iterlines(iterator: Iterable[bytes]) -> Iterator[bytes]:
7879 5c4026a289a4 templater: ability to display diffstat for log-like commands Alexander Solovyov <piranha at piranha.org.ua> parents: 7875 diff changeset	3019 for chunk in iterator:
5c4026a289a4 templater: ability to display diffstat for log-like commands Alexander Solovyov <piranha at piranha.org.ua> parents: 7875 diff changeset	3020 for line in chunk.splitlines():
5c4026a289a4 templater: ability to display diffstat for log-like commands Alexander Solovyov <piranha at piranha.org.ua> parents: 7875 diff changeset	3021 yield line
9610 d78fe60f6bda make path expanding more consistent Alexander Solovyov <piranha@piranha.org.ua> parents: 9569 diff changeset	3022
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3023
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	3024 def expandpath(path: bytes) -> bytes:
9610 d78fe60f6bda make path expanding more consistent Alexander Solovyov <piranha@piranha.org.ua> parents: 9569 diff changeset	3025 return os.path.expanduser(os.path.expandvars(path))
10239 8e4be44a676f Find right hg command for detached process Patrick Mezard <pmezard@gmail.com> parents: 10199 diff changeset	3026
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3027
13392 777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	3028 def interpolate(prefix, mapping, s, fn=None, escape_prefix=False):
11988 8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3029 """Return the result of interpolating items in the mapping into string s.
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3030
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3031 prefix is a single character string, or a two character string with
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3032 a backslash as the first character if the prefix needs to be escaped in
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3033 a regular expression.
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3034
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3035 fn is an optional function that will be applied to the replacement text
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3036 just before replacement.
13392 777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	3037
777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	3038 escape_prefix is an optional flag that allows using doubled prefix for
777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	3039 its escaping.
11988 8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3040 """
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3041 fn = fn or (lambda s: s)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3042 patterns = b'\|'.join(mapping.keys())
13392 777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	3043 if escape_prefix:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3044 patterns += b'\|' + prefix
13392 777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	3045 if len(prefix) > 1:
777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	3046 prefix_char = prefix[1:]
777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	3047 else:
777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	3048 prefix_char = prefix
777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	3049 mapping[prefix_char] = prefix_char
35145 25c543944bc0 py3: add b'' to regular expressions which are raw strings Pulkit Goyal <7895pulkit@gmail.com> parents: 35014 diff changeset	3050 r = remod.compile(br'%s(%s)' % (prefix, patterns))
11988 8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3051 return r.sub(lambda x: fn(mapping[x.group()[1:]]), s)
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	3052
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3053
18736 af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3054 timecount = unitcountfn(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3055 (1, 1e3, _(b'%.0f s')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3056 (100, 1, _(b'%.1f s')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3057 (10, 1, _(b'%.2f s')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3058 (1, 1, _(b'%.3f s')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3059 (100, 0.001, _(b'%.1f ms')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3060 (10, 0.001, _(b'%.2f ms')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3061 (1, 0.001, _(b'%.3f ms')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3062 (100, 0.000001, _(b'%.1f us')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3063 (10, 0.000001, _(b'%.2f us')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3064 (1, 0.000001, _(b'%.3f us')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3065 (100, 0.000000001, _(b'%.1f ns')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3066 (10, 0.000000001, _(b'%.2f ns')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3067 (1, 0.000000001, _(b'%.3f ns')),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3068 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3069
18736 af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3070
38797 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3071 @attr.s
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48940 diff changeset	3072 class timedcmstats:
38797 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3073 """Stats information produced by the timedcm context manager on entering."""
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3074
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3075 # the starting value of the timer as a float (meaning and resulution is
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3076 # platform dependent, see util.timer)
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3077 start = attr.ib(default=attr.Factory(lambda: timer()))
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3078 # the number of seconds as a floating point value; starts at 0, updated when
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3079 # the context is exited.
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3080 elapsed = attr.ib(default=0)
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3081 # the number of nested timedcm context managers.
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3082 level = attr.ib(default=1)
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3083
38812 9d49bb117dde util: make new timedcmstats class Python 3 compatible Martijn Pieters <mj@zopatista.com> parents: 38797 diff changeset	3084 def __bytes__(self):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3085 return timecount(self.elapsed) if self.elapsed else b'<unknown>'
38797 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3086
38812 9d49bb117dde util: make new timedcmstats class Python 3 compatible Martijn Pieters <mj@zopatista.com> parents: 38797 diff changeset	3087 __str__ = encoding.strmethod(__bytes__)
9d49bb117dde util: make new timedcmstats class Python 3 compatible Martijn Pieters <mj@zopatista.com> parents: 38797 diff changeset	3088
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3089
38797 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3090 @contextlib.contextmanager
39259 e00123f63410 util: make timedcm require the label (API) Augie Fackler <augie@google.com> parents: 39258 diff changeset	3091 def timedcm(whencefmt, *whenceargs):
38797 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3092 """A context manager that produces timing information for a given context.
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3093
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3094 On entering a timedcmstats instance is produced.
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3095
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3096 This context manager is reentrant.
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3097
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3098 """
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3099 # track nested context managers
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3100 timedcm._nested += 1
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3101 timing_stats = timedcmstats(level=timedcm._nested)
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3102 try:
39257 497effb0a04a util: make timedcm context manager also emit trace events Augie Fackler <augie@google.com> parents: 39209 diff changeset	3103 with tracing.log(whencefmt, *whenceargs):
497effb0a04a util: make timedcm context manager also emit trace events Augie Fackler <augie@google.com> parents: 39209 diff changeset	3104 yield timing_stats
38797 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3105 finally:
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3106 timing_stats.elapsed = timer() - timing_stats.start
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3107 timedcm._nested -= 1
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3108
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3109
38797 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3110 timedcm._nested = 0
18736 af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3111
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3112
18736 af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3113 def timed(func):
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	3114 """Report the execution time of a function call to stderr.
18736 af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3115
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3116 During development, use as a decorator when you need to measure
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3117 the cost of a function, e.g. as follows:
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3118
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3119 @util.timed
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3120 def foo(a, b, c):
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3121 pass
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	3122 """
18736 af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3123
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3124 def wrapper(args, *kwargs):
39258 331ab85e910b cleanup: make all uses of timedcm specify what they're timing Augie Fackler <augie@google.com> parents: 39257 diff changeset	3125 with timedcm(pycompat.bytestr(func.__name__)) as time_stats:
38797 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3126 result = func(args, *kwargs)
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3127 stderr = procutil.stderr
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3128 stderr.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3129 b'%s%s: %s\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3130 % (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3131 b' ' * time_stats.level * 2,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3132 pycompat.bytestr(func.__name__),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3133 time_stats,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3134 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3135 )
38797 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3136 return result
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3137
18736 af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3138 return wrapper
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3139
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3140
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3141 _sizeunits = (
51699 ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51698 diff changeset	3142 (b'm', 2**20),
ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51698 diff changeset	3143 (b'k', 2**10),
ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51698 diff changeset	3144 (b'g', 2**30),
ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51698 diff changeset	3145 (b'kb', 2**10),
ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51698 diff changeset	3146 (b'mb', 2**20),
ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51698 diff changeset	3147 (b'gb', 2**30),
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3148 (b'b', 1),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3149 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3150
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3151
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	3152 def sizetoint(s: bytes) -> int:
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	3153 """Convert a space specifier to a byte count.
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3154
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	3155 >>> sizetoint(b'30')
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3156 30
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	3157 >>> sizetoint(b'2.2kb')
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3158 2252
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34084 diff changeset	3159 >>> sizetoint(b'6M')
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3160 6291456
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	3161 """
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3162 t = s.strip().lower()
1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3163 try:
1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3164 for k, u in _sizeunits:
1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3165 if t.endswith(k):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3166 return int(float(t[: -len(k)]) * u)
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3167 return int(t)
1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3168 except ValueError:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3169 raise error.ParseError(_(b"couldn't parse size: %s") % s)
19211 3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3170
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3171
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48940 diff changeset	3172 class hooks:
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	3173 """A collection of hook functions that can be used to extend a
26098 ce26928cbe41 spelling: behaviour -> behavior timeless@mozdev.org parents: 25672 diff changeset	3174 function's behavior. Hooks are called in lexicographic order,
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	3175 based on the names of their sources."""
19211 3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3176
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3177 def __init__(self):
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3178 self._hooks = []
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3179
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3180 def add(self, source, hook):
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3181 self._hooks.append((source, hook))
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3182
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3183 def __call__(self, *args):
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3184 self._hooks.sort(key=lambda x: x[0])
21046 cc13addbd62b util: enable "hooks" to return list of the values returned from each hooks FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 21018 diff changeset	3185 results = []
19211 3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3186 for source, hook in self._hooks:
21046 cc13addbd62b util: enable "hooks" to return list of the values returned from each hooks FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 21018 diff changeset	3187 results.append(hook(*args))
cc13addbd62b util: enable "hooks" to return list of the values returned from each hooks FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 21018 diff changeset	3188 return results
20244 47d0843647d1 util: introduce util.debugstacktrace for showing a stack trace without crashing Mads Kiilerich <madski@unity3d.com> parents: 20202 diff changeset	3189
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3190
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3191 def getstackframes(skip=0, line=b' %-*s in %s\n', fileline=b'%s:%d', depth=0):
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	3192 """Yields lines for a nicely formatted stacktrace.
31315 78ac7061f840 util: add debugstacktrace depth limit Mads Kiilerich <madski@unity3d.com> parents: 31314 diff changeset	3193 Skips the 'skip' last entries, then return the last 'depth' entries.
28497 906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3194 Each file+linenumber is formatted according to fileline.
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3195 Each line is formatted according to line.
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3196 If line is None, it yields:
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3197 length of longest filepath+line number,
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3198 filepath+linenumber,
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3199 function
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3200
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3201 Not be used in production code but very convenient while developing.
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	3202 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3203 entries = [
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3204 (fileline % (pycompat.sysbytes(fn), ln), pycompat.sysbytes(func))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3205 for fn, ln, func, _text in traceback.extract_stack()[: -skip - 1]
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3206 ][-depth:]
28497 906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3207 if entries:
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3208 fnmax = max(len(entry[0]) for entry in entries)
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3209 for fnln, func in entries:
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3210 if line is None:
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3211 yield (fnmax, fnln, func)
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3212 else:
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3213 yield line % (fnmax, fnln, func)
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3214
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3215
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3216 def debugstacktrace(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3217 msg=b'stacktrace',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3218 skip=0,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3219 f=procutil.stderr,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3220 otherf=procutil.stdout,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3221 depth=0,
43754 02ededbef627 util: add an optional `prefix` argument to debugstacktrace Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43735 diff changeset	3222 prefix=b'',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3223 ):
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	3224 """Writes a message to f (stderr) with a nicely formatted stacktrace.
31315 78ac7061f840 util: add debugstacktrace depth limit Mads Kiilerich <madski@unity3d.com> parents: 31314 diff changeset	3225 Skips the 'skip' entries closest to the call, then show 'depth' entries.
78ac7061f840 util: add debugstacktrace depth limit Mads Kiilerich <madski@unity3d.com> parents: 31314 diff changeset	3226 By default it will flush stdout first.
28496 b592564a803c util: reword debugstacktrace comment timeless <timeless@mozdev.org> parents: 28027 diff changeset	3227 It can be used everywhere and intentionally does not require an ui object.
20244 47d0843647d1 util: introduce util.debugstacktrace for showing a stack trace without crashing Mads Kiilerich <madski@unity3d.com> parents: 20202 diff changeset	3228 Not be used in production code but very convenient while developing.
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45146 diff changeset	3229 """
20542 be27652675ce util: debugstacktrace, flush before and after writing Mads Kiilerich <madski@unity3d.com> parents: 20353 diff changeset	3230 if otherf:
be27652675ce util: debugstacktrace, flush before and after writing Mads Kiilerich <madski@unity3d.com> parents: 20353 diff changeset	3231 otherf.flush()
43754 02ededbef627 util: add an optional `prefix` argument to debugstacktrace Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43735 diff changeset	3232 f.write(b'%s%s at:\n' % (prefix, msg.rstrip()))
31315 78ac7061f840 util: add debugstacktrace depth limit Mads Kiilerich <madski@unity3d.com> parents: 31314 diff changeset	3233 for line in getstackframes(skip + 1, depth=depth):
43754 02ededbef627 util: add an optional `prefix` argument to debugstacktrace Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43735 diff changeset	3234 f.write(prefix + line)
20542 be27652675ce util: debugstacktrace, flush before and after writing Mads Kiilerich <madski@unity3d.com> parents: 20353 diff changeset	3235 f.flush()
20244 47d0843647d1 util: introduce util.debugstacktrace for showing a stack trace without crashing Mads Kiilerich <madski@unity3d.com> parents: 20202 diff changeset	3236
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3237
20244 47d0843647d1 util: introduce util.debugstacktrace for showing a stack trace without crashing Mads Kiilerich <madski@unity3d.com> parents: 20202 diff changeset	3238 # convenient shortcut
47d0843647d1 util: introduce util.debugstacktrace for showing a stack trace without crashing Mads Kiilerich <madski@unity3d.com> parents: 20202 diff changeset	3239 dst = debugstacktrace
34554 6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3240
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3241
34554 6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3242 def safename(f, tag, ctx, others=None):
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3243 """
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3244 Generate a name that it is safe to rename f to in the given context.
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3245
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3246 f: filename to rename
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3247 tag: a string tag that will be included in the new name
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3248 ctx: a context, in which the new name must not exist
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3249 others: a set of other filenames that the new name must not be in
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3250
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3251 Returns a file name of the form oldname~tag[~number] which does not exist
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3252 in the provided context and is not in the set of other names.
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3253 """
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3254 if others is None:
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3255 others = set()
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3256
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3257 fn = b'%s~%s' % (f, tag)
34554 6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3258 if fn not in ctx and fn not in others:
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3259 return fn
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3260 for n in itertools.count(1):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3261 fn = b'%s~%s~%s' % (f, tag, n)
34554 6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3262 if fn not in ctx and fn not in others:
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34467 diff changeset	3263 return fn
35754 fb0be099063f util: move 'readexactly' in the util module Boris Feld <boris.feld@octobus.net> parents: 35582 diff changeset	3264
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3265
35754 fb0be099063f util: move 'readexactly' in the util module Boris Feld <boris.feld@octobus.net> parents: 35582 diff changeset	3266 def readexactly(stream, n):
fb0be099063f util: move 'readexactly' in the util module Boris Feld <boris.feld@octobus.net> parents: 35582 diff changeset	3267 '''read n bytes from stream.read and abort if less was available'''
fb0be099063f util: move 'readexactly' in the util module Boris Feld <boris.feld@octobus.net> parents: 35582 diff changeset	3268 s = stream.read(n)
fb0be099063f util: move 'readexactly' in the util module Boris Feld <boris.feld@octobus.net> parents: 35582 diff changeset	3269 if len(s) < n:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3270 raise error.Abort(
43117 8ff1ecfadcd1 cleanup: join string literals that are already on one line Martin von Zweigbergk <martinvonz@google.com> parents: 43106 diff changeset	3271 _(b"stream ended unexpectedly (got %d bytes, expected %d)")
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3272 % (len(s), n)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3273 )
35754 fb0be099063f util: move 'readexactly' in the util module Boris Feld <boris.feld@octobus.net> parents: 35582 diff changeset	3274 return s
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3275
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3276
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3277 def uvarintencode(value):
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3278 """Encode an unsigned integer value to a varint.
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3279
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3280 A varint is a variable length integer of 1 or more bytes. Each byte
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3281 except the last has the most significant bit set. The lower 7 bits of
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3282 each byte store the 2's complement representation, least significant group
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3283 first.
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3284
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3285 >>> uvarintencode(0)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3286 '\\x00'
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3287 >>> uvarintencode(1)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3288 '\\x01'
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3289 >>> uvarintencode(127)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3290 '\\x7f'
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3291 >>> uvarintencode(1337)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3292 '\\xb9\\n'
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3293 >>> uvarintencode(65536)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3294 '\\x80\\x80\\x04'
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3295 >>> uvarintencode(-1)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3296 Traceback (most recent call last):
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3297 ...
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3298 ProgrammingError: negative value for uvarint: -1
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3299 """
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3300 if value < 0:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3301 raise error.ProgrammingError(b'negative value for uvarint: %d' % value)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3302 bits = value & 0x7F
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3303 value >>= 7
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3304 bytes = []
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3305 while value:
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3306 bytes.append(pycompat.bytechr(0x80 \| bits))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3307 bits = value & 0x7F
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3308 value >>= 7
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3309 bytes.append(pycompat.bytechr(bits))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3310
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3311 return b''.join(bytes)
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3312
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3313
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3314 def uvarintdecodestream(fh):
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3315 """Decode an unsigned variable length integer from a stream.
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3316
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3317 The passed argument is anything that has a ``.read(N)`` method.
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3318
49813 54114bba7c7e tests: drop some obsolete py2 handling in util.py doctest Matt Harbison <matt_harbison@yahoo.com> parents: 49572 diff changeset	3319 >>> from io import BytesIO
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3320 >>> uvarintdecodestream(BytesIO(b'\\x00'))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3321 0
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3322 >>> uvarintdecodestream(BytesIO(b'\\x01'))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3323 1
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3324 >>> uvarintdecodestream(BytesIO(b'\\x7f'))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3325 127
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3326 >>> uvarintdecodestream(BytesIO(b'\\xb9\\n'))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3327 1337
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3328 >>> uvarintdecodestream(BytesIO(b'\\x80\\x80\\x04'))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3329 65536
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3330 >>> uvarintdecodestream(BytesIO(b'\\x80'))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3331 Traceback (most recent call last):
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3332 ...
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3333 Abort: stream ended unexpectedly (got 0 bytes, expected 1)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3334 """
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3335 result = 0
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3336 shift = 0
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3337 while True:
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3338 byte = ord(readexactly(fh, 1))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42766 diff changeset	3339 result \|= (byte & 0x7F) << shift
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3340 if not (byte & 0x80):
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3341 return result
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3342 shift += 7
45011 1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3343
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3344
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3345 # Passing the '' locale means that the locale should be set according to the
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3346 # user settings (environment variables).
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3347 # Python sometimes avoids setting the global locale settings. When interfacing
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3348 # with C code (e.g. the curses module or the Subversion bindings), the global
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3349 # locale settings must be initialized correctly. Python 2 does not initialize
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3350 # the global locale settings on interpreter startup. Python 3 sometimes
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3351 # initializes LC_CTYPE, but not consistently at least on Windows. Therefore we
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3352 # explicitly initialize it to get consistent behavior if it's not already
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3353 # initialized. Since CPython commit 177d921c8c03d30daa32994362023f777624b10d,
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3354 # LC_CTYPE is always initialized. If we require Python 3.8+, we should re-check
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3355 # if we can remove this code.
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3356 @contextlib.contextmanager
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3357 def with_lc_ctype():
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3358 oldloc = locale.setlocale(locale.LC_CTYPE, None)
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3359 if oldloc == 'C':
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3360 try:
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3361 try:
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3362 locale.setlocale(locale.LC_CTYPE, '')
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3363 except locale.Error:
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3364 # The likely case is that the locale from the environment
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3365 # variables is unknown.
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3366 pass
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3367 yield
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3368 finally:
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3369 locale.setlocale(locale.LC_CTYPE, oldloc)
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3370 else:
1bab6b61b62b curses: do not initialize LC_ALL to user settings (issue6358) Manuel Jacob <me@manueljacob.de> parents: 44629 diff changeset	3371 yield
45061 02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3372
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3373
51284 f15cb5111a1e pytype: move some type comment to proper annotation Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51282 diff changeset	3374 def _estimatememory() -> Optional[int]:
45061 02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3375 """Provide an estimate for the available system memory in Bytes.
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3376
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3377 If no estimate can be provided on the platform, returns None.
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3378 """
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3379 if pycompat.sysplatform.startswith(b'win'):
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3380 # On Windows, use the GlobalMemoryStatusEx kernel function directly.
51876 8f3cbea2547c util: add a comment to suppress a PyCharm warning about a PEP 8 violation Matt Harbison <matt_harbison@yahoo.com> parents: 51859 diff changeset	3381 # noinspection PyPep8Naming
45061 02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3382 from ctypes import c_long as DWORD, c_ulonglong as DWORDLONG
46645 7711853110b9 typing: add some type annotations to mercurial/util.py Matt Harbison <matt_harbison@yahoo.com> parents: 46113 diff changeset	3383 from ctypes.wintypes import ( # pytype: disable=import-error
7711853110b9 typing: add some type annotations to mercurial/util.py Matt Harbison <matt_harbison@yahoo.com> parents: 46113 diff changeset	3384 Structure,
7711853110b9 typing: add some type annotations to mercurial/util.py Matt Harbison <matt_harbison@yahoo.com> parents: 46113 diff changeset	3385 byref,
7711853110b9 typing: add some type annotations to mercurial/util.py Matt Harbison <matt_harbison@yahoo.com> parents: 46113 diff changeset	3386 sizeof,
7711853110b9 typing: add some type annotations to mercurial/util.py Matt Harbison <matt_harbison@yahoo.com> parents: 46113 diff changeset	3387 windll,
7711853110b9 typing: add some type annotations to mercurial/util.py Matt Harbison <matt_harbison@yahoo.com> parents: 46113 diff changeset	3388 )
45061 02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3389
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3390 class MEMORYSTATUSEX(Structure):
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3391 _fields_ = [
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3392 ('dwLength', DWORD),
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3393 ('dwMemoryLoad', DWORD),
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3394 ('ullTotalPhys', DWORDLONG),
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3395 ('ullAvailPhys', DWORDLONG),
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3396 ('ullTotalPageFile', DWORDLONG),
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3397 ('ullAvailPageFile', DWORDLONG),
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3398 ('ullTotalVirtual', DWORDLONG),
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3399 ('ullAvailVirtual', DWORDLONG),
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3400 ('ullExtendedVirtual', DWORDLONG),
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3401 ]
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3402
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3403 x = MEMORYSTATUSEX()
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3404 x.dwLength = sizeof(x)
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3405 windll.kernel32.GlobalMemoryStatusEx(byref(x))
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3406 return x.ullAvailPhys
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3407
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3408 # On newer Unix-like systems and Mac OSX, the sysconf interface
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3409 # can be used. _SC_PAGE_SIZE is part of POSIX; _SC_PHYS_PAGES
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3410 # seems to be implemented on most systems.
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3411 try:
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3412 pagesize = os.sysconf(os.sysconf_names['SC_PAGE_SIZE'])
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3413 pages = os.sysconf(os.sysconf_names['SC_PHYS_PAGES'])
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3414 return pagesize * pages
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3415 except OSError: # sysconf can fail
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3416 pass
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3417 except KeyError: # unknown parameter
02b17231f6c3 util: provide a helper function to estimate RAM size Joerg Sonnenberger <joerg@bec.de> parents: 45019 diff changeset	3418 pass

Mercurial > public > mercurial-scm > hg

annotate mercurial/util.py @ 52049:af54626bf358