Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/pycompat.py @ 48962:79009cca491e

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
pycompat: remove large Python 2 block We no longer support Python 2. So we can delete its compatibility code and remove the conditional and dedent the Python 3 code. In order to make the linter happy, we had to inline imports in the stanza at the top of the file. Differential Revision: https://phab.mercurial-scm.org/D12250
author Gregory Szorc <gregory.szorc@gmail.com>
date Thu, 03 Mar 2022 07:59:42 -0800
parents 57b58413dad1
children 5aafc3c5bdec
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
28818
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
1 # pycompat.py - portability shim for python 3
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
2 #
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
3 # This software may be used and distributed according to the terms of the
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
4 # GNU General Public License version 2 or any later version.
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
5
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
6 """Mercurial portability shim for python 3.
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
7
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
8 This contains aliases to hide python version-specific details from the core.
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
9 """
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
10
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
11 from __future__ import absolute_import
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
12
48960
57b58413dad1 pycompat: remove first not ispy3 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48926
diff changeset
13 import builtins
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
14 import codecs
48960
57b58413dad1 pycompat: remove first not ispy3 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48926
diff changeset
15 import concurrent.futures as futures
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
16 import functools
30583
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30509
diff changeset
17 import getopt
48960
57b58413dad1 pycompat: remove first not ispy3 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48926
diff changeset
18 import http.client as httplib
57b58413dad1 pycompat: remove first not ispy3 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48926
diff changeset
19 import http.cookiejar as cookielib
36216
646002338365 py3: introduce and use pycompat.getargspec
Augie Fackler <augie@google.com>
parents: 36083
diff changeset
20 import inspect
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
21 import io
43380
579672b347d2 py3: define and use json.loads polyfill
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43117
diff changeset
22 import json
30315
3874ddba1ab4 py3: add a bytes version of os.name
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30313
diff changeset
23 import os
48960
57b58413dad1 pycompat: remove first not ispy3 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48926
diff changeset
24 import pickle
57b58413dad1 pycompat: remove first not ispy3 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48926
diff changeset
25 import queue
30681
caf7e1c5efe4 py3: have a bytes version of shlex.split()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30671
diff changeset
26 import shlex
48960
57b58413dad1 pycompat: remove first not ispy3 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48926
diff changeset
27 import socketserver
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
28 import struct
29584
06587edd1233 pycompat: make pycompat demandimport friendly
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29566
diff changeset
29 import sys
38197
aac4be30e250 py3: wrap tempfile.mkstemp() to use bytes path
Yuya Nishihara <yuya@tcha.org>
parents: 37890
diff changeset
30 import tempfile
48960
57b58413dad1 pycompat: remove first not ispy3 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48926
diff changeset
31 import xmlrpc.client as xmlrpclib
29584
06587edd1233 pycompat: make pycompat demandimport friendly
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29566
diff changeset
32
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
33
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43021
diff changeset
34 ispy3 = sys.version_info[0] >= 3
43554
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43551
diff changeset
35 ispypy = '__pypy__' in sys.builtin_module_names
43793
7b14d649af1b typing: consolidate "if not globals():" trick
Yuya Nishihara <yuya@tcha.org>
parents: 43788
diff changeset
36 TYPE_CHECKING = False
7b14d649af1b typing: consolidate "if not globals():" trick
Yuya Nishihara <yuya@tcha.org>
parents: 43788
diff changeset
37
7b14d649af1b typing: consolidate "if not globals():" trick
Yuya Nishihara <yuya@tcha.org>
parents: 43788
diff changeset
38 if not globals(): # hide this from non-pytype users
7b14d649af1b typing: consolidate "if not globals():" trick
Yuya Nishihara <yuya@tcha.org>
parents: 43788
diff changeset
39 import typing
7b14d649af1b typing: consolidate "if not globals():" trick
Yuya Nishihara <yuya@tcha.org>
parents: 43788
diff changeset
40
7b14d649af1b typing: consolidate "if not globals():" trick
Yuya Nishihara <yuya@tcha.org>
parents: 43788
diff changeset
41 TYPE_CHECKING = typing.TYPE_CHECKING
30031
0f6d6fdd3c2a pycompat: provide 'ispy3' constant
Yuya Nishihara <yuya@tcha.org>
parents: 29812
diff changeset
42
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43021
diff changeset
43
48960
57b58413dad1 pycompat: remove first not ispy3 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48926
diff changeset
44 def future_set_exception_info(f, exc_info):
57b58413dad1 pycompat: remove first not ispy3 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48926
diff changeset
45 f.set_exception(exc_info[0])
29431
80880ad3fccd py3: conditionalize the urlparse import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29414
diff changeset
46
37669
1cb54e6193a6 py3: paper over differences in future exception handling
Augie Fackler <augie@google.com>
parents: 37628
diff changeset
47
48960
57b58413dad1 pycompat: remove first not ispy3 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48926
diff changeset
48 FileNotFoundError = builtins.FileNotFoundError
47852
2b76255a4f74 template: FileNotFoundError is actually a built in exception
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46380
diff changeset
49
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43021
diff changeset
50
31777
7d2cbe11ae48 pycompat: introduce identity function as a compat stub
Yuya Nishihara <yuya@tcha.org>
parents: 31579
diff changeset
51 def identity(a):
7d2cbe11ae48 pycompat: introduce identity function as a compat stub
Yuya Nishihara <yuya@tcha.org>
parents: 31579
diff changeset
52 return a
7d2cbe11ae48 pycompat: introduce identity function as a compat stub
Yuya Nishihara <yuya@tcha.org>
parents: 31579
diff changeset
53
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43021
diff changeset
54
38576
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
55 def _rapply(f, xs):
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
56 if xs is None:
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
57 # assume None means non-value of optional data
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
58 return xs
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
59 if isinstance(xs, (list, set, tuple)):
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
60 return type(xs)(_rapply(f, x) for x in xs)
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
61 if isinstance(xs, dict):
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
62 return type(xs)((_rapply(f, k), _rapply(f, v)) for k, v in xs.items())
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
63 return f(xs)
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
64
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43021
diff changeset
65
38576
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
66 def rapply(f, xs):
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
67 """Apply function recursively to every item preserving the data structure
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
68
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
69 >>> def f(x):
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
70 ... return 'f(%s)' % x
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
71 >>> rapply(f, None) is None
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
72 True
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
73 >>> rapply(f, 'a')
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
74 'f(a)'
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
75 >>> rapply(f, {'a'}) == {'f(a)'}
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
76 True
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
77 >>> rapply(f, ['a', 'b', None, {'c': 'd'}, []])
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
78 ['f(a)', 'f(b)', None, {'f(c)': 'f(d)'}, []]
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
79
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
80 >>> xs = [object()]
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
81 >>> rapply(identity, xs) is xs
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
82 True
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
83 """
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
84 if f is identity:
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
85 # fast path mainly for py2
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
86 return xs
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
87 return _rapply(f, xs)
152f4822d210 pycompat: move rapply() from util
Yuya Nishihara <yuya@tcha.org>
parents: 38341
diff changeset
88
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43021
diff changeset
89
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
90 if os.name == r'nt' and sys.version_info >= (3, 6):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
91 # MBCS (or ANSI) filesystem encoding must be used as before.
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
92 # Otherwise non-ASCII filenames in existing repositories would be
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
93 # corrupted.
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
94 # This must be set once prior to any fsencode/fsdecode calls.
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
95 sys._enablelegacywindowsfsencoding() # pytype: disable=module-attr
43484
8d5489b048b7 py3: enable legacy fs encoding to fix filename compatibility on Windows
Yuya Nishihara <yuya@tcha.org>
parents: 43117
diff changeset
96
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
97 fsencode = os.fsencode
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
98 fsdecode = os.fsdecode
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
99 oscurdir = os.curdir.encode('ascii')
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
100 oslinesep = os.linesep.encode('ascii')
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
101 osname = os.name.encode('ascii')
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
102 ospathsep = os.pathsep.encode('ascii')
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
103 ospardir = os.pardir.encode('ascii')
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
104 ossep = os.sep.encode('ascii')
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
105 osaltsep = os.altsep
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
106 if osaltsep:
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
107 osaltsep = osaltsep.encode('ascii')
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
108 osdevnull = os.devnull.encode('ascii')
39823
24e493ec2229 py3: rename pycompat.getcwd() to encoding.getcwd() (API)
Matt Harbison <matt_harbison@yahoo.com>
parents: 39658
diff changeset
109
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
110 sysplatform = sys.platform.encode('ascii')
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
111 sysexecutable = sys.executable
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
112 if sysexecutable:
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
113 sysexecutable = os.fsencode(sysexecutable)
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
114 bytesio = io.BytesIO
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
115 # TODO deprecate stringio name, as it is a lie on Python 3.
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
116 stringio = bytesio
36940
dbae581010ea pycompat: name maplist() and ziplist() for better traceback message
Yuya Nishihara <yuya@tcha.org>
parents: 36677
diff changeset
117
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
118
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
119 def maplist(*args):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
120 return list(map(*args))
36940
dbae581010ea pycompat: name maplist() and ziplist() for better traceback message
Yuya Nishihara <yuya@tcha.org>
parents: 36677
diff changeset
121
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
122
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
123 def rangelist(*args):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
124 return list(range(*args))
37067
434e520adb8c annotate: do not construct attr.s object per line while computing history
Yuya Nishihara <yuya@tcha.org>
parents: 36964
diff changeset
125
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
126
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
127 def ziplist(*args):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
128 return list(zip(*args))
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
129
36940
dbae581010ea pycompat: name maplist() and ziplist() for better traceback message
Yuya Nishihara <yuya@tcha.org>
parents: 36677
diff changeset
130
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
131 rawinput = input
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
132 getargspec = inspect.getfullargspec
30344
19d8e19fde5b py3: document why os.fsencode() can be used to get back bytes argv
Yuya Nishihara <yuya@tcha.org>
parents: 30340
diff changeset
133
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
134 long = int
39473
8d858fbf2759 cbor: teach the encoder to handle python `long` type for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 39329
diff changeset
135
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
136 if getattr(sys, 'argv', None) is not None:
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
137 # On POSIX, the char** argv array is converted to Python str using
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
138 # Py_DecodeLocale(). The inverse of this is Py_EncodeLocale(), which
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
139 # isn't directly callable from Python code. In practice, os.fsencode()
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
140 # can be used instead (this is recommended by Python's documentation
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
141 # for sys.argv).
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
142 #
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
143 # On Windows, the wchar_t **argv is passed into the interpreter as-is.
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
144 # Like POSIX, we need to emulate what Py_EncodeLocale() would do. But
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
145 # there's an additional wrinkle. What we really want to access is the
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
146 # ANSI codepage representation of the arguments, as this is what
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
147 # `int main()` would receive if Python 3 didn't define `int wmain()`
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
148 # (this is how Python 2 worked). To get that, we encode with the mbcs
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
149 # encoding, which will pass CP_ACP to the underlying Windows API to
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
150 # produce bytes.
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
151 if os.name == r'nt':
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
152 sysargv = [a.encode("mbcs", "ignore") for a in sys.argv]
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
153 else:
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
154 sysargv = [fsencode(a) for a in sys.argv]
29808
965c91bad9e3 py3: move xrange alias next to import lines
Yuya Nishihara <yuya@tcha.org>
parents: 29790
diff changeset
155
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
156 bytechr = struct.Struct('>B').pack
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
157 byterepr = b'%r'.__mod__
31448
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
158
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
159
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
160 class bytestr(bytes):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
161 """A bytes which mostly acts as a Python 2 str
32488
548478efc46c pycompat: try __bytes__() to convert object to bytestr
Yuya Nishihara <yuya@tcha.org>
parents: 32232
diff changeset
162
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
163 >>> bytestr(), bytestr(bytearray(b'foo')), bytestr(u'ascii'), bytestr(1)
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
164 ('', 'foo', 'ascii', '1')
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
165 >>> s = bytestr(b'foo')
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
166 >>> assert s is bytestr(s)
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
167
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
168 __bytes__() should be called if provided:
32488
548478efc46c pycompat: try __bytes__() to convert object to bytestr
Yuya Nishihara <yuya@tcha.org>
parents: 32232
diff changeset
169
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
170 >>> class bytesable(object):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
171 ... def __bytes__(self):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
172 ... return b'bytes'
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
173 >>> bytestr(bytesable())
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
174 'bytes'
31448
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
175
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
176 There's no implicit conversion from non-ascii str as its encoding is
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
177 unknown:
31448
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
178
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
179 >>> bytestr(chr(0x80)) # doctest: +ELLIPSIS
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
180 Traceback (most recent call last):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
181 ...
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
182 UnicodeEncodeError: ...
31448
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
183
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
184 Comparison between bytestr and bytes should work:
31448
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
185
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
186 >>> assert bytestr(b'foo') == b'foo'
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
187 >>> assert b'foo' == bytestr(b'foo')
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
188 >>> assert b'f' in bytestr(b'foo')
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
189 >>> assert bytestr(b'f') in b'foo'
31448
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
190
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
191 Sliced elements should be bytes, not integer:
31448
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
192
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
193 >>> s[1], s[:2]
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
194 (b'o', b'fo')
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
195 >>> list(s), list(reversed(s))
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
196 ([b'f', b'o', b'o'], [b'o', b'o', b'f'])
31448
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
197
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
198 As bytestr type isn't propagated across operations, you need to cast
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
199 bytes to bytestr explicitly:
31448
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
200
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
201 >>> s = bytestr(b'foo').upper()
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
202 >>> t = bytestr(s)
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
203 >>> s[0], t[0]
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
204 (70, b'F')
48029
1fda8c9358ce typing: add a fake `__init__()` to bytestr to distract pytype
Matt Harbison <matt_harbison@yahoo.com>
parents: 47852
diff changeset
205
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
206 Be careful to not pass a bytestr object to a function which expects
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
207 bytearray-like behavior.
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
208
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
209 >>> t = bytes(t) # cast to bytes
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
210 >>> assert type(t) is bytes
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
211 """
48029
1fda8c9358ce typing: add a fake `__init__()` to bytestr to distract pytype
Matt Harbison <matt_harbison@yahoo.com>
parents: 47852
diff changeset
212
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
213 # Trick pytype into not demanding Iterable[int] be passed to __new__(),
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
214 # since the appropriate bytes format is done internally.
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
215 #
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
216 # https://github.com/google/pytype/issues/500
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
217 if TYPE_CHECKING:
31448
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
218
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
219 def __init__(self, s=b''):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
220 pass
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
221
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
222 def __new__(cls, s=b''):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
223 if isinstance(s, bytestr):
31448
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
224 return s
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
225 if not isinstance(
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
226 s, (bytes, bytearray)
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
227 ) and not hasattr( # hasattr-py3-only
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
228 s, u'__bytes__'
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
229 ):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
230 s = str(s).encode('ascii')
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
231 return bytes.__new__(cls, s)
35943
1a31111e6239 py3: always drop b'' prefix from repr() of bytestr
Yuya Nishihara <yuya@tcha.org>
parents: 35412
diff changeset
232
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
233 def __getitem__(self, key):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
234 s = bytes.__getitem__(self, key)
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
235 if not isinstance(s, bytes):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
236 s = bytechr(s)
35944
fc44c2657dc5 py3: drop b'' from repr() of smartset
Yuya Nishihara <yuya@tcha.org>
parents: 35943
diff changeset
237 return s
fc44c2657dc5 py3: drop b'' from repr() of smartset
Yuya Nishihara <yuya@tcha.org>
parents: 35943
diff changeset
238
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
239 def __iter__(self):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
240 return iterbytestr(bytes.__iter__(self))
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
241
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
242 def __repr__(self):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
243 return bytes.__repr__(self)[1:] # drop b''
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
244
31820
45761ef1bc93 py3: have registrar process docstrings in bytes
Yuya Nishihara <yuya@tcha.org>
parents: 31778
diff changeset
245
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
246 def iterbytestr(s):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
247 """Iterate bytes as if it were a str object of Python 2"""
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
248 return map(bytechr, s)
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
249
31820
45761ef1bc93 py3: have registrar process docstrings in bytes
Yuya Nishihara <yuya@tcha.org>
parents: 31778
diff changeset
250
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
251 def maybebytestr(s):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
252 """Promote bytes to bytestr"""
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
253 if isinstance(s, bytes):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
254 return bytestr(s)
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
255 return s
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
256
30032
2219f4f82ede pycompat: extract function that converts attribute or encoding name to str
Yuya Nishihara <yuya@tcha.org>
parents: 30031
diff changeset
257
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
258 def sysbytes(s):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
259 """Convert an internal str (e.g. keyword, __doc__) back to bytes
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
260
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
261 This never raises UnicodeEncodeError, but only ASCII characters
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
262 can be round-trip by sysstr(sysbytes(s)).
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
263 """
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
264 if isinstance(s, bytes):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
265 return s
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
266 return s.encode('utf-8')
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
267
30032
2219f4f82ede pycompat: extract function that converts attribute or encoding name to str
Yuya Nishihara <yuya@tcha.org>
parents: 30031
diff changeset
268
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
269 def sysstr(s):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
270 """Return a keyword str to be passed to Python functions such as
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
271 getattr() and str.encode()
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
272
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
273 This never raises UnicodeDecodeError. Non-ascii characters are
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
274 considered invalid and mapped to arbitrary but unique code points
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
275 such that 'sysstr(a) != sysstr(b)' for all 'a != b'.
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
276 """
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
277 if isinstance(s, builtins.str):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
278 return s
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
279 return s.decode('latin-1')
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
280
32877
a05f3675c46a py3: add a new strurl() which will convert a bytes url to str
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32638
diff changeset
281
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
282 def strurl(url):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
283 """Converts a bytes url back to str"""
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
284 if isinstance(url, bytes):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
285 return url.decode('ascii')
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
286 return url
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
287
32878
f22f39d56bb5 py3: add a new bytesurl() to convert a str url into bytes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32877
diff changeset
288
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
289 def bytesurl(url):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
290 """Converts a str url to bytes by encoding in ascii"""
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
291 if isinstance(url, str):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
292 return url.encode('ascii')
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
293 return url
32232
76f9a0009b4b pycompat: extract helper to raise exception with traceback
Yuya Nishihara <yuya@tcha.org>
parents: 31942
diff changeset
294
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
295
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
296 def raisewithtb(exc, tb):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
297 """Raise exception with the given traceback"""
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
298 raise exc.with_traceback(tb)
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
299
32638
c9318beb7c1a py3: convert __doc__ back to bytes in help.py
Yuya Nishihara <yuya@tcha.org>
parents: 32488
diff changeset
300
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
301 def getdoc(obj):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
302 """Get docstring as bytes; may be None so gettext() won't confuse it
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
303 with _('')"""
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
304 doc = getattr(obj, '__doc__', None)
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
305 if doc is None:
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
306 return doc
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
307 return sysbytes(doc)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43021
diff changeset
308
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
309
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
310 def _wrapattrfunc(f):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
311 @functools.wraps(f)
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
312 def w(object, name, *args):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
313 return f(object, sysstr(name), *args)
29810
45fa8de47a0f py3: provide (del|get|has|set)attr wrappers that accepts bytes
Yuya Nishihara <yuya@tcha.org>
parents: 29809
diff changeset
314
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
315 return w
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
316
29810
45fa8de47a0f py3: provide (del|get|has|set)attr wrappers that accepts bytes
Yuya Nishihara <yuya@tcha.org>
parents: 29809
diff changeset
317
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
318 # these wrappers are automagically imported by hgloader
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
319 delattr = _wrapattrfunc(builtins.delattr)
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
320 getattr = _wrapattrfunc(builtins.getattr)
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
321 hasattr = _wrapattrfunc(builtins.hasattr)
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
322 setattr = _wrapattrfunc(builtins.setattr)
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
323 xrange = builtins.range
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
324 unicode = str
31159
76a64c1e5439 py3: add pycompat.open and replace open() calls
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30820
diff changeset
325
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
326
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
327 def open(name, mode=b'r', buffering=-1, encoding=None):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
328 return builtins.open(name, sysstr(mode), buffering, encoding)
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
329
37102
6ca5f825a0ca util: make safehasattr() a pycompat function
Yuya Nishihara <yuya@tcha.org>
parents: 37067
diff changeset
330
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
331 safehasattr = _wrapattrfunc(builtins.hasattr)
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
332
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
333
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
334 def _getoptbwrapper(orig, args, shortlist, namelist):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
335 """
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
336 Takes bytes arguments, converts them to unicode, pass them to
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
337 getopt.getopt(), convert the returned values back to bytes and then
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
338 return them for Python 3 compatibility as getopt.getopt() don't accepts
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
339 bytes on Python 3.
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
340 """
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
341 args = [a.decode('latin-1') for a in args]
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
342 shortlist = shortlist.decode('latin-1')
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
343 namelist = [a.decode('latin-1') for a in namelist]
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
344 opts, args = orig(args, shortlist, namelist)
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
345 opts = [(a[0].encode('latin-1'), a[1].encode('latin-1')) for a in opts]
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
346 args = [a.encode('latin-1') for a in args]
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
347 return opts, args
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
348
30583
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30509
diff changeset
349
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
350 def strkwargs(dic):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
351 """
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
352 Converts the keys of a python dictonary to str i.e. unicodes so that
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
353 they can be passed as keyword arguments as dictionaries with bytes keys
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
354 can't be passed as keyword arguments to functions on Python 3.
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
355 """
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
356 dic = {k.decode('latin-1'): v for k, v in dic.items()}
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
357 return dic
30584
fbc3f73dc802 py3: utility functions to convert keys of kwargs to bytes/unicodes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30583
diff changeset
358
fbc3f73dc802 py3: utility functions to convert keys of kwargs to bytes/unicodes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30583
diff changeset
359
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
360 def byteskwargs(dic):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
361 """
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
362 Converts keys of python dictionaries to bytes as they were converted to
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
363 str to pass that dictonary as a keyword argument on Python 3.
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
364 """
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
365 dic = {k.encode('latin-1'): v for k, v in dic.items()}
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
366 return dic
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
367
30681
caf7e1c5efe4 py3: have a bytes version of shlex.split()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30671
diff changeset
368
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
369 # TODO: handle shlex.shlex().
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
370 def shlexsplit(s, comments=False, posix=True):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
371 """
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
372 Takes bytes argument, convert it to str i.e. unicodes, pass that into
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
373 shlex.split(), convert the returned value to bytes and return that for
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
374 Python 3 compatibility as shelx.split() don't accept bytes on Python 3.
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
375 """
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
376 ret = shlex.split(s.decode('latin-1'), comments, posix)
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
377 return [a.encode('latin-1') for a in ret]
43021
2cc453284d5c patchbomb: protect email addresses from shell
Floris Bruynooghe <flub@google.com>
parents: 40577
diff changeset
378
43380
579672b347d2 py3: define and use json.loads polyfill
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43117
diff changeset
379
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
380 iteritems = lambda x: x.items()
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
381 itervalues = lambda x: x.values()
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
382
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
383 # Python 3.5's json.load and json.loads require str. We polyfill its
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
384 # code for detecting encoding from bytes.
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
385 if sys.version_info[0:2] < (3, 6):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
386
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
387 def _detect_encoding(b):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
388 bstartswith = b.startswith
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
389 if bstartswith((codecs.BOM_UTF32_BE, codecs.BOM_UTF32_LE)):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
390 return 'utf-32'
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
391 if bstartswith((codecs.BOM_UTF16_BE, codecs.BOM_UTF16_LE)):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
392 return 'utf-16'
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
393 if bstartswith(codecs.BOM_UTF8):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
394 return 'utf-8-sig'
43380
579672b347d2 py3: define and use json.loads polyfill
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43117
diff changeset
395
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
396 if len(b) >= 4:
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
397 if not b[0]:
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
398 # 00 00 -- -- - utf-32-be
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
399 # 00 XX -- -- - utf-16-be
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
400 return 'utf-16-be' if b[1] else 'utf-32-be'
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
401 if not b[1]:
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
402 # XX 00 00 00 - utf-32-le
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
403 # XX 00 00 XX - utf-16-le
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
404 # XX 00 XX -- - utf-16-le
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
405 return 'utf-16-le' if b[2] or b[3] else 'utf-32-le'
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
406 elif len(b) == 2:
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
407 if not b[0]:
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
408 # 00 XX - utf-16-be
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
409 return 'utf-16-be'
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
410 if not b[1]:
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
411 # XX 00 - utf-16-le
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
412 return 'utf-16-le'
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
413 # default
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
414 return 'utf-8'
43380
579672b347d2 py3: define and use json.loads polyfill
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43117
diff changeset
415
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
416 def json_loads(s, *args, **kwargs):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
417 if isinstance(s, (bytes, bytearray)):
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
418 s = s.decode(_detect_encoding(s), 'surrogatepass')
43380
579672b347d2 py3: define and use json.loads polyfill
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43117
diff changeset
419
48962
79009cca491e pycompat: remove large Python 2 block
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48960
diff changeset
420 return json.loads(s, *args, **kwargs)
43380
579672b347d2 py3: define and use json.loads polyfill
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43117
diff changeset
421
579672b347d2 py3: define and use json.loads polyfill
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43117
diff changeset
422
30032
2219f4f82ede pycompat: extract function that converts attribute or encoding name to str
Yuya Nishihara <yuya@tcha.org>
parents: 30031
diff changeset
423 else:
43380
579672b347d2 py3: define and use json.loads polyfill
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43117
diff changeset
424 json_loads = json.loads
34639
a568a46751b6 selectors2: do not use platform.system()
Jun Wu <quark@fb.com>
parents: 34468
diff changeset
425
39658
a407f9009392 py3: byteify strings in pycompat
Matt Harbison <matt_harbison@yahoo.com>
parents: 39473
diff changeset
426 isjython = sysplatform.startswith(b'java')
34644
c0a6c19690ff pycompat: define operating system constants
Jun Wu <quark@fb.com>
parents: 34639
diff changeset
427
40577
1b49b84d5ed5 pycompat: adding Linux detection and fixing Mac
rdamazio@google.com
parents: 39823
diff changeset
428 isdarwin = sysplatform.startswith(b'darwin')
1b49b84d5ed5 pycompat: adding Linux detection and fixing Mac
rdamazio@google.com
parents: 39823
diff changeset
429 islinux = sysplatform.startswith(b'linux')
39658
a407f9009392 py3: byteify strings in pycompat
Matt Harbison <matt_harbison@yahoo.com>
parents: 39473
diff changeset
430 isposix = osname == b'posix'
a407f9009392 py3: byteify strings in pycompat
Matt Harbison <matt_harbison@yahoo.com>
parents: 39473
diff changeset
431 iswindows = osname == b'nt'
35235
5b569d512fbd fancyopts: use getopt.gnu_getopt()
Yuya Nishihara <yuya@tcha.org>
parents: 34644
diff changeset
432
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43021
diff changeset
433
35235
5b569d512fbd fancyopts: use getopt.gnu_getopt()
Yuya Nishihara <yuya@tcha.org>
parents: 34644
diff changeset
434 def getoptb(args, shortlist, namelist):
5b569d512fbd fancyopts: use getopt.gnu_getopt()
Yuya Nishihara <yuya@tcha.org>
parents: 34644
diff changeset
435 return _getoptbwrapper(getopt.getopt, args, shortlist, namelist)
5b569d512fbd fancyopts: use getopt.gnu_getopt()
Yuya Nishihara <yuya@tcha.org>
parents: 34644
diff changeset
436
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43021
diff changeset
437
35235
5b569d512fbd fancyopts: use getopt.gnu_getopt()
Yuya Nishihara <yuya@tcha.org>
parents: 34644
diff changeset
438 def gnugetoptb(args, shortlist, namelist):
5b569d512fbd fancyopts: use getopt.gnu_getopt()
Yuya Nishihara <yuya@tcha.org>
parents: 34644
diff changeset
439 return _getoptbwrapper(getopt.gnu_getopt, args, shortlist, namelist)
38197
aac4be30e250 py3: wrap tempfile.mkstemp() to use bytes path
Yuya Nishihara <yuya@tcha.org>
parents: 37890
diff changeset
440
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43021
diff changeset
441
38198
2ce60954b1b7 py3: wrap tempfile.mkdtemp() to use bytes path
Yuya Nishihara <yuya@tcha.org>
parents: 38197
diff changeset
442 def mkdtemp(suffix=b'', prefix=b'tmp', dir=None):
2ce60954b1b7 py3: wrap tempfile.mkdtemp() to use bytes path
Yuya Nishihara <yuya@tcha.org>
parents: 38197
diff changeset
443 return tempfile.mkdtemp(suffix, prefix, dir)
2ce60954b1b7 py3: wrap tempfile.mkdtemp() to use bytes path
Yuya Nishihara <yuya@tcha.org>
parents: 38197
diff changeset
444
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43021
diff changeset
445
38197
aac4be30e250 py3: wrap tempfile.mkstemp() to use bytes path
Yuya Nishihara <yuya@tcha.org>
parents: 37890
diff changeset
446 # text=True is not supported; use util.from/tonativeeol() instead
aac4be30e250 py3: wrap tempfile.mkstemp() to use bytes path
Yuya Nishihara <yuya@tcha.org>
parents: 37890
diff changeset
447 def mkstemp(suffix=b'', prefix=b'tmp', dir=None):
aac4be30e250 py3: wrap tempfile.mkstemp() to use bytes path
Yuya Nishihara <yuya@tcha.org>
parents: 37890
diff changeset
448 return tempfile.mkstemp(suffix, prefix, dir)
38199
cc9aa88792fe py3: wrap tempfile.NamedTemporaryFile() to return bytes fp.name
Yuya Nishihara <yuya@tcha.org>
parents: 38198
diff changeset
449
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43021
diff changeset
450
45727
d4c4391aa7f2 pycompat: add an entry for unnamedtmpfile
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45726
diff changeset
451 # TemporaryFile does not support an "encoding=" argument on python2.
d4c4391aa7f2 pycompat: add an entry for unnamedtmpfile
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45726
diff changeset
452 # This wrapper file are always open in byte mode.
d4c4391aa7f2 pycompat: add an entry for unnamedtmpfile
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45726
diff changeset
453 def unnamedtempfile(mode=None, *args, **kwargs):
d4c4391aa7f2 pycompat: add an entry for unnamedtmpfile
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45726
diff changeset
454 if mode is None:
46380
7a29d9002250 pycompat: fix a bytes vs str issue in `unnamedtempfile()`
Matt Harbison <matt_harbison@yahoo.com>
parents: 46178
diff changeset
455 mode = 'w+b'
45727
d4c4391aa7f2 pycompat: add an entry for unnamedtmpfile
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45726
diff changeset
456 else:
d4c4391aa7f2 pycompat: add an entry for unnamedtmpfile
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45726
diff changeset
457 mode = sysstr(mode)
d4c4391aa7f2 pycompat: add an entry for unnamedtmpfile
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45726
diff changeset
458 assert 'b' in mode
d4c4391aa7f2 pycompat: add an entry for unnamedtmpfile
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45726
diff changeset
459 return tempfile.TemporaryFile(mode, *args, **kwargs)
d4c4391aa7f2 pycompat: add an entry for unnamedtmpfile
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45726
diff changeset
460
d4c4391aa7f2 pycompat: add an entry for unnamedtmpfile
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45726
diff changeset
461
45726
d1072cba8aff pycompat: update comment about unnamedtempfile
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45057
diff changeset
462 # NamedTemporaryFile does not support an "encoding=" argument on python2.
d1072cba8aff pycompat: update comment about unnamedtempfile
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45057
diff changeset
463 # This wrapper file are always open in byte mode.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43021
diff changeset
464 def namedtempfile(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43021
diff changeset
465 mode=b'w+b', bufsize=-1, suffix=b'', prefix=b'tmp', dir=None, delete=True
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43021
diff changeset
466 ):
38199
cc9aa88792fe py3: wrap tempfile.NamedTemporaryFile() to return bytes fp.name
Yuya Nishihara <yuya@tcha.org>
parents: 38198
diff changeset
467 mode = sysstr(mode)
43554
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43551
diff changeset
468 assert 'b' in mode
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43021
diff changeset
469 return tempfile.NamedTemporaryFile(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43021
diff changeset
470 mode, bufsize, suffix=suffix, prefix=prefix, dir=dir, delete=delete
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43021
diff changeset
471 )