Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/pycompat.py @ 36216:646002338365

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
py3: introduce and use pycompat.getargspec This is getfullargspec on py3, which means we can't use namedtuple named accessors for all fields (eg keywords from getargspec is varkw from getfullargspec, with the same meaning). Solves some warning issues on Python 3. I didn't clean up httpclient because that's vendored code I think we should probably discard, and I didn't touch the manpage generator for now either. Differential Revision: https://phab.mercurial-scm.org/D2251
author Augie Fackler <augie@google.com>
date Tue, 13 Feb 2018 23:00:01 -0700
parents 04984f2e50ae
children b44fac3a49fb
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
28818
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
1 # pycompat.py - portability shim for python 3
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
2 #
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
3 # This software may be used and distributed according to the terms of the
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
4 # GNU General Public License version 2 or any later version.
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
5
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
6 """Mercurial portability shim for python 3.
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
7
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
8 This contains aliases to hide python version-specific details from the core.
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
9 """
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
10
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
11 from __future__ import absolute_import
6041fb8f2da8 pycompat: add empty and queue to handle py3 divergence
timeless <timeless@mozdev.org>
parents:
diff changeset
12
30583
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30509
diff changeset
13 import getopt
36216
646002338365 py3: introduce and use pycompat.getargspec
Augie Fackler <augie@google.com>
parents: 36083
diff changeset
14 import inspect
30315
3874ddba1ab4 py3: add a bytes version of os.name
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30313
diff changeset
15 import os
30681
caf7e1c5efe4 py3: have a bytes version of shlex.split()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30671
diff changeset
16 import shlex
29584
06587edd1233 pycompat: make pycompat demandimport friendly
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29566
diff changeset
17 import sys
06587edd1233 pycompat: make pycompat demandimport friendly
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29566
diff changeset
18
30031
0f6d6fdd3c2a pycompat: provide 'ispy3' constant
Yuya Nishihara <yuya@tcha.org>
parents: 29812
diff changeset
19 ispy3 = (sys.version_info[0] >= 3)
33624
524b13fc711f util: fix sortdict.update() to call __setitem__() on PyPy (issue5639)
Yuya Nishihara <yuya@tcha.org>
parents: 32883
diff changeset
20 ispypy = (r'__pypy__' in sys.builtin_module_names)
30031
0f6d6fdd3c2a pycompat: provide 'ispy3' constant
Yuya Nishihara <yuya@tcha.org>
parents: 29812
diff changeset
21
0f6d6fdd3c2a pycompat: provide 'ispy3' constant
Yuya Nishihara <yuya@tcha.org>
parents: 29812
diff changeset
22 if not ispy3:
31934
12aca6770046 util: make cookielib module available
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31843
diff changeset
23 import cookielib
29324
b501579147f1 py3: conditionalize cPickle import by adding in util
Pulkit Goyal <7895pulkit@gmail.com>
parents: 28882
diff changeset
24 import cPickle as pickle
29455
0c741fd6158a py3: conditionalize httplib import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29433
diff changeset
25 import httplib
29584
06587edd1233 pycompat: make pycompat demandimport friendly
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29566
diff changeset
26 import Queue as _queue
29433
33770d2b6cf9 py3: conditionalize SocketServer import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29432
diff changeset
27 import SocketServer as socketserver
29432
34b914ac573e py3: conditionalize xmlrpclib import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29431
diff changeset
28 import xmlrpclib
29584
06587edd1233 pycompat: make pycompat demandimport friendly
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29566
diff changeset
29 else:
31942
bc0579a25f82 pycompat: import correct cookie module on Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31934
diff changeset
30 import http.cookiejar as cookielib
29584
06587edd1233 pycompat: make pycompat demandimport friendly
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29566
diff changeset
31 import http.client as httplib
06587edd1233 pycompat: make pycompat demandimport friendly
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29566
diff changeset
32 import pickle
06587edd1233 pycompat: make pycompat demandimport friendly
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29566
diff changeset
33 import queue as _queue
06587edd1233 pycompat: make pycompat demandimport friendly
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29566
diff changeset
34 import socketserver
29432
34b914ac573e py3: conditionalize xmlrpclib import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29431
diff changeset
35 import xmlrpc.client as xmlrpclib
29431
80880ad3fccd py3: conditionalize the urlparse import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29414
diff changeset
36
32883
6e38b4212661 pycompat: move the queue related definitions below queue import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32882
diff changeset
37 empty = _queue.Empty
6e38b4212661 pycompat: move the queue related definitions below queue import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32882
diff changeset
38 queue = _queue.Queue
6e38b4212661 pycompat: move the queue related definitions below queue import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32882
diff changeset
39
31777
7d2cbe11ae48 pycompat: introduce identity function as a compat stub
Yuya Nishihara <yuya@tcha.org>
parents: 31579
diff changeset
40 def identity(a):
7d2cbe11ae48 pycompat: introduce identity function as a compat stub
Yuya Nishihara <yuya@tcha.org>
parents: 31579
diff changeset
41 return a
7d2cbe11ae48 pycompat: introduce identity function as a compat stub
Yuya Nishihara <yuya@tcha.org>
parents: 31579
diff changeset
42
30031
0f6d6fdd3c2a pycompat: provide 'ispy3' constant
Yuya Nishihara <yuya@tcha.org>
parents: 29812
diff changeset
43 if ispy3:
29808
965c91bad9e3 py3: move xrange alias next to import lines
Yuya Nishihara <yuya@tcha.org>
parents: 29790
diff changeset
44 import builtins
29810
45fa8de47a0f py3: provide (del|get|has|set)attr wrappers that accepts bytes
Yuya Nishihara <yuya@tcha.org>
parents: 29809
diff changeset
45 import functools
31381
06440ba06bc0 pycompat: move imports of cStringIO/io to where they are used
Yuya Nishihara <yuya@tcha.org>
parents: 31368
diff changeset
46 import io
31433
4acc49335a6e py3: optimize py3 compat.bytechr using Struct.pack
Martin von Zweigbergk <martinvonz@google.com>
parents: 31409
diff changeset
47 import struct
31381
06440ba06bc0 pycompat: move imports of cStringIO/io to where they are used
Yuya Nishihara <yuya@tcha.org>
parents: 31368
diff changeset
48
30119
f4a5e0e86a7e py3: add an os.fsencode backport to ease path handling
Martijn Pieters <mjpieters@fb.com>
parents: 30086
diff changeset
49 fsencode = os.fsencode
30313
42af0590f4b9 py3: add os.fsdecode() as pycompat.fsdecode()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30133
diff changeset
50 fsdecode = os.fsdecode
31778
8181f378b073 pycompat: provide bytes os.linesep
Yuya Nishihara <yuya@tcha.org>
parents: 31777
diff changeset
51 oslinesep = os.linesep.encode('ascii')
30315
3874ddba1ab4 py3: add a bytes version of os.name
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30313
diff changeset
52 osname = os.name.encode('ascii')
30316
ad40d307a9f0 py3: have pycompat.ospathsep and pycompat.ossep
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30315
diff changeset
53 ospathsep = os.pathsep.encode('ascii')
ad40d307a9f0 py3: have pycompat.ospathsep and pycompat.ossep
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30315
diff changeset
54 ossep = os.sep.encode('ascii')
30628
c6026c20a3ce py3: have a bytes version of os.altsep
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30584
diff changeset
55 osaltsep = os.altsep
c6026c20a3ce py3: have a bytes version of os.altsep
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30584
diff changeset
56 if osaltsep:
c6026c20a3ce py3: have a bytes version of os.altsep
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30584
diff changeset
57 osaltsep = osaltsep.encode('ascii')
30509
fc0cfe6c87d7 py3: add os.getcwdb() to have bytes path
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30481
diff changeset
58 # os.getcwd() on Python 3 returns string, but it has os.getcwdb() which
fc0cfe6c87d7 py3: add os.getcwdb() to have bytes path
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30481
diff changeset
59 # returns bytes.
fc0cfe6c87d7 py3: add os.getcwdb() to have bytes path
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30481
diff changeset
60 getcwd = os.getcwdb
30629
a82a6eee2613 py3: have a bytes version of sys.platform
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30628
diff changeset
61 sysplatform = sys.platform.encode('ascii')
30671
3fcaf0f660ce py3: have bytes version of sys.executable
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30666
diff changeset
62 sysexecutable = sys.executable
3fcaf0f660ce py3: have bytes version of sys.executable
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30666
diff changeset
63 if sysexecutable:
3fcaf0f660ce py3: have bytes version of sys.executable
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30666
diff changeset
64 sysexecutable = os.fsencode(sysexecutable)
31368
73b3bee8febe pycompat: default to BytesIO instead of StringIO
Pulkit Goyal <7895pulkit@gmail.com>
parents: 31310
diff changeset
65 stringio = io.BytesIO
31508
a1e40ceee640 pycompat: add maplist alias for old map behavior
Augie Fackler <augie@google.com>
parents: 31448
diff changeset
66 maplist = lambda *args: list(map(*args))
35412
e66d6e938d2d py3: introduce pycompat.ziplist as zip is a generator on Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents: 35235
diff changeset
67 ziplist = lambda *args: list(zip(*args))
33874
cfcfbe6c96f8 py3: select input or raw_input by pycompat
Yuya Nishihara <yuya@tcha.org>
parents: 33624
diff changeset
68 rawinput = input
36216
646002338365 py3: introduce and use pycompat.getargspec
Augie Fackler <augie@google.com>
parents: 36083
diff changeset
69 getargspec = inspect.getfullargspec
30344
19d8e19fde5b py3: document why os.fsencode() can be used to get back bytes argv
Yuya Nishihara <yuya@tcha.org>
parents: 30340
diff changeset
70
30481
277f4fe6d01a py3: provide bytes stdin/out/err through util module
Yuya Nishihara <yuya@tcha.org>
parents: 30344
diff changeset
71 # TODO: .buffer might not exist if std streams were replaced; we'll need
277f4fe6d01a py3: provide bytes stdin/out/err through util module
Yuya Nishihara <yuya@tcha.org>
parents: 30344
diff changeset
72 # a silly wrapper to make a bytes stream backed by a unicode one.
277f4fe6d01a py3: provide bytes stdin/out/err through util module
Yuya Nishihara <yuya@tcha.org>
parents: 30344
diff changeset
73 stdin = sys.stdin.buffer
277f4fe6d01a py3: provide bytes stdin/out/err through util module
Yuya Nishihara <yuya@tcha.org>
parents: 30344
diff changeset
74 stdout = sys.stdout.buffer
277f4fe6d01a py3: provide bytes stdin/out/err through util module
Yuya Nishihara <yuya@tcha.org>
parents: 30344
diff changeset
75 stderr = sys.stderr.buffer
277f4fe6d01a py3: provide bytes stdin/out/err through util module
Yuya Nishihara <yuya@tcha.org>
parents: 30344
diff changeset
76
30344
19d8e19fde5b py3: document why os.fsencode() can be used to get back bytes argv
Yuya Nishihara <yuya@tcha.org>
parents: 30340
diff changeset
77 # Since Python 3 converts argv to wchar_t type by Py_DecodeLocale() on Unix,
19d8e19fde5b py3: document why os.fsencode() can be used to get back bytes argv
Yuya Nishihara <yuya@tcha.org>
parents: 30340
diff changeset
78 # we can use os.fsencode() to get back bytes argv.
19d8e19fde5b py3: document why os.fsencode() can be used to get back bytes argv
Yuya Nishihara <yuya@tcha.org>
parents: 30340
diff changeset
79 #
19d8e19fde5b py3: document why os.fsencode() can be used to get back bytes argv
Yuya Nishihara <yuya@tcha.org>
parents: 30340
diff changeset
80 # https://hg.python.org/cpython/file/v3.5.1/Programs/python.c#l55
19d8e19fde5b py3: document why os.fsencode() can be used to get back bytes argv
Yuya Nishihara <yuya@tcha.org>
parents: 30340
diff changeset
81 #
19d8e19fde5b py3: document why os.fsencode() can be used to get back bytes argv
Yuya Nishihara <yuya@tcha.org>
parents: 30340
diff changeset
82 # TODO: On Windows, the native argv is wchar_t, so we'll need a different
19d8e19fde5b py3: document why os.fsencode() can be used to get back bytes argv
Yuya Nishihara <yuya@tcha.org>
parents: 30340
diff changeset
83 # workaround to simulate the Python 2 (i.e. ANSI Win32 API) behavior.
30888
86cd1f2cfff5 pycompat: verify sys.argv exists before forwarding it (issue5493)
Augie Fackler <augie@google.com>
parents: 30820
diff changeset
84 if getattr(sys, 'argv', None) is not None:
86cd1f2cfff5 pycompat: verify sys.argv exists before forwarding it (issue5493)
Augie Fackler <augie@google.com>
parents: 30820
diff changeset
85 sysargv = list(map(os.fsencode, sys.argv))
29808
965c91bad9e3 py3: move xrange alias next to import lines
Yuya Nishihara <yuya@tcha.org>
parents: 29790
diff changeset
86
31433
4acc49335a6e py3: optimize py3 compat.bytechr using Struct.pack
Martin von Zweigbergk <martinvonz@google.com>
parents: 31409
diff changeset
87 bytechr = struct.Struct('>B').pack
31263
64596338ba10 py3: factor out bytechr() function
Yuya Nishihara <yuya@tcha.org>
parents: 31159
diff changeset
88
31448
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
89 class bytestr(bytes):
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
90 """A bytes which mostly acts as a Python 2 str
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
91
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
92 >>> bytestr(), bytestr(bytearray(b'foo')), bytestr(u'ascii'), bytestr(1)
35943
1a31111e6239 py3: always drop b'' prefix from repr() of bytestr
Yuya Nishihara <yuya@tcha.org>
parents: 35412
diff changeset
93 ('', 'foo', 'ascii', '1')
31448
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
94 >>> s = bytestr(b'foo')
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
95 >>> assert s is bytestr(s)
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
96
32488
548478efc46c pycompat: try __bytes__() to convert object to bytestr
Yuya Nishihara <yuya@tcha.org>
parents: 32232
diff changeset
97 __bytes__() should be called if provided:
548478efc46c pycompat: try __bytes__() to convert object to bytestr
Yuya Nishihara <yuya@tcha.org>
parents: 32232
diff changeset
98
548478efc46c pycompat: try __bytes__() to convert object to bytestr
Yuya Nishihara <yuya@tcha.org>
parents: 32232
diff changeset
99 >>> class bytesable(object):
548478efc46c pycompat: try __bytes__() to convert object to bytestr
Yuya Nishihara <yuya@tcha.org>
parents: 32232
diff changeset
100 ... def __bytes__(self):
548478efc46c pycompat: try __bytes__() to convert object to bytestr
Yuya Nishihara <yuya@tcha.org>
parents: 32232
diff changeset
101 ... return b'bytes'
548478efc46c pycompat: try __bytes__() to convert object to bytestr
Yuya Nishihara <yuya@tcha.org>
parents: 32232
diff changeset
102 >>> bytestr(bytesable())
35943
1a31111e6239 py3: always drop b'' prefix from repr() of bytestr
Yuya Nishihara <yuya@tcha.org>
parents: 35412
diff changeset
103 'bytes'
32488
548478efc46c pycompat: try __bytes__() to convert object to bytestr
Yuya Nishihara <yuya@tcha.org>
parents: 32232
diff changeset
104
31448
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
105 There's no implicit conversion from non-ascii str as its encoding is
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
106 unknown:
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
107
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
108 >>> bytestr(chr(0x80)) # doctest: +ELLIPSIS
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
109 Traceback (most recent call last):
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
110 ...
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
111 UnicodeEncodeError: ...
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
112
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
113 Comparison between bytestr and bytes should work:
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
114
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
115 >>> assert bytestr(b'foo') == b'foo'
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
116 >>> assert b'foo' == bytestr(b'foo')
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
117 >>> assert b'f' in bytestr(b'foo')
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
118 >>> assert bytestr(b'f') in b'foo'
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
119
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
120 Sliced elements should be bytes, not integer:
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
121
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
122 >>> s[1], s[:2]
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
123 (b'o', b'fo')
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
124 >>> list(s), list(reversed(s))
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
125 ([b'f', b'o', b'o'], [b'o', b'o', b'f'])
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
126
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
127 As bytestr type isn't propagated across operations, you need to cast
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
128 bytes to bytestr explicitly:
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
129
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
130 >>> s = bytestr(b'foo').upper()
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
131 >>> t = bytestr(s)
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
132 >>> s[0], t[0]
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
133 (70, b'F')
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
134
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
135 Be careful to not pass a bytestr object to a function which expects
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
136 bytearray-like behavior.
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
137
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
138 >>> t = bytes(t) # cast to bytes
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
139 >>> assert type(t) is bytes
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
140 """
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
141
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
142 def __new__(cls, s=b''):
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
143 if isinstance(s, bytestr):
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
144 return s
32488
548478efc46c pycompat: try __bytes__() to convert object to bytestr
Yuya Nishihara <yuya@tcha.org>
parents: 32232
diff changeset
145 if (not isinstance(s, (bytes, bytearray))
548478efc46c pycompat: try __bytes__() to convert object to bytestr
Yuya Nishihara <yuya@tcha.org>
parents: 32232
diff changeset
146 and not hasattr(s, u'__bytes__')): # hasattr-py3-only
31448
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
147 s = str(s).encode(u'ascii')
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
148 return bytes.__new__(cls, s)
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
149
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
150 def __getitem__(self, key):
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
151 s = bytes.__getitem__(self, key)
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
152 if not isinstance(s, bytes):
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
153 s = bytechr(s)
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
154 return s
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
155
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
156 def __iter__(self):
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
157 return iterbytestr(bytes.__iter__(self))
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
158
35943
1a31111e6239 py3: always drop b'' prefix from repr() of bytestr
Yuya Nishihara <yuya@tcha.org>
parents: 35412
diff changeset
159 def __repr__(self):
1a31111e6239 py3: always drop b'' prefix from repr() of bytestr
Yuya Nishihara <yuya@tcha.org>
parents: 35412
diff changeset
160 return bytes.__repr__(self)[1:] # drop b''
1a31111e6239 py3: always drop b'' prefix from repr() of bytestr
Yuya Nishihara <yuya@tcha.org>
parents: 35412
diff changeset
161
31391
c9fd842dc886 pycompat: add helper to iterate each char in bytes
Yuya Nishihara <yuya@tcha.org>
parents: 31381
diff changeset
162 def iterbytestr(s):
c9fd842dc886 pycompat: add helper to iterate each char in bytes
Yuya Nishihara <yuya@tcha.org>
parents: 31381
diff changeset
163 """Iterate bytes as if it were a str object of Python 2"""
31434
63a39d647888 py3: make py3 compat.iterbytestr simpler and faster
Martin von Zweigbergk <martinvonz@google.com>
parents: 31433
diff changeset
164 return map(bytechr, s)
31391
c9fd842dc886 pycompat: add helper to iterate each char in bytes
Yuya Nishihara <yuya@tcha.org>
parents: 31381
diff changeset
165
35944
fc44c2657dc5 py3: drop b'' from repr() of smartset
Yuya Nishihara <yuya@tcha.org>
parents: 35943
diff changeset
166 def maybebytestr(s):
fc44c2657dc5 py3: drop b'' from repr() of smartset
Yuya Nishihara <yuya@tcha.org>
parents: 35943
diff changeset
167 """Promote bytes to bytestr"""
fc44c2657dc5 py3: drop b'' from repr() of smartset
Yuya Nishihara <yuya@tcha.org>
parents: 35943
diff changeset
168 if isinstance(s, bytes):
fc44c2657dc5 py3: drop b'' from repr() of smartset
Yuya Nishihara <yuya@tcha.org>
parents: 35943
diff changeset
169 return bytestr(s)
fc44c2657dc5 py3: drop b'' from repr() of smartset
Yuya Nishihara <yuya@tcha.org>
parents: 35943
diff changeset
170 return s
fc44c2657dc5 py3: drop b'' from repr() of smartset
Yuya Nishihara <yuya@tcha.org>
parents: 35943
diff changeset
171
31820
45761ef1bc93 py3: have registrar process docstrings in bytes
Yuya Nishihara <yuya@tcha.org>
parents: 31778
diff changeset
172 def sysbytes(s):
45761ef1bc93 py3: have registrar process docstrings in bytes
Yuya Nishihara <yuya@tcha.org>
parents: 31778
diff changeset
173 """Convert an internal str (e.g. keyword, __doc__) back to bytes
45761ef1bc93 py3: have registrar process docstrings in bytes
Yuya Nishihara <yuya@tcha.org>
parents: 31778
diff changeset
174
45761ef1bc93 py3: have registrar process docstrings in bytes
Yuya Nishihara <yuya@tcha.org>
parents: 31778
diff changeset
175 This never raises UnicodeEncodeError, but only ASCII characters
45761ef1bc93 py3: have registrar process docstrings in bytes
Yuya Nishihara <yuya@tcha.org>
parents: 31778
diff changeset
176 can be round-trip by sysstr(sysbytes(s)).
45761ef1bc93 py3: have registrar process docstrings in bytes
Yuya Nishihara <yuya@tcha.org>
parents: 31778
diff changeset
177 """
45761ef1bc93 py3: have registrar process docstrings in bytes
Yuya Nishihara <yuya@tcha.org>
parents: 31778
diff changeset
178 return s.encode(u'utf-8')
45761ef1bc93 py3: have registrar process docstrings in bytes
Yuya Nishihara <yuya@tcha.org>
parents: 31778
diff changeset
179
30032
2219f4f82ede pycompat: extract function that converts attribute or encoding name to str
Yuya Nishihara <yuya@tcha.org>
parents: 30031
diff changeset
180 def sysstr(s):
2219f4f82ede pycompat: extract function that converts attribute or encoding name to str
Yuya Nishihara <yuya@tcha.org>
parents: 30031
diff changeset
181 """Return a keyword str to be passed to Python functions such as
2219f4f82ede pycompat: extract function that converts attribute or encoding name to str
Yuya Nishihara <yuya@tcha.org>
parents: 30031
diff changeset
182 getattr() and str.encode()
2219f4f82ede pycompat: extract function that converts attribute or encoding name to str
Yuya Nishihara <yuya@tcha.org>
parents: 30031
diff changeset
183
2219f4f82ede pycompat: extract function that converts attribute or encoding name to str
Yuya Nishihara <yuya@tcha.org>
parents: 30031
diff changeset
184 This never raises UnicodeDecodeError. Non-ascii characters are
2219f4f82ede pycompat: extract function that converts attribute or encoding name to str
Yuya Nishihara <yuya@tcha.org>
parents: 30031
diff changeset
185 considered invalid and mapped to arbitrary but unique code points
2219f4f82ede pycompat: extract function that converts attribute or encoding name to str
Yuya Nishihara <yuya@tcha.org>
parents: 30031
diff changeset
186 such that 'sysstr(a) != sysstr(b)' for all 'a != b'.
2219f4f82ede pycompat: extract function that converts attribute or encoding name to str
Yuya Nishihara <yuya@tcha.org>
parents: 30031
diff changeset
187 """
2219f4f82ede pycompat: extract function that converts attribute or encoding name to str
Yuya Nishihara <yuya@tcha.org>
parents: 30031
diff changeset
188 if isinstance(s, builtins.str):
2219f4f82ede pycompat: extract function that converts attribute or encoding name to str
Yuya Nishihara <yuya@tcha.org>
parents: 30031
diff changeset
189 return s
2219f4f82ede pycompat: extract function that converts attribute or encoding name to str
Yuya Nishihara <yuya@tcha.org>
parents: 30031
diff changeset
190 return s.decode(u'latin-1')
2219f4f82ede pycompat: extract function that converts attribute or encoding name to str
Yuya Nishihara <yuya@tcha.org>
parents: 30031
diff changeset
191
32877
a05f3675c46a py3: add a new strurl() which will convert a bytes url to str
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32638
diff changeset
192 def strurl(url):
a05f3675c46a py3: add a new strurl() which will convert a bytes url to str
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32638
diff changeset
193 """Converts a bytes url back to str"""
a05f3675c46a py3: add a new strurl() which will convert a bytes url to str
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32638
diff changeset
194 return url.decode(u'ascii')
a05f3675c46a py3: add a new strurl() which will convert a bytes url to str
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32638
diff changeset
195
32878
f22f39d56bb5 py3: add a new bytesurl() to convert a str url into bytes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32877
diff changeset
196 def bytesurl(url):
f22f39d56bb5 py3: add a new bytesurl() to convert a str url into bytes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32877
diff changeset
197 """Converts a str url to bytes by encoding in ascii"""
f22f39d56bb5 py3: add a new bytesurl() to convert a str url into bytes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32877
diff changeset
198 return url.encode(u'ascii')
f22f39d56bb5 py3: add a new bytesurl() to convert a str url into bytes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32877
diff changeset
199
32232
76f9a0009b4b pycompat: extract helper to raise exception with traceback
Yuya Nishihara <yuya@tcha.org>
parents: 31942
diff changeset
200 def raisewithtb(exc, tb):
76f9a0009b4b pycompat: extract helper to raise exception with traceback
Yuya Nishihara <yuya@tcha.org>
parents: 31942
diff changeset
201 """Raise exception with the given traceback"""
76f9a0009b4b pycompat: extract helper to raise exception with traceback
Yuya Nishihara <yuya@tcha.org>
parents: 31942
diff changeset
202 raise exc.with_traceback(tb)
76f9a0009b4b pycompat: extract helper to raise exception with traceback
Yuya Nishihara <yuya@tcha.org>
parents: 31942
diff changeset
203
32638
c9318beb7c1a py3: convert __doc__ back to bytes in help.py
Yuya Nishihara <yuya@tcha.org>
parents: 32488
diff changeset
204 def getdoc(obj):
c9318beb7c1a py3: convert __doc__ back to bytes in help.py
Yuya Nishihara <yuya@tcha.org>
parents: 32488
diff changeset
205 """Get docstring as bytes; may be None so gettext() won't confuse it
c9318beb7c1a py3: convert __doc__ back to bytes in help.py
Yuya Nishihara <yuya@tcha.org>
parents: 32488
diff changeset
206 with _('')"""
c9318beb7c1a py3: convert __doc__ back to bytes in help.py
Yuya Nishihara <yuya@tcha.org>
parents: 32488
diff changeset
207 doc = getattr(obj, u'__doc__', None)
c9318beb7c1a py3: convert __doc__ back to bytes in help.py
Yuya Nishihara <yuya@tcha.org>
parents: 32488
diff changeset
208 if doc is None:
c9318beb7c1a py3: convert __doc__ back to bytes in help.py
Yuya Nishihara <yuya@tcha.org>
parents: 32488
diff changeset
209 return doc
c9318beb7c1a py3: convert __doc__ back to bytes in help.py
Yuya Nishihara <yuya@tcha.org>
parents: 32488
diff changeset
210 return sysbytes(doc)
c9318beb7c1a py3: convert __doc__ back to bytes in help.py
Yuya Nishihara <yuya@tcha.org>
parents: 32488
diff changeset
211
29810
45fa8de47a0f py3: provide (del|get|has|set)attr wrappers that accepts bytes
Yuya Nishihara <yuya@tcha.org>
parents: 29809
diff changeset
212 def _wrapattrfunc(f):
45fa8de47a0f py3: provide (del|get|has|set)attr wrappers that accepts bytes
Yuya Nishihara <yuya@tcha.org>
parents: 29809
diff changeset
213 @functools.wraps(f)
45fa8de47a0f py3: provide (del|get|has|set)attr wrappers that accepts bytes
Yuya Nishihara <yuya@tcha.org>
parents: 29809
diff changeset
214 def w(object, name, *args):
30032
2219f4f82ede pycompat: extract function that converts attribute or encoding name to str
Yuya Nishihara <yuya@tcha.org>
parents: 30031
diff changeset
215 return f(object, sysstr(name), *args)
29810
45fa8de47a0f py3: provide (del|get|has|set)attr wrappers that accepts bytes
Yuya Nishihara <yuya@tcha.org>
parents: 29809
diff changeset
216 return w
45fa8de47a0f py3: provide (del|get|has|set)attr wrappers that accepts bytes
Yuya Nishihara <yuya@tcha.org>
parents: 29809
diff changeset
217
29811
178c89e8519a py3: import builtin wrappers automagically by code transformer
Yuya Nishihara <yuya@tcha.org>
parents: 29810
diff changeset
218 # these wrappers are automagically imported by hgloader
29810
45fa8de47a0f py3: provide (del|get|has|set)attr wrappers that accepts bytes
Yuya Nishihara <yuya@tcha.org>
parents: 29809
diff changeset
219 delattr = _wrapattrfunc(builtins.delattr)
45fa8de47a0f py3: provide (del|get|has|set)attr wrappers that accepts bytes
Yuya Nishihara <yuya@tcha.org>
parents: 29809
diff changeset
220 getattr = _wrapattrfunc(builtins.getattr)
45fa8de47a0f py3: provide (del|get|has|set)attr wrappers that accepts bytes
Yuya Nishihara <yuya@tcha.org>
parents: 29809
diff changeset
221 hasattr = _wrapattrfunc(builtins.hasattr)
45fa8de47a0f py3: provide (del|get|has|set)attr wrappers that accepts bytes
Yuya Nishihara <yuya@tcha.org>
parents: 29809
diff changeset
222 setattr = _wrapattrfunc(builtins.setattr)
29811
178c89e8519a py3: import builtin wrappers automagically by code transformer
Yuya Nishihara <yuya@tcha.org>
parents: 29810
diff changeset
223 xrange = builtins.range
31843
526e4597cca5 py3: add pycompat.unicode and add it to importer
Pulkit Goyal <7895pulkit@gmail.com>
parents: 31842
diff changeset
224 unicode = str
29810
45fa8de47a0f py3: provide (del|get|has|set)attr wrappers that accepts bytes
Yuya Nishihara <yuya@tcha.org>
parents: 29809
diff changeset
225
31159
76a64c1e5439 py3: add pycompat.open and replace open() calls
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30820
diff changeset
226 def open(name, mode='r', buffering=-1):
76a64c1e5439 py3: add pycompat.open and replace open() calls
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30820
diff changeset
227 return builtins.open(name, sysstr(mode), buffering)
76a64c1e5439 py3: add pycompat.open and replace open() calls
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30820
diff changeset
228
35235
5b569d512fbd fancyopts: use getopt.gnu_getopt()
Yuya Nishihara <yuya@tcha.org>
parents: 34644
diff changeset
229 def _getoptbwrapper(orig, args, shortlist, namelist):
32882
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
230 """
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
231 Takes bytes arguments, converts them to unicode, pass them to
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
232 getopt.getopt(), convert the returned values back to bytes and then
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
233 return them for Python 3 compatibility as getopt.getopt() don't accepts
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
234 bytes on Python 3.
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
235 """
30583
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30509
diff changeset
236 args = [a.decode('latin-1') for a in args]
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30509
diff changeset
237 shortlist = shortlist.decode('latin-1')
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30509
diff changeset
238 namelist = [a.decode('latin-1') for a in namelist]
35235
5b569d512fbd fancyopts: use getopt.gnu_getopt()
Yuya Nishihara <yuya@tcha.org>
parents: 34644
diff changeset
239 opts, args = orig(args, shortlist, namelist)
30583
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30509
diff changeset
240 opts = [(a[0].encode('latin-1'), a[1].encode('latin-1'))
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30509
diff changeset
241 for a in opts]
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30509
diff changeset
242 args = [a.encode('latin-1') for a in args]
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30509
diff changeset
243 return opts, args
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30509
diff changeset
244
30584
fbc3f73dc802 py3: utility functions to convert keys of kwargs to bytes/unicodes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30583
diff changeset
245 def strkwargs(dic):
32882
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
246 """
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
247 Converts the keys of a python dictonary to str i.e. unicodes so that
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
248 they can be passed as keyword arguments as dictonaries with bytes keys
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
249 can't be passed as keyword arguments to functions on Python 3.
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
250 """
30584
fbc3f73dc802 py3: utility functions to convert keys of kwargs to bytes/unicodes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30583
diff changeset
251 dic = dict((k.decode('latin-1'), v) for k, v in dic.iteritems())
fbc3f73dc802 py3: utility functions to convert keys of kwargs to bytes/unicodes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30583
diff changeset
252 return dic
fbc3f73dc802 py3: utility functions to convert keys of kwargs to bytes/unicodes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30583
diff changeset
253
fbc3f73dc802 py3: utility functions to convert keys of kwargs to bytes/unicodes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30583
diff changeset
254 def byteskwargs(dic):
32882
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
255 """
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
256 Converts keys of python dictonaries to bytes as they were converted to
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
257 str to pass that dictonary as a keyword argument on Python 3.
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
258 """
30584
fbc3f73dc802 py3: utility functions to convert keys of kwargs to bytes/unicodes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30583
diff changeset
259 dic = dict((k.encode('latin-1'), v) for k, v in dic.iteritems())
fbc3f73dc802 py3: utility functions to convert keys of kwargs to bytes/unicodes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30583
diff changeset
260 return dic
fbc3f73dc802 py3: utility functions to convert keys of kwargs to bytes/unicodes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30583
diff changeset
261
30681
caf7e1c5efe4 py3: have a bytes version of shlex.split()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30671
diff changeset
262 # TODO: handle shlex.shlex().
caf7e1c5efe4 py3: have a bytes version of shlex.split()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30671
diff changeset
263 def shlexsplit(s):
32882
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
264 """
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
265 Takes bytes argument, convert it to str i.e. unicodes, pass that into
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
266 shlex.split(), convert the returned value to bytes and return that for
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
267 Python 3 compatibility as shelx.split() don't accept bytes on Python 3.
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
268 """
30681
caf7e1c5efe4 py3: have a bytes version of shlex.split()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30671
diff changeset
269 ret = shlex.split(s.decode('latin-1'))
caf7e1c5efe4 py3: have a bytes version of shlex.split()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30671
diff changeset
270 return [a.encode('latin-1') for a in ret]
caf7e1c5efe4 py3: have a bytes version of shlex.split()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30671
diff changeset
271
36083
04984f2e50ae py3: use email parser that operates on bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35944
diff changeset
272 def emailparser(*args, **kwargs):
04984f2e50ae py3: use email parser that operates on bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35944
diff changeset
273 import email.parser
04984f2e50ae py3: use email parser that operates on bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35944
diff changeset
274 return email.parser.BytesParser(*args, **kwargs)
04984f2e50ae py3: use email parser that operates on bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35944
diff changeset
275
30032
2219f4f82ede pycompat: extract function that converts attribute or encoding name to str
Yuya Nishihara <yuya@tcha.org>
parents: 30031
diff changeset
276 else:
31381
06440ba06bc0 pycompat: move imports of cStringIO/io to where they are used
Yuya Nishihara <yuya@tcha.org>
parents: 31368
diff changeset
277 import cStringIO
06440ba06bc0 pycompat: move imports of cStringIO/io to where they are used
Yuya Nishihara <yuya@tcha.org>
parents: 31368
diff changeset
278
31263
64596338ba10 py3: factor out bytechr() function
Yuya Nishihara <yuya@tcha.org>
parents: 31159
diff changeset
279 bytechr = chr
31448
b70407bd84d5 pycompat: add bytestr wrapper which mostly acts as a Python 2 str
Yuya Nishihara <yuya@tcha.org>
parents: 31434
diff changeset
280 bytestr = str
31391
c9fd842dc886 pycompat: add helper to iterate each char in bytes
Yuya Nishihara <yuya@tcha.org>
parents: 31381
diff changeset
281 iterbytestr = iter
35944
fc44c2657dc5 py3: drop b'' from repr() of smartset
Yuya Nishihara <yuya@tcha.org>
parents: 35943
diff changeset
282 maybebytestr = identity
31820
45761ef1bc93 py3: have registrar process docstrings in bytes
Yuya Nishihara <yuya@tcha.org>
parents: 31778
diff changeset
283 sysbytes = identity
31777
7d2cbe11ae48 pycompat: introduce identity function as a compat stub
Yuya Nishihara <yuya@tcha.org>
parents: 31579
diff changeset
284 sysstr = identity
32877
a05f3675c46a py3: add a new strurl() which will convert a bytes url to str
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32638
diff changeset
285 strurl = identity
32878
f22f39d56bb5 py3: add a new bytesurl() to convert a str url into bytes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32877
diff changeset
286 bytesurl = identity
30032
2219f4f82ede pycompat: extract function that converts attribute or encoding name to str
Yuya Nishihara <yuya@tcha.org>
parents: 30031
diff changeset
287
32232
76f9a0009b4b pycompat: extract helper to raise exception with traceback
Yuya Nishihara <yuya@tcha.org>
parents: 31942
diff changeset
288 # this can't be parsed on Python 3
76f9a0009b4b pycompat: extract helper to raise exception with traceback
Yuya Nishihara <yuya@tcha.org>
parents: 31942
diff changeset
289 exec('def raisewithtb(exc, tb):\n'
76f9a0009b4b pycompat: extract helper to raise exception with traceback
Yuya Nishihara <yuya@tcha.org>
parents: 31942
diff changeset
290 ' raise exc, None, tb\n')
76f9a0009b4b pycompat: extract helper to raise exception with traceback
Yuya Nishihara <yuya@tcha.org>
parents: 31942
diff changeset
291
30133
f6dcda7505f9 pycompat: only accept a bytestring filepath in Python 2
Martijn Pieters <mjpieters@fb.com>
parents: 30119
diff changeset
292 def fsencode(filename):
32882
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
293 """
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
294 Partial backport from os.py in Python 3, which only accepts bytes.
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
295 In Python 2, our paths should only ever be bytes, a unicode path
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
296 indicates a bug.
f57f1f37290d pycompat: move multiline comments above a function to function doc
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32878
diff changeset
297 """
30133
f6dcda7505f9 pycompat: only accept a bytestring filepath in Python 2
Martijn Pieters <mjpieters@fb.com>
parents: 30119
diff changeset
298 if isinstance(filename, str):
f6dcda7505f9 pycompat: only accept a bytestring filepath in Python 2
Martijn Pieters <mjpieters@fb.com>
parents: 30119
diff changeset
299 return filename
30119
f4a5e0e86a7e py3: add an os.fsencode backport to ease path handling
Martijn Pieters <mjpieters@fb.com>
parents: 30086
diff changeset
300 else:
30133
f6dcda7505f9 pycompat: only accept a bytestring filepath in Python 2
Martijn Pieters <mjpieters@fb.com>
parents: 30119
diff changeset
301 raise TypeError(
f6dcda7505f9 pycompat: only accept a bytestring filepath in Python 2
Martijn Pieters <mjpieters@fb.com>
parents: 30119
diff changeset
302 "expect str, not %s" % type(filename).__name__)
30119
f4a5e0e86a7e py3: add an os.fsencode backport to ease path handling
Martijn Pieters <mjpieters@fb.com>
parents: 30086
diff changeset
303
30313
42af0590f4b9 py3: add os.fsdecode() as pycompat.fsdecode()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30133
diff changeset
304 # In Python 2, fsdecode() has a very chance to receive bytes. So it's
42af0590f4b9 py3: add os.fsdecode() as pycompat.fsdecode()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30133
diff changeset
305 # better not to touch Python 2 part as it's already working fine.
31777
7d2cbe11ae48 pycompat: introduce identity function as a compat stub
Yuya Nishihara <yuya@tcha.org>
parents: 31579
diff changeset
306 fsdecode = identity
30313
42af0590f4b9 py3: add os.fsdecode() as pycompat.fsdecode()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30133
diff changeset
307
32638
c9318beb7c1a py3: convert __doc__ back to bytes in help.py
Yuya Nishihara <yuya@tcha.org>
parents: 32488
diff changeset
308 def getdoc(obj):
c9318beb7c1a py3: convert __doc__ back to bytes in help.py
Yuya Nishihara <yuya@tcha.org>
parents: 32488
diff changeset
309 return getattr(obj, '__doc__', None)
c9318beb7c1a py3: convert __doc__ back to bytes in help.py
Yuya Nishihara <yuya@tcha.org>
parents: 32488
diff changeset
310
35235
5b569d512fbd fancyopts: use getopt.gnu_getopt()
Yuya Nishihara <yuya@tcha.org>
parents: 34644
diff changeset
311 def _getoptbwrapper(orig, args, shortlist, namelist):
5b569d512fbd fancyopts: use getopt.gnu_getopt()
Yuya Nishihara <yuya@tcha.org>
parents: 34644
diff changeset
312 return orig(args, shortlist, namelist)
30583
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30509
diff changeset
313
31777
7d2cbe11ae48 pycompat: introduce identity function as a compat stub
Yuya Nishihara <yuya@tcha.org>
parents: 31579
diff changeset
314 strkwargs = identity
7d2cbe11ae48 pycompat: introduce identity function as a compat stub
Yuya Nishihara <yuya@tcha.org>
parents: 31579
diff changeset
315 byteskwargs = identity
30584
fbc3f73dc802 py3: utility functions to convert keys of kwargs to bytes/unicodes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30583
diff changeset
316
31778
8181f378b073 pycompat: provide bytes os.linesep
Yuya Nishihara <yuya@tcha.org>
parents: 31777
diff changeset
317 oslinesep = os.linesep
30315
3874ddba1ab4 py3: add a bytes version of os.name
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30313
diff changeset
318 osname = os.name
30316
ad40d307a9f0 py3: have pycompat.ospathsep and pycompat.ossep
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30315
diff changeset
319 ospathsep = os.pathsep
ad40d307a9f0 py3: have pycompat.ospathsep and pycompat.ossep
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30315
diff changeset
320 ossep = os.sep
30628
c6026c20a3ce py3: have a bytes version of os.altsep
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30584
diff changeset
321 osaltsep = os.altsep
30481
277f4fe6d01a py3: provide bytes stdin/out/err through util module
Yuya Nishihara <yuya@tcha.org>
parents: 30344
diff changeset
322 stdin = sys.stdin
277f4fe6d01a py3: provide bytes stdin/out/err through util module
Yuya Nishihara <yuya@tcha.org>
parents: 30344
diff changeset
323 stdout = sys.stdout
277f4fe6d01a py3: provide bytes stdin/out/err through util module
Yuya Nishihara <yuya@tcha.org>
parents: 30344
diff changeset
324 stderr = sys.stderr
30888
86cd1f2cfff5 pycompat: verify sys.argv exists before forwarding it (issue5493)
Augie Fackler <augie@google.com>
parents: 30820
diff changeset
325 if getattr(sys, 'argv', None) is not None:
86cd1f2cfff5 pycompat: verify sys.argv exists before forwarding it (issue5493)
Augie Fackler <augie@google.com>
parents: 30820
diff changeset
326 sysargv = sys.argv
30629
a82a6eee2613 py3: have a bytes version of sys.platform
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30628
diff changeset
327 sysplatform = sys.platform
30509
fc0cfe6c87d7 py3: add os.getcwdb() to have bytes path
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30481
diff changeset
328 getcwd = os.getcwd
30671
3fcaf0f660ce py3: have bytes version of sys.executable
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30666
diff changeset
329 sysexecutable = sys.executable
30681
caf7e1c5efe4 py3: have a bytes version of shlex.split()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30671
diff changeset
330 shlexsplit = shlex.split
31381
06440ba06bc0 pycompat: move imports of cStringIO/io to where they are used
Yuya Nishihara <yuya@tcha.org>
parents: 31368
diff changeset
331 stringio = cStringIO.StringIO
31508
a1e40ceee640 pycompat: add maplist alias for old map behavior
Augie Fackler <augie@google.com>
parents: 31448
diff changeset
332 maplist = map
35412
e66d6e938d2d py3: introduce pycompat.ziplist as zip is a generator on Python 3
Pulkit Goyal <7895pulkit@gmail.com>
parents: 35235
diff changeset
333 ziplist = zip
33874
cfcfbe6c96f8 py3: select input or raw_input by pycompat
Yuya Nishihara <yuya@tcha.org>
parents: 33624
diff changeset
334 rawinput = raw_input
36216
646002338365 py3: introduce and use pycompat.getargspec
Augie Fackler <augie@google.com>
parents: 36083
diff changeset
335 getargspec = inspect.getargspec
34639
a568a46751b6 selectors2: do not use platform.system()
Jun Wu <quark@fb.com>
parents: 34468
diff changeset
336
36083
04984f2e50ae py3: use email parser that operates on bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35944
diff changeset
337 def emailparser(*args, **kwargs):
04984f2e50ae py3: use email parser that operates on bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35944
diff changeset
338 import email.parser
04984f2e50ae py3: use email parser that operates on bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35944
diff changeset
339 return email.parser.Parser(*args, **kwargs)
04984f2e50ae py3: use email parser that operates on bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35944
diff changeset
340
34639
a568a46751b6 selectors2: do not use platform.system()
Jun Wu <quark@fb.com>
parents: 34468
diff changeset
341 isjython = sysplatform.startswith('java')
34644
c0a6c19690ff pycompat: define operating system constants
Jun Wu <quark@fb.com>
parents: 34639
diff changeset
342
c0a6c19690ff pycompat: define operating system constants
Jun Wu <quark@fb.com>
parents: 34639
diff changeset
343 isdarwin = sysplatform == 'darwin'
c0a6c19690ff pycompat: define operating system constants
Jun Wu <quark@fb.com>
parents: 34639
diff changeset
344 isposix = osname == 'posix'
c0a6c19690ff pycompat: define operating system constants
Jun Wu <quark@fb.com>
parents: 34639
diff changeset
345 iswindows = osname == 'nt'
35235
5b569d512fbd fancyopts: use getopt.gnu_getopt()
Yuya Nishihara <yuya@tcha.org>
parents: 34644
diff changeset
346
5b569d512fbd fancyopts: use getopt.gnu_getopt()
Yuya Nishihara <yuya@tcha.org>
parents: 34644
diff changeset
347 def getoptb(args, shortlist, namelist):
5b569d512fbd fancyopts: use getopt.gnu_getopt()
Yuya Nishihara <yuya@tcha.org>
parents: 34644
diff changeset
348 return _getoptbwrapper(getopt.getopt, args, shortlist, namelist)
5b569d512fbd fancyopts: use getopt.gnu_getopt()
Yuya Nishihara <yuya@tcha.org>
parents: 34644
diff changeset
349
5b569d512fbd fancyopts: use getopt.gnu_getopt()
Yuya Nishihara <yuya@tcha.org>
parents: 34644
diff changeset
350 def gnugetoptb(args, shortlist, namelist):
5b569d512fbd fancyopts: use getopt.gnu_getopt()
Yuya Nishihara <yuya@tcha.org>
parents: 34644
diff changeset
351 return _getoptbwrapper(getopt.gnu_getopt, args, shortlist, namelist)