Mercurial > public > mercurial-scm > hg
annotate mercurial/pure/osutil.py @ 26117:4dc5b51f38fe
revlog: change generaldelta delta parent heuristic
The old generaldelta heuristic was "if p1 (or p2) was closer than the last full text,
use it, otherwise use prev". This was problematic when a repo contained multiple
branches that were very different. If commits to branch A were pushed, and the
last full text was branch B, it would generate a fulltext. Then if branch B was
pushed, it would generate another fulltext. The problem is that the last
fulltext (and delta'ing against `prev` in general) has no correlation with the
contents of the incoming revision, and therefore will always have degenerate
cases.
According to the blame, that algorithm was chosen to minimize the chain length.
Since there is already code that protects against that (the delta-vs-fulltext
code), and since it has been improved since the original generaldelta algorithm
went in (2011), I believe the chain length criteria will still be preserved.
The new algorithm always diffs against p1 (or p2 if it's closer), unless the
resulting delta will fail the delta-vs-fulltext check, in which case we delta
against prev.
Some before and after stats on manifest.d size.
internal large repo
old heuristic - 2.0 GB
new heuristic - 1.2 GB
mozilla-central
old heuristic - 242 MB
new heuristic - 261 MB
The regression in mozilla central is due to the new heuristic choosing p2r as
the delta when it's closer to the tip. Switching the algorithm to always prefer
p1r brings the size back down (242 MB). This is result of the way in which
mozilla does merges and pushes, and the result could easily swing the other
direction in other repos (depending on if they merge X into Y or Y into X), but
will never be as degenerate as before.
I future patch will address the regression by introducing an optional, even more
aggressive delta heuristic which will knock the mozilla manifest size down
dramatically.
author | Durham Goode <durham@fb.com> |
---|---|
date | Sun, 30 Aug 2015 13:58:11 -0700 |
parents | 977102cb12fc |
children | 810337ae1b76 |
rev | line source |
---|---|
8232
823f25b25dea
pure/osutil: add copyright and license header
Martin Geisler <mg@lazybytes.net>
parents:
7704
diff
changeset
|
1 # osutil.py - pure Python version of osutil.c |
823f25b25dea
pure/osutil: add copyright and license header
Martin Geisler <mg@lazybytes.net>
parents:
7704
diff
changeset
|
2 # |
823f25b25dea
pure/osutil: add copyright and license header
Martin Geisler <mg@lazybytes.net>
parents:
7704
diff
changeset
|
3 # Copyright 2009 Matt Mackall <mpm@selenic.com> and others |
823f25b25dea
pure/osutil: add copyright and license header
Martin Geisler <mg@lazybytes.net>
parents:
7704
diff
changeset
|
4 # |
823f25b25dea
pure/osutil: add copyright and license header
Martin Geisler <mg@lazybytes.net>
parents:
7704
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
10263 | 6 # GNU General Public License version 2 or any later version. |
8232
823f25b25dea
pure/osutil: add copyright and license header
Martin Geisler <mg@lazybytes.net>
parents:
7704
diff
changeset
|
7 |
7704
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
8 import os |
10651
5f091fc1bab7
style: use consistent variable names (*mod) with imports which would shadow
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
10282
diff
changeset
|
9 import stat as statmod |
7704
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
10 |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
11 def _mode_to_kind(mode): |
10651
5f091fc1bab7
style: use consistent variable names (*mod) with imports which would shadow
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
10282
diff
changeset
|
12 if statmod.S_ISREG(mode): |
5f091fc1bab7
style: use consistent variable names (*mod) with imports which would shadow
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
10282
diff
changeset
|
13 return statmod.S_IFREG |
5f091fc1bab7
style: use consistent variable names (*mod) with imports which would shadow
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
10282
diff
changeset
|
14 if statmod.S_ISDIR(mode): |
5f091fc1bab7
style: use consistent variable names (*mod) with imports which would shadow
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
10282
diff
changeset
|
15 return statmod.S_IFDIR |
5f091fc1bab7
style: use consistent variable names (*mod) with imports which would shadow
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
10282
diff
changeset
|
16 if statmod.S_ISLNK(mode): |
5f091fc1bab7
style: use consistent variable names (*mod) with imports which would shadow
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
10282
diff
changeset
|
17 return statmod.S_IFLNK |
5f091fc1bab7
style: use consistent variable names (*mod) with imports which would shadow
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
10282
diff
changeset
|
18 if statmod.S_ISBLK(mode): |
5f091fc1bab7
style: use consistent variable names (*mod) with imports which would shadow
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
10282
diff
changeset
|
19 return statmod.S_IFBLK |
5f091fc1bab7
style: use consistent variable names (*mod) with imports which would shadow
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
10282
diff
changeset
|
20 if statmod.S_ISCHR(mode): |
5f091fc1bab7
style: use consistent variable names (*mod) with imports which would shadow
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
10282
diff
changeset
|
21 return statmod.S_IFCHR |
5f091fc1bab7
style: use consistent variable names (*mod) with imports which would shadow
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
10282
diff
changeset
|
22 if statmod.S_ISFIFO(mode): |
5f091fc1bab7
style: use consistent variable names (*mod) with imports which would shadow
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
10282
diff
changeset
|
23 return statmod.S_IFIFO |
5f091fc1bab7
style: use consistent variable names (*mod) with imports which would shadow
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
10282
diff
changeset
|
24 if statmod.S_ISSOCK(mode): |
5f091fc1bab7
style: use consistent variable names (*mod) with imports which would shadow
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
10282
diff
changeset
|
25 return statmod.S_IFSOCK |
7704
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
26 return mode |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
27 |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
28 def listdir(path, stat=False, skip=None): |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
29 '''listdir(path, stat=False) -> list_of_tuples |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
30 |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
31 Return a sorted list containing information about the entries |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
32 in the directory. |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
33 |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
34 If stat is True, each element is a 3-tuple: |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
35 |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
36 (name, type, stat object) |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
37 |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
38 Otherwise, each element is a 2-tuple: |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
39 |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
40 (name, type) |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
41 ''' |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
42 result = [] |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
43 prefix = path |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
44 if not prefix.endswith(os.sep): |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
45 prefix += os.sep |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
46 names = os.listdir(path) |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
47 names.sort() |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
48 for fn in names: |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
49 st = os.lstat(prefix + fn) |
10651
5f091fc1bab7
style: use consistent variable names (*mod) with imports which would shadow
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents:
10282
diff
changeset
|
50 if fn == skip and statmod.S_ISDIR(st.st_mode): |
7704
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
51 return [] |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
52 if stat: |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
53 result.append((fn, _mode_to_kind(st.st_mode), st)) |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
54 else: |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
55 result.append((fn, _mode_to_kind(st.st_mode))) |
30d1d313370b
move mercurial.osutil to mercurial.pure.osutil
Martin Geisler <mg@daimi.au.dk>
parents:
diff
changeset
|
56 return result |
8421
b6d0fa8c7685
posixfile: remove posixfile_nt and fix import bug in windows.py
Sune Foldager <cryo@cyanite.org>
parents:
8232
diff
changeset
|
57 |
14413
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
58 if os.name != 'nt': |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
59 posixfile = open |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
60 else: |
16474
ee553e6cd8c4
pure/osutil: use Python's msvcrt module (issue3380)
Adrian Buehlmann <adrian@cadifra.com>
parents:
15040
diff
changeset
|
61 import ctypes, msvcrt |
14413
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
62 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
63 _kernel32 = ctypes.windll.kernel32 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
64 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
65 _DWORD = ctypes.c_ulong |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
66 _LPCSTR = _LPSTR = ctypes.c_char_p |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
67 _HANDLE = ctypes.c_void_p |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
68 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
69 _INVALID_HANDLE_VALUE = _HANDLE(-1).value |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
70 |
18959
2f6418d8a4c9
check-code: catch trailing space in comments
Mads Kiilerich <madski@unity3d.com>
parents:
17429
diff
changeset
|
71 # CreateFile |
14413
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
72 _FILE_SHARE_READ = 0x00000001 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
73 _FILE_SHARE_WRITE = 0x00000002 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
74 _FILE_SHARE_DELETE = 0x00000004 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
75 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
76 _CREATE_ALWAYS = 2 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
77 _OPEN_EXISTING = 3 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
78 _OPEN_ALWAYS = 4 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
79 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
80 _GENERIC_READ = 0x80000000 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
81 _GENERIC_WRITE = 0x40000000 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
82 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
83 _FILE_ATTRIBUTE_NORMAL = 0x80 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
84 |
17429
72fa4ef2245f
declare local constants instead of using magic values and comments
Mads Kiilerich <mads@kiilerich.com>
parents:
16686
diff
changeset
|
85 # open_osfhandle flags |
14413
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
86 _O_RDONLY = 0x0000 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
87 _O_RDWR = 0x0002 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
88 _O_APPEND = 0x0008 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
89 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
90 _O_TEXT = 0x4000 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
91 _O_BINARY = 0x8000 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
92 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
93 # types of parameters of C functions used (required by pypy) |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
94 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
95 _kernel32.CreateFileA.argtypes = [_LPCSTR, _DWORD, _DWORD, ctypes.c_void_p, |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
96 _DWORD, _DWORD, _HANDLE] |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
97 _kernel32.CreateFileA.restype = _HANDLE |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
98 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
99 def _raiseioerror(name): |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
100 err = ctypes.WinError() |
25645
977102cb12fc
osutil: remove Python 2.4 errno conversion workaround
Gregory Szorc <gregory.szorc@gmail.com>
parents:
19465
diff
changeset
|
101 raise IOError(err.errno, '%s: %s' % (name, err.strerror)) |
14413
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
102 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
103 class posixfile(object): |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
104 '''a file object aiming for POSIX-like semantics |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
105 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
106 CPython's open() returns a file that was opened *without* setting the |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
107 _FILE_SHARE_DELETE flag, which causes rename and unlink to abort. |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
108 This even happens if any hardlinked copy of the file is in open state. |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
109 We set _FILE_SHARE_DELETE here, so files opened with posixfile can be |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
110 renamed and deleted while they are held open. |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
111 Note that if a file opened with posixfile is unlinked, the file |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
112 remains but cannot be opened again or be recreated under the same name, |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
113 until all reading processes have closed the file.''' |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
114 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
115 def __init__(self, name, mode='r', bufsize=-1): |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
116 if 'b' in mode: |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
117 flags = _O_BINARY |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
118 else: |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
119 flags = _O_TEXT |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
120 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
121 m0 = mode[0] |
16686
67964cda8701
cleanup: "not x in y" -> "x not in y"
Brodie Rao <brodie@sf.io>
parents:
16474
diff
changeset
|
122 if m0 == 'r' and '+' not in mode: |
14413
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
123 flags |= _O_RDONLY |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
124 access = _GENERIC_READ |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
125 else: |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
126 # work around http://support.microsoft.com/kb/899149 and |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
127 # set _O_RDWR for 'w' and 'a', even if mode has no '+' |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
128 flags |= _O_RDWR |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
129 access = _GENERIC_READ | _GENERIC_WRITE |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
130 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
131 if m0 == 'r': |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
132 creation = _OPEN_EXISTING |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
133 elif m0 == 'w': |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
134 creation = _CREATE_ALWAYS |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
135 elif m0 == 'a': |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
136 creation = _OPEN_ALWAYS |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
137 flags |= _O_APPEND |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
138 else: |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
139 raise ValueError("invalid mode: %s" % mode) |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
140 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
141 fh = _kernel32.CreateFileA(name, access, |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
142 _FILE_SHARE_READ | _FILE_SHARE_WRITE | _FILE_SHARE_DELETE, |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
143 None, creation, _FILE_ATTRIBUTE_NORMAL, None) |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
144 if fh == _INVALID_HANDLE_VALUE: |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
145 _raiseioerror(name) |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
146 |
16474
ee553e6cd8c4
pure/osutil: use Python's msvcrt module (issue3380)
Adrian Buehlmann <adrian@cadifra.com>
parents:
15040
diff
changeset
|
147 fd = msvcrt.open_osfhandle(fh, flags) |
14413
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
148 if fd == -1: |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
149 _kernel32.CloseHandle(fh) |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
150 _raiseioerror(name) |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
151 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
152 f = os.fdopen(fd, mode, bufsize) |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
153 # unfortunately, f.name is '<fdopen>' at this point -- so we store |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
154 # the name on this wrapper. We cannot just assign to f.name, |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
155 # because that attribute is read-only. |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
156 object.__setattr__(self, 'name', name) |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
157 object.__setattr__(self, '_file', f) |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
158 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
159 def __iter__(self): |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
160 return self._file |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
161 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
162 def __getattr__(self, name): |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
163 return getattr(self._file, name) |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
164 |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
165 def __setattr__(self, name, value): |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
166 '''mimics the read-only attributes of Python file objects |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
167 by raising 'TypeError: readonly attribute' if someone tries: |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
168 f = posixfile('foo.txt') |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
169 f.name = 'bla' ''' |
5ef18e28df19
pure: provide more correct implementation of posixfile for Windows
Adrian Buehlmann <adrian@cadifra.com>
parents:
10651
diff
changeset
|
170 return self._file.__setattr__(name, value) |