Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/util.py @ 1258:1945754e466b

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add file encoding/decoding support
author mpm@selenic.com
date Thu, 15 Sep 2005 02:59:16 -0500
parents 3b4f05ff3130
children fc3b41570082
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1082
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
1 """
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
2 util.py - Mercurial utility functions and platform specfic implementations
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
3
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
4 Copyright 2005 K. Thananchayan <thananck@yahoo.com>
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
5
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
6 This software may be used and distributed according to the terms
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
7 of the GNU General Public License, incorporated herein by reference.
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
8
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
9 This contains helper routines that are independent of the SCM core and hide
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
10 platform-specific details from the core.
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
11 """
419
28511fc21073 [PATCH] file seperator handling for the other 'OS'
mpm@selenic.com
parents:
diff changeset
12
704
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
13 import os, errno
724
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
14 from demandload import *
1258
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
15 demandload(globals(), "re cStringIO shutil popen2 threading")
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
16
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
17 def filter(s, cmd):
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
18 "filter a string through a command that transforms its input to its output"
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
19 (pout, pin) = popen2.popen2(cmd, -1, 'b')
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
20 def writer():
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
21 pin.write(s)
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
22 pin.close()
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
23
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
24 # we should use select instead on UNIX, but this will work on most
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
25 # systems, including Windows
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
26 w = threading.Thread(target=writer)
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
27 w.start()
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
28 f = pout.read()
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
29 pout.close()
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
30 w.join()
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
31 return f
419
28511fc21073 [PATCH] file seperator handling for the other 'OS'
mpm@selenic.com
parents:
diff changeset
32
1015
22571b8d35d3 Add automatic binary file detection to diff and export
mpm@selenic.com
parents: 917
diff changeset
33 def binary(s):
1082
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
34 """return true if a string is binary data using diff's heuristic"""
1015
22571b8d35d3 Add automatic binary file detection to diff and export
mpm@selenic.com
parents: 917
diff changeset
35 if s and '\0' in s[:4096]:
22571b8d35d3 Add automatic binary file detection to diff and export
mpm@selenic.com
parents: 917
diff changeset
36 return True
22571b8d35d3 Add automatic binary file detection to diff and export
mpm@selenic.com
parents: 917
diff changeset
37 return False
22571b8d35d3 Add automatic binary file detection to diff and export
mpm@selenic.com
parents: 917
diff changeset
38
556
f6c6fa15ff70 Move dirstate.uniq to util.unique
mpm@selenic.com
parents: 521
diff changeset
39 def unique(g):
1082
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
40 """return the uniq elements of iterable g"""
556
f6c6fa15ff70 Move dirstate.uniq to util.unique
mpm@selenic.com
parents: 521
diff changeset
41 seen = {}
f6c6fa15ff70 Move dirstate.uniq to util.unique
mpm@selenic.com
parents: 521
diff changeset
42 for f in g:
f6c6fa15ff70 Move dirstate.uniq to util.unique
mpm@selenic.com
parents: 521
diff changeset
43 if f not in seen:
f6c6fa15ff70 Move dirstate.uniq to util.unique
mpm@selenic.com
parents: 521
diff changeset
44 seen[f] = 1
f6c6fa15ff70 Move dirstate.uniq to util.unique
mpm@selenic.com
parents: 521
diff changeset
45 yield f
f6c6fa15ff70 Move dirstate.uniq to util.unique
mpm@selenic.com
parents: 521
diff changeset
46
870
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
47 class Abort(Exception):
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
48 """Raised if a command needs to print an error and exit."""
508
42a660abaf75 [PATCH] Harden os.system
mpm@selenic.com
parents: 464
diff changeset
49
724
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
50 def always(fn): return True
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
51 def never(fn): return False
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
52
1062
6d5a62a549fa pep-0008 cleanup
benoit.boissinot@ens-lyon.fr
parents: 1031
diff changeset
53 def globre(pat, head='^', tail='$'):
724
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
54 "convert a glob pattern into a regexp"
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
55 i, n = 0, len(pat)
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
56 res = ''
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
57 group = False
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
58 def peek(): return i < n and pat[i]
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
59 while i < n:
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
60 c = pat[i]
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
61 i = i+1
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
62 if c == '*':
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
63 if peek() == '*':
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
64 i += 1
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
65 res += '.*'
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
66 else:
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
67 res += '[^/]*'
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
68 elif c == '?':
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
69 res += '.'
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
70 elif c == '[':
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
71 j = i
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
72 if j < n and pat[j] in '!]':
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
73 j += 1
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
74 while j < n and pat[j] != ']':
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
75 j += 1
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
76 if j >= n:
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
77 res += '\\['
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
78 else:
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
79 stuff = pat[i:j].replace('\\','\\\\')
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
80 i = j + 1
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
81 if stuff[0] == '!':
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
82 stuff = '^' + stuff[1:]
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
83 elif stuff[0] == '^':
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
84 stuff = '\\' + stuff
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
85 res = '%s[%s]' % (res, stuff)
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
86 elif c == '{':
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
87 group = True
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
88 res += '(?:'
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
89 elif c == '}' and group:
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
90 res += ')'
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
91 group = False
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
92 elif c == ',' and group:
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
93 res += '|'
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
94 else:
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
95 res += re.escape(c)
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
96 return head + res + tail
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
97
812
b65af904d6d7 Reduce the amount of stat traffic generated by a walk.
Bryan O'Sullivan <bos@serpentine.com>
parents: 782
diff changeset
98 _globchars = {'[': 1, '{': 1, '*': 1, '?': 1}
b65af904d6d7 Reduce the amount of stat traffic generated by a walk.
Bryan O'Sullivan <bos@serpentine.com>
parents: 782
diff changeset
99
884
087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types.
Bryan O'Sullivan <bos@serpentine.com>
parents: 878
diff changeset
100 def pathto(n1, n2):
886
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
101 '''return the relative path from one place to another.
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
102 this returns a path in the form used by the local filesystem, not hg.'''
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
103 if not n1: return localpath(n2)
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
104 a, b = n1.split('/'), n2.split('/')
884
087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types.
Bryan O'Sullivan <bos@serpentine.com>
parents: 878
diff changeset
105 a.reverse(), b.reverse()
087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types.
Bryan O'Sullivan <bos@serpentine.com>
parents: 878
diff changeset
106 while a and b and a[-1] == b[-1]:
087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types.
Bryan O'Sullivan <bos@serpentine.com>
parents: 878
diff changeset
107 a.pop(), b.pop()
087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types.
Bryan O'Sullivan <bos@serpentine.com>
parents: 878
diff changeset
108 b.reverse()
087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types.
Bryan O'Sullivan <bos@serpentine.com>
parents: 878
diff changeset
109 return os.sep.join((['..'] * len(a)) + b)
087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types.
Bryan O'Sullivan <bos@serpentine.com>
parents: 878
diff changeset
110
1081
8b7d63489db3 Change canonpath to not know about repo objects
mpm@selenic.com
parents: 1075
diff changeset
111 def canonpath(root, cwd, myname):
1082
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
112 """return the canonical path of myname, given cwd and root"""
1081
8b7d63489db3 Change canonpath to not know about repo objects
mpm@selenic.com
parents: 1075
diff changeset
113 rootsep = root + os.sep
870
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
114 name = myname
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
115 if not name.startswith(os.sep):
1081
8b7d63489db3 Change canonpath to not know about repo objects
mpm@selenic.com
parents: 1075
diff changeset
116 name = os.path.join(root, cwd, name)
870
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
117 name = os.path.normpath(name)
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
118 if name.startswith(rootsep):
886
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
119 return pconvert(name[len(rootsep):])
1081
8b7d63489db3 Change canonpath to not know about repo objects
mpm@selenic.com
parents: 1075
diff changeset
120 elif name == root:
870
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
121 return ''
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
122 else:
1081
8b7d63489db3 Change canonpath to not know about repo objects
mpm@selenic.com
parents: 1075
diff changeset
123 raise Abort('%s not under root' % myname)
897
fe30f5434b51 Fix bug with empty inc and exc
mpm@selenic.com
parents: 896
diff changeset
124
1081
8b7d63489db3 Change canonpath to not know about repo objects
mpm@selenic.com
parents: 1075
diff changeset
125 def matcher(canonroot, cwd, names, inc, exc, head=''):
1082
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
126 """build a function to match a set of file patterns
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
127
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
128 arguments:
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
129 canonroot - the canonical root of the tree you're matching against
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
130 cwd - the current working directory, if relevant
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
131 names - patterns to find
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
132 inc - patterns to include
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
133 exc - patterns to exclude
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
134 head - a regex to prepend to patterns to control whether a match is rooted
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
135
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
136 a pattern is one of:
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
137 're:<regex>'
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
138 'glob:<shellglob>'
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
139 'path:<explicit path>'
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
140 'relpath:<relative path>'
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
141 '<relative path>'
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
142
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
143 returns:
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
144 a 3-tuple containing
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
145 - list of explicit non-pattern names passed in
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
146 - a bool match(filename) function
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
147 - a bool indicating if any patterns were passed in
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
148
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
149 todo:
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
150 make head regex a rooted bool
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
151 """
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
152
820
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
153 def patkind(name):
888
e7a943e8c52b Fix up handling of regexp paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 886
diff changeset
154 for prefix in 're:', 'glob:', 'path:', 'relpath:':
820
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
155 if name.startswith(prefix): return name.split(':', 1)
812
b65af904d6d7 Reduce the amount of stat traffic generated by a walk.
Bryan O'Sullivan <bos@serpentine.com>
parents: 782
diff changeset
156 for c in name:
820
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
157 if c in _globchars: return 'glob', name
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
158 return 'relpath', name
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
159
888
e7a943e8c52b Fix up handling of regexp paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 886
diff changeset
160 def regex(kind, name, tail):
742
092937de2ad7 Refactor matchpats and walk
mpm@selenic.com
parents: 740
diff changeset
161 '''convert a pattern into a regular expression'''
820
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
162 if kind == 're':
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
163 return name
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
164 elif kind == 'path':
888
e7a943e8c52b Fix up handling of regexp paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 886
diff changeset
165 return '^' + re.escape(name) + '(?:/|$)'
e7a943e8c52b Fix up handling of regexp paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 886
diff changeset
166 elif kind == 'relpath':
e7a943e8c52b Fix up handling of regexp paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 886
diff changeset
167 return head + re.escape(name) + tail
742
092937de2ad7 Refactor matchpats and walk
mpm@selenic.com
parents: 740
diff changeset
168 return head + globre(name, '', tail)
092937de2ad7 Refactor matchpats and walk
mpm@selenic.com
parents: 740
diff changeset
169
092937de2ad7 Refactor matchpats and walk
mpm@selenic.com
parents: 740
diff changeset
170 def matchfn(pats, tail):
092937de2ad7 Refactor matchpats and walk
mpm@selenic.com
parents: 740
diff changeset
171 """build a matching function from a set of patterns"""
092937de2ad7 Refactor matchpats and walk
mpm@selenic.com
parents: 740
diff changeset
172 if pats:
888
e7a943e8c52b Fix up handling of regexp paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 886
diff changeset
173 pat = '(?:%s)' % '|'.join([regex(k, p, tail) for (k, p) in pats])
742
092937de2ad7 Refactor matchpats and walk
mpm@selenic.com
parents: 740
diff changeset
174 return re.compile(pat).match
092937de2ad7 Refactor matchpats and walk
mpm@selenic.com
parents: 740
diff changeset
175
820
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
176 def globprefix(pat):
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
177 '''return the non-glob prefix of a path, e.g. foo/* -> foo'''
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
178 root = []
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
179 for p in pat.split(os.sep):
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
180 if patkind(p)[0] == 'glob': break
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
181 root.append(p)
886
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
182 return '/'.join(root)
820
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
183
870
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
184 pats = []
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
185 files = []
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
186 roots = []
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
187 for kind, name in map(patkind, names):
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
188 if kind in ('glob', 'relpath'):
1081
8b7d63489db3 Change canonpath to not know about repo objects
mpm@selenic.com
parents: 1075
diff changeset
189 name = canonpath(canonroot, cwd, name)
870
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
190 if name == '':
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
191 kind, name = 'glob', '**'
888
e7a943e8c52b Fix up handling of regexp paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 886
diff changeset
192 if kind in ('glob', 'path', 're'):
e7a943e8c52b Fix up handling of regexp paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 886
diff changeset
193 pats.append((kind, name))
870
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
194 if kind == 'glob':
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
195 root = globprefix(name)
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
196 if root: roots.append(root)
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
197 elif kind == 'relpath':
888
e7a943e8c52b Fix up handling of regexp paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 886
diff changeset
198 files.append((kind, name))
870
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
199 roots.append(name)
897
fe30f5434b51 Fix bug with empty inc and exc
mpm@selenic.com
parents: 896
diff changeset
200
820
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
201 patmatch = matchfn(pats, '$') or always
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
202 filematch = matchfn(files, '(?:/|$)') or always
897
fe30f5434b51 Fix bug with empty inc and exc
mpm@selenic.com
parents: 896
diff changeset
203 incmatch = always
fe30f5434b51 Fix bug with empty inc and exc
mpm@selenic.com
parents: 896
diff changeset
204 if inc:
fe30f5434b51 Fix bug with empty inc and exc
mpm@selenic.com
parents: 896
diff changeset
205 incmatch = matchfn(map(patkind, inc), '(?:/|$)')
fe30f5434b51 Fix bug with empty inc and exc
mpm@selenic.com
parents: 896
diff changeset
206 excmatch = lambda fn: False
fe30f5434b51 Fix bug with empty inc and exc
mpm@selenic.com
parents: 896
diff changeset
207 if exc:
fe30f5434b51 Fix bug with empty inc and exc
mpm@selenic.com
parents: 896
diff changeset
208 excmatch = matchfn(map(patkind, exc), '(?:/|$)')
742
092937de2ad7 Refactor matchpats and walk
mpm@selenic.com
parents: 740
diff changeset
209
1031
503aaf19a040 Rewrite log command. New version is faster and more featureful.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1015
diff changeset
210 return (roots,
503aaf19a040 Rewrite log command. New version is faster and more featureful.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1015
diff changeset
211 lambda fn: (incmatch(fn) and not excmatch(fn) and
503aaf19a040 Rewrite log command. New version is faster and more featureful.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1015
diff changeset
212 (fn.endswith('/') or
503aaf19a040 Rewrite log command. New version is faster and more featureful.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1015
diff changeset
213 (not pats and not files) or
503aaf19a040 Rewrite log command. New version is faster and more featureful.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1015
diff changeset
214 (pats and patmatch(fn)) or
503aaf19a040 Rewrite log command. New version is faster and more featureful.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1015
diff changeset
215 (files and filematch(fn)))),
503aaf19a040 Rewrite log command. New version is faster and more featureful.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1015
diff changeset
216 (inc or exc or (pats and pats != [('glob', '**')])) and True)
742
092937de2ad7 Refactor matchpats and walk
mpm@selenic.com
parents: 740
diff changeset
217
521
0fb8ade0f756 [PATCH] Fix use of util.CommandError
mpm@selenic.com
parents: 515
diff changeset
218 def system(cmd, errprefix=None):
508
42a660abaf75 [PATCH] Harden os.system
mpm@selenic.com
parents: 464
diff changeset
219 """execute a shell command that must succeed"""
42a660abaf75 [PATCH] Harden os.system
mpm@selenic.com
parents: 464
diff changeset
220 rc = os.system(cmd)
42a660abaf75 [PATCH] Harden os.system
mpm@selenic.com
parents: 464
diff changeset
221 if rc:
521
0fb8ade0f756 [PATCH] Fix use of util.CommandError
mpm@selenic.com
parents: 515
diff changeset
222 errmsg = "%s %s" % (os.path.basename(cmd.split(None, 1)[0]),
0fb8ade0f756 [PATCH] Fix use of util.CommandError
mpm@selenic.com
parents: 515
diff changeset
223 explain_exit(rc)[0])
0fb8ade0f756 [PATCH] Fix use of util.CommandError
mpm@selenic.com
parents: 515
diff changeset
224 if errprefix:
0fb8ade0f756 [PATCH] Fix use of util.CommandError
mpm@selenic.com
parents: 515
diff changeset
225 errmsg = "%s: %s" % (errprefix, errmsg)
870
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
226 raise Abort(errmsg)
508
42a660abaf75 [PATCH] Harden os.system
mpm@selenic.com
parents: 464
diff changeset
227
421
43b8da7420a9 [PATCH] rename under the other OS
mpm@selenic.com
parents: 419
diff changeset
228 def rename(src, dst):
1082
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
229 """forcibly rename a file"""
421
43b8da7420a9 [PATCH] rename under the other OS
mpm@selenic.com
parents: 419
diff changeset
230 try:
43b8da7420a9 [PATCH] rename under the other OS
mpm@selenic.com
parents: 419
diff changeset
231 os.rename(src, dst)
43b8da7420a9 [PATCH] rename under the other OS
mpm@selenic.com
parents: 419
diff changeset
232 except:
43b8da7420a9 [PATCH] rename under the other OS
mpm@selenic.com
parents: 419
diff changeset
233 os.unlink(dst)
43b8da7420a9 [PATCH] rename under the other OS
mpm@selenic.com
parents: 419
diff changeset
234 os.rename(src, dst)
43b8da7420a9 [PATCH] rename under the other OS
mpm@selenic.com
parents: 419
diff changeset
235
1241
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
236 def copyfiles(src, dst, hardlink=None):
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
237 """Copy a directory tree using hardlinks if possible"""
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
238
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
239 if hardlink is None:
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
240 hardlink = (os.stat(src).st_dev ==
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
241 os.stat(os.path.dirname(dst)).st_dev)
698
df78d8ccac4c Use python function instead of external 'cp' command when cloning repos.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 667
diff changeset
242
1207
a7b8812973d9 Rewrite copytree as copyfiles
mpm@selenic.com
parents: 1200
diff changeset
243 if os.path.isdir(src):
a7b8812973d9 Rewrite copytree as copyfiles
mpm@selenic.com
parents: 1200
diff changeset
244 os.mkdir(dst)
a7b8812973d9 Rewrite copytree as copyfiles
mpm@selenic.com
parents: 1200
diff changeset
245 for name in os.listdir(src):
a7b8812973d9 Rewrite copytree as copyfiles
mpm@selenic.com
parents: 1200
diff changeset
246 srcname = os.path.join(src, name)
a7b8812973d9 Rewrite copytree as copyfiles
mpm@selenic.com
parents: 1200
diff changeset
247 dstname = os.path.join(dst, name)
1241
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
248 copyfiles(srcname, dstname, hardlink)
1207
a7b8812973d9 Rewrite copytree as copyfiles
mpm@selenic.com
parents: 1200
diff changeset
249 else:
1241
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
250 if hardlink:
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
251 try:
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
252 os_link(src, dst)
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
253 except:
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
254 hardlink = False
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
255 shutil.copy2(src, dst)
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
256 else:
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
257 shutil.copy2(src, dst)
698
df78d8ccac4c Use python function instead of external 'cp' command when cloning repos.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 667
diff changeset
258
1090
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
259 def opener(base):
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
260 """
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
261 return a function that opens files relative to base
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
262
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
263 this function is used to hide the details of COW semantics and
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
264 remote file access from higher level code.
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
265 """
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
266 p = base
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
267 def o(path, mode="r"):
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
268 f = os.path.join(p, path)
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
269
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
270 mode += "b" # for that other OS
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
271
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
272 if mode[0] != "r":
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
273 try:
1241
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
274 nlink = nlinks(f)
1090
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
275 except OSError:
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
276 d = os.path.dirname(f)
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
277 if not os.path.isdir(d):
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
278 os.makedirs(d)
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
279 else:
1241
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
280 if nlink > 1:
1090
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
281 file(f + ".tmp", "wb").write(file(f, "rb").read())
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
282 rename(f+".tmp", f)
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
283
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
284 return file(f, mode)
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
285
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
286 return o
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
287
704
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
288 def _makelock_file(info, pathname):
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
289 ld = os.open(pathname, os.O_CREAT | os.O_WRONLY | os.O_EXCL)
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
290 os.write(ld, info)
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
291 os.close(ld)
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
292
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
293 def _readlock_file(pathname):
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
294 return file(pathname).read()
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
295
1241
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
296 def nlinks(pathname):
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
297 """Return number of hardlinks for the given file."""
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
298 return os.stat(pathname).st_nlink
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
299
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
300 if hasattr(os, 'link'):
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
301 os_link = os.link
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
302 else:
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
303 def os_link(src, dst):
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
304 raise OSError(0, "Hardlinks not supported")
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
305
1082
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
306 # Platform specific variants
419
28511fc21073 [PATCH] file seperator handling for the other 'OS'
mpm@selenic.com
parents:
diff changeset
307 if os.name == 'nt':
461
9ae0034f2772 [PATCH] /dev/null for other OS
mpm@selenic.com
parents: 441
diff changeset
308 nulldev = 'NUL:'
9ae0034f2772 [PATCH] /dev/null for other OS
mpm@selenic.com
parents: 441
diff changeset
309
1241
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
310 try: # ActivePython can create hard links using win32file module
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
311 import win32file
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
312
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
313 def os_link(src, dst): # NB will only succeed on NTFS
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
314 win32file.CreateHardLink(dst, src)
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
315
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
316 def nlinks(pathname):
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
317 """Return number of hardlinks for the given file."""
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
318 try:
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
319 fh = win32file.CreateFile(pathname,
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
320 win32file.GENERIC_READ, win32file.FILE_SHARE_READ,
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
321 None, win32file.OPEN_EXISTING, 0, None)
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
322 res = win32file.GetFileInformationByHandle(fh)
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
323 fh.Close()
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
324 return res[7]
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
325 except:
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
326 return os.stat(pathname).st_nlink
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
327
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
328 except ImportError:
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
329 pass
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
330
441
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
331 def is_exec(f, last):
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
332 return last
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
333
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
334 def set_exec(f, mode):
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
335 pass
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 508
diff changeset
336
419
28511fc21073 [PATCH] file seperator handling for the other 'OS'
mpm@selenic.com
parents:
diff changeset
337 def pconvert(path):
28511fc21073 [PATCH] file seperator handling for the other 'OS'
mpm@selenic.com
parents:
diff changeset
338 return path.replace("\\", "/")
422
10c43444a38e [PATCH] Enables lock work under the other 'OS'
mpm@selenic.com
parents: 421
diff changeset
339
886
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
340 def localpath(path):
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
341 return path.replace('/', '\\')
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
342
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
343 def normpath(path):
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
344 return pconvert(os.path.normpath(path))
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
345
704
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
346 makelock = _makelock_file
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
347 readlock = _readlock_file
461
9ae0034f2772 [PATCH] /dev/null for other OS
mpm@selenic.com
parents: 441
diff changeset
348
782
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
349 def explain_exit(code):
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
350 return "exited with status %d" % code, code
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
351
419
28511fc21073 [PATCH] file seperator handling for the other 'OS'
mpm@selenic.com
parents:
diff changeset
352 else:
461
9ae0034f2772 [PATCH] /dev/null for other OS
mpm@selenic.com
parents: 441
diff changeset
353 nulldev = '/dev/null'
9ae0034f2772 [PATCH] /dev/null for other OS
mpm@selenic.com
parents: 441
diff changeset
354
441
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
355 def is_exec(f, last):
1082
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
356 """check whether a file is executable"""
441
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
357 return (os.stat(f).st_mode & 0100 != 0)
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
358
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
359 def set_exec(f, mode):
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
360 s = os.stat(f).st_mode
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
361 if (s & 0100 != 0) == mode:
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
362 return
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
363 if mode:
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
364 # Turn on +x for every +r bit when making a file executable
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
365 # and obey umask.
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
366 umask = os.umask(0)
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
367 os.umask(umask)
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
368 os.chmod(f, s | (s & 0444) >> 2 & ~umask)
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
369 else:
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
370 os.chmod(f, s & 0666)
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
371
419
28511fc21073 [PATCH] file seperator handling for the other 'OS'
mpm@selenic.com
parents:
diff changeset
372 def pconvert(path):
28511fc21073 [PATCH] file seperator handling for the other 'OS'
mpm@selenic.com
parents:
diff changeset
373 return path
28511fc21073 [PATCH] file seperator handling for the other 'OS'
mpm@selenic.com
parents:
diff changeset
374
886
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
375 def localpath(path):
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
376 return path
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
377
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
378 normpath = os.path.normpath
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
379
422
10c43444a38e [PATCH] Enables lock work under the other 'OS'
mpm@selenic.com
parents: 421
diff changeset
380 def makelock(info, pathname):
704
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
381 try:
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
382 os.symlink(info, pathname)
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
383 except OSError, why:
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
384 if why.errno == errno.EEXIST:
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
385 raise
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
386 else:
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
387 _makelock_file(info, pathname)
422
10c43444a38e [PATCH] Enables lock work under the other 'OS'
mpm@selenic.com
parents: 421
diff changeset
388
10c43444a38e [PATCH] Enables lock work under the other 'OS'
mpm@selenic.com
parents: 421
diff changeset
389 def readlock(pathname):
704
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
390 try:
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
391 return os.readlink(pathname)
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
392 except OSError, why:
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
393 if why.errno == errno.EINVAL:
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
394 return _readlock_file(pathname)
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
395 else:
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
396 raise
782
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
397
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
398 def explain_exit(code):
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
399 """return a 2-tuple (desc, code) describing a process's status"""
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
400 if os.WIFEXITED(code):
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
401 val = os.WEXITSTATUS(code)
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
402 return "exited with status %d" % val, val
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
403 elif os.WIFSIGNALED(code):
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
404 val = os.WTERMSIG(code)
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
405 return "killed by signal %d" % val, val
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
406 elif os.WIFSTOPPED(code):
912
302f83b85054 Minor tweak: os.STOPSIG -> os.WSTOPSIG. Pychecker spotted this one.
mark.williamson@cl.cam.ac.uk
parents: 897
diff changeset
407 val = os.WSTOPSIG(code)
782
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
408 return "stopped by signal %d" % val, val
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
409 raise ValueError("invalid exit code")
1199
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
410
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
411 class chunkbuffer(object):
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
412 """Allow arbitrary sized chunks of data to be efficiently read from an
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
413 iterator over chunks of arbitrary size."""
1200
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
414
1199
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
415 def __init__(self, in_iter, targetsize = 2**16):
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
416 """in_iter is the iterator that's iterating over the input chunks.
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
417 targetsize is how big a buffer to try to maintain."""
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
418 self.in_iter = iter(in_iter)
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
419 self.buf = ''
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
420 self.targetsize = int(targetsize)
1200
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
421 if self.targetsize <= 0:
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
422 raise ValueError("targetsize must be greater than 0, was %d" %
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
423 targetsize)
1199
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
424 self.iterempty = False
1200
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
425
1199
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
426 def fillbuf(self):
1200
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
427 """Ignore target size; read every chunk from iterator until empty."""
1199
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
428 if not self.iterempty:
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
429 collector = cStringIO.StringIO()
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
430 collector.write(self.buf)
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
431 for ch in self.in_iter:
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
432 collector.write(ch)
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
433 self.buf = collector.getvalue()
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
434 self.iterempty = True
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
435
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
436 def read(self, l):
1200
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
437 """Read L bytes of data from the iterator of chunks of data.
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
438 Returns less than L bytes if the iterator runs dry."""
1199
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
439 if l > len(self.buf) and not self.iterempty:
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
440 # Clamp to a multiple of self.targetsize
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
441 targetsize = self.targetsize * ((l // self.targetsize) + 1)
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
442 collector = cStringIO.StringIO()
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
443 collector.write(self.buf)
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
444 collected = len(self.buf)
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
445 for chunk in self.in_iter:
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
446 collector.write(chunk)
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
447 collected += len(chunk)
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
448 if collected >= targetsize:
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
449 break
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
450 if collected < targetsize:
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
451 self.iterempty = True
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
452 self.buf = collector.getvalue()
1200
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
453 s, self.buf = self.buf[:l], buffer(self.buf, l)
1199
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
454 return s
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
455
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
456 def filechunkiter(f, size = 65536):
1200
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
457 """Create a generator that produces all the data in the file size
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
458 (default 65536) bytes at a time. Chunks may be less than size
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
459 bytes if the chunk is the last chunk in the file, or the file is a
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
460 socket or some other type of file that sometimes reads less data
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
461 than is requested."""
1199
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
462 s = f.read(size)
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
463 while len(s) >= 0:
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
464 yield s
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
465 s = f.read(size)