Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/changegroup.py @ 3660:8500a13ec44b

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
create a readbundle function
author Matt Mackall <mpm@selenic.com>
date Wed, 15 Nov 2006 15:51:58 -0600
parents 025f68f22ae2
children f4dc02d7fb71
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
1 """
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
2 changegroup.py - Mercurial changegroup manipulation functions
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
3
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
4 Copyright 2006 Matt Mackall <mpm@selenic.com>
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
5
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
6 This software may be used and distributed according to the terms
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
7 of the GNU General Public License, incorporated herein by reference.
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
8 """
1997
802e8a029d99 Added missing gettext import to changegroup.py.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1981
diff changeset
9 from i18n import gettext as _
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
10 from demandload import *
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
11 demandload(globals(), "struct os bz2 util tempfile")
1981
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
12
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
13 def getchunk(source):
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
14 """get a chunk from a changegroup"""
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
15 d = source.read(4)
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
16 if not d:
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
17 return ""
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
18 l = struct.unpack(">l", d)[0]
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
19 if l <= 4:
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
20 return ""
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
21 d = source.read(l - 4)
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
22 if len(d) < l - 4:
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
23 raise util.Abort(_("premature EOF reading chunk"
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
24 " (got %d bytes, expected %d)")
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
25 % (len(d), l - 4))
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
26 return d
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
27
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
28 def chunkiter(source):
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
29 """iterate through the chunks in source"""
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
30 while 1:
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
31 c = getchunk(source)
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
32 if not c:
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
33 break
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
34 yield c
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
35
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
36 def genchunk(data):
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
37 """build a changegroup chunk"""
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
38 header = struct.pack(">l", len(data)+ 4)
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
39 return "%s%s" % (header, data)
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
40
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
41 def closechunk():
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
42 return struct.pack(">l", 0)
736b6c96bbbc make incoming work via ssh (issue139); move chunk code into separate module.
Thomas Arendsen Hein <thomas@intevation.de>
parents:
diff changeset
43
3659
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
44 class nocompress(object):
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
45 def compress(self, x):
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
46 return x
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
47 def flush(self):
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
48 return ""
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
49
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
50 def writebundle(cg, filename, compress):
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
51 """Write a bundle file and return its filename.
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
52
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
53 Existing files will not be overwritten.
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
54 If no filename is specified, a temporary file is created.
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
55 bz2 compression can be turned off.
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
56 The bundle file will be deleted in case of errors.
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
57 """
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
58
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
59 fh = None
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
60 cleanup = None
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
61 try:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
62 if filename:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
63 if os.path.exists(filename):
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
64 raise util.Abort(_("file '%s' already exists") % filename)
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
65 fh = open(filename, "wb")
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
66 else:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
67 fd, filename = tempfile.mkstemp(prefix="hg-bundle-", suffix=".hg")
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
68 fh = os.fdopen(fd, "wb")
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
69 cleanup = filename
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
70
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
71 if compress:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
72 fh.write("HG10")
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
73 z = bz2.BZ2Compressor(9)
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
74 else:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
75 fh.write("HG10UN")
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
76 z = nocompress()
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
77 # parse the changegroup data, otherwise we will block
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
78 # in case of sshrepo because we don't know the end of the stream
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
79
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
80 # an empty chunkiter is the end of the changegroup
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
81 empty = False
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
82 while not empty:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
83 empty = True
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
84 for chunk in chunkiter(cg):
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
85 empty = False
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
86 fh.write(z.compress(genchunk(chunk)))
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
87 fh.write(z.compress(closechunk()))
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
88 fh.write(z.flush())
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
89 cleanup = None
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
90 return filename
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
91 finally:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
92 if fh is not None:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
93 fh.close()
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
94 if cleanup is not None:
025f68f22ae2 move write_bundle to changegroup.py
Matt Mackall <mpm@selenic.com>
parents: 2470
diff changeset
95 os.unlink(cleanup)
3660
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
96
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
97 def readbundle(fh):
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
98 header = fh.read(6)
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
99 if not header.startswith("HG"):
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
100 raise util.Abort(_("%s: not a Mercurial bundle file") % fname)
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
101 elif not header.startswith("HG10"):
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
102 raise util.Abort(_("%s: unknown bundle version") % fname)
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
103
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
104 if header == "HG10BZ":
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
105 def generator(f):
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
106 zd = bz2.BZ2Decompressor()
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
107 zd.decompress("BZ")
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
108 for chunk in util.filechunkiter(f, 4096):
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
109 yield zd.decompress(chunk)
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
110 return util.chunkbuffer(generator(fh))
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
111 elif header == "HG10UN":
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
112 return fh
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
113
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
114 raise util.Abort(_("%s: unknown bundle compression type")
8500a13ec44b create a readbundle function
Matt Mackall <mpm@selenic.com>
parents: 3659
diff changeset
115 % fname)