annotate mercurial/parsers.c @ 18988:5bae936764bb

parsers: a C implementation of the new ancestors algorithm The performance of both the old and new Python ancestor algorithms depends on the number of revs they need to traverse. Although the new algorithm performs far better than the old when revs are numerically and topologically close, both algorithms become slow under other circumstances, taking up to 1.8 seconds to give answers in a Linux kernel repo. This C implementation of the new algorithm is a fairly straightforward transliteration. The only corner case of interest is that it raises an OverflowError if the number of GCA candidates found during the first pass is greater than 24, to avoid the dual perils of fixnum overflow and trying to allocate too much memory. (If this exception is raised, the Python implementation is used instead.) Performance numbers are good: in a Linux kernel repo, time for "hg debugancestors" on two distant revs (24bf01de7537 and c2a8808f5943) is as follows: Old Python: 0.36 sec New Python: 0.42 sec New C: 0.02 sec For a case where the new algorithm should perform well: Old Python: 1.84 sec New Python: 0.07 sec New C: measures as zero when using --time (This commit includes a paranoid cross-check to ensure that the Python and C implementations give identical answers. The above performance numbers were measured with that check disabled.)
author Bryan O'Sullivan <bryano@fb.com>
date Tue, 16 Apr 2013 10:08:20 -0700
parents 02ee846b246a
children 48d6f436363e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
1 /*
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
2 parsers.c - efficient content parsing
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
3
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
4 Copyright 2008 Matt Mackall <mpm@selenic.com> and others
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
5
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
6 This software may be used and distributed according to the terms of
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
7 the GNU General Public License, incorporated herein by reference.
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
8 */
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
9
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
10 #include <Python.h>
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
11 #include <ctype.h>
17356
511dfb34b412 parsers: fix an integer size warning issued by clang
Bryan O'Sullivan <bryano@fb.com>
parents: 17353
diff changeset
12 #include <stddef.h>
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
13 #include <string.h>
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
14
11361
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
15 #include "util.h"
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
16
16617
4fb16743049d parsers: change the type signature of hexdigit
Bryan O'Sullivan <bryano@fb.com>
parents: 16616
diff changeset
17 static inline int hexdigit(const char *p, Py_ssize_t off)
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
18 {
16617
4fb16743049d parsers: change the type signature of hexdigit
Bryan O'Sullivan <bryano@fb.com>
parents: 16616
diff changeset
19 char c = p[off];
4fb16743049d parsers: change the type signature of hexdigit
Bryan O'Sullivan <bryano@fb.com>
parents: 16616
diff changeset
20
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
21 if (c >= '0' && c <= '9')
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
22 return c - '0';
7092
fb3fc27617a2 parsers: speed up hex decoding for manifests
Matt Mackall <mpm@selenic.com>
parents: 7091
diff changeset
23 if (c >= 'a' && c <= 'f')
fb3fc27617a2 parsers: speed up hex decoding for manifests
Matt Mackall <mpm@selenic.com>
parents: 7091
diff changeset
24 return c - 'a' + 10;
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
25 if (c >= 'A' && c <= 'F')
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
26 return c - 'A' + 10;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
27
7092
fb3fc27617a2 parsers: speed up hex decoding for manifests
Matt Mackall <mpm@selenic.com>
parents: 7091
diff changeset
28 PyErr_SetString(PyExc_ValueError, "input contains non-hex character");
fb3fc27617a2 parsers: speed up hex decoding for manifests
Matt Mackall <mpm@selenic.com>
parents: 7091
diff changeset
29 return 0;
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
30 }
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
31
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
32 /*
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
33 * Turn a hex-encoded string into binary.
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
34 */
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
35 static PyObject *unhexlify(const char *str, int len)
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
36 {
7092
fb3fc27617a2 parsers: speed up hex decoding for manifests
Matt Mackall <mpm@selenic.com>
parents: 7091
diff changeset
37 PyObject *ret;
6395
3f0294536b24 fix const annotation warning
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 6389
diff changeset
38 char *d;
16617
4fb16743049d parsers: change the type signature of hexdigit
Bryan O'Sullivan <bryano@fb.com>
parents: 16616
diff changeset
39 int i;
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
40
11361
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
41 ret = PyBytes_FromStringAndSize(NULL, len / 2);
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
42
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
43 if (!ret)
7092
fb3fc27617a2 parsers: speed up hex decoding for manifests
Matt Mackall <mpm@selenic.com>
parents: 7091
diff changeset
44 return NULL;
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
45
11361
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
46 d = PyBytes_AsString(ret);
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
47
16617
4fb16743049d parsers: change the type signature of hexdigit
Bryan O'Sullivan <bryano@fb.com>
parents: 16616
diff changeset
48 for (i = 0; i < len;) {
4fb16743049d parsers: change the type signature of hexdigit
Bryan O'Sullivan <bryano@fb.com>
parents: 16616
diff changeset
49 int hi = hexdigit(str, i++);
4fb16743049d parsers: change the type signature of hexdigit
Bryan O'Sullivan <bryano@fb.com>
parents: 16616
diff changeset
50 int lo = hexdigit(str, i++);
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
51 *d++ = (hi << 4) | lo;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
52 }
7091
12b35ae03365 parsers: clean up whitespace
Matt Mackall <mpm@selenic.com>
parents: 6395
diff changeset
53
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
54 return ret;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
55 }
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
56
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
57 /*
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
58 * This code assumes that a manifest is stitched together with newline
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
59 * ('\n') characters.
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
60 */
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
61 static PyObject *parse_manifest(PyObject *self, PyObject *args)
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
62 {
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
63 PyObject *mfdict, *fdict;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
64 char *str, *cur, *start, *zero;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
65 int len;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
66
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
67 if (!PyArg_ParseTuple(args, "O!O!s#:parse_manifest",
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
68 &PyDict_Type, &mfdict,
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
69 &PyDict_Type, &fdict,
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
70 &str, &len))
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
71 goto quit;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
72
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
73 for (start = cur = str, zero = NULL; cur < str + len; cur++) {
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
74 PyObject *file = NULL, *node = NULL;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
75 PyObject *flags = NULL;
17356
511dfb34b412 parsers: fix an integer size warning issued by clang
Bryan O'Sullivan <bryano@fb.com>
parents: 17353
diff changeset
76 ptrdiff_t nlen;
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
77
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
78 if (!*cur) {
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
79 zero = cur;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
80 continue;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
81 }
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
82 else if (*cur != '\n')
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
83 continue;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
84
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
85 if (!zero) {
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
86 PyErr_SetString(PyExc_ValueError,
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
87 "manifest entry has no separator");
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
88 goto quit;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
89 }
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
90
11361
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
91 file = PyBytes_FromStringAndSize(start, zero - start);
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
92
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
93 if (!file)
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
94 goto bail;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
95
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
96 nlen = cur - zero - 1;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
97
17356
511dfb34b412 parsers: fix an integer size warning issued by clang
Bryan O'Sullivan <bryano@fb.com>
parents: 17353
diff changeset
98 node = unhexlify(zero + 1, nlen > 40 ? 40 : (int)nlen);
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
99 if (!node)
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
100 goto bail;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
101
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
102 if (nlen > 40) {
11361
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
103 flags = PyBytes_FromStringAndSize(zero + 41,
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
104 nlen - 40);
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
105 if (!flags)
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
106 goto bail;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
107
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
108 if (PyDict_SetItem(fdict, file, flags) == -1)
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
109 goto bail;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
110 }
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
111
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
112 if (PyDict_SetItem(mfdict, file, node) == -1)
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
113 goto bail;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
114
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
115 start = cur + 1;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
116 zero = NULL;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
117
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
118 Py_XDECREF(flags);
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
119 Py_XDECREF(node);
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
120 Py_XDECREF(file);
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
121 continue;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
122 bail:
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
123 Py_XDECREF(flags);
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
124 Py_XDECREF(node);
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
125 Py_XDECREF(file);
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
126 goto quit;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
127 }
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
128
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
129 if (len > 0 && *(cur - 1) != '\n') {
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
130 PyErr_SetString(PyExc_ValueError,
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
131 "manifest contains trailing garbage");
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
132 goto quit;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
133 }
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
134
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
135 Py_INCREF(Py_None);
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
136 return Py_None;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
137 quit:
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
138 return NULL;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
139 }
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
140
7093
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
141 static PyObject *parse_dirstate(PyObject *self, PyObject *args)
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
142 {
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
143 PyObject *dmap, *cmap, *parents = NULL, *ret = NULL;
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
144 PyObject *fname = NULL, *cname = NULL, *entry = NULL;
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
145 char *str, *cur, *end, *cpos;
7174
4da87407b845 parsers.c: fix integer overflows
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7168
diff changeset
146 int state, mode, size, mtime;
4da87407b845 parsers.c: fix integer overflows
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7168
diff changeset
147 unsigned int flen;
7093
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
148 int len;
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
149
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
150 if (!PyArg_ParseTuple(args, "O!O!s#:parse_dirstate",
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
151 &PyDict_Type, &dmap,
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
152 &PyDict_Type, &cmap,
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
153 &str, &len))
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
154 goto quit;
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
155
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
156 /* read parents */
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
157 if (len < 40)
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
158 goto quit;
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
159
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
160 parents = Py_BuildValue("s#s#", str, 20, str + 20, 20);
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
161 if (!parents)
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
162 goto quit;
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
163
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
164 /* read filenames */
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
165 cur = str + 40;
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
166 end = str + len;
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
167
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
168 while (cur < end - 17) {
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
169 /* unpack header */
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
170 state = *cur;
16437
d126a0d16856 util.h: replace ntohl/htonl with get/putbe32
Matt Mackall <mpm@selenic.com>
parents: 16414
diff changeset
171 mode = getbe32(cur + 1);
d126a0d16856 util.h: replace ntohl/htonl with get/putbe32
Matt Mackall <mpm@selenic.com>
parents: 16414
diff changeset
172 size = getbe32(cur + 5);
d126a0d16856 util.h: replace ntohl/htonl with get/putbe32
Matt Mackall <mpm@selenic.com>
parents: 16414
diff changeset
173 mtime = getbe32(cur + 9);
d126a0d16856 util.h: replace ntohl/htonl with get/putbe32
Matt Mackall <mpm@selenic.com>
parents: 16414
diff changeset
174 flen = getbe32(cur + 13);
7093
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
175 cur += 17;
10449
7c8266c1d15a parsers: fix some signed comparison issues
Matt Mackall <mpm@selenic.com>
parents: 10282
diff changeset
176 if (cur + flen > end || cur + flen < cur) {
7174
4da87407b845 parsers.c: fix integer overflows
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7168
diff changeset
177 PyErr_SetString(PyExc_ValueError, "overflow in dirstate");
7093
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
178 goto quit;
7174
4da87407b845 parsers.c: fix integer overflows
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7168
diff changeset
179 }
7093
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
180
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
181 entry = Py_BuildValue("ciii", state, mode, size, mtime);
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
182 if (!entry)
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
183 goto quit;
7175
5d8626b2c1db parsers.c: do not try to untrack after a failure
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7174
diff changeset
184 PyObject_GC_UnTrack(entry); /* don't waste time with this */
7093
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
185
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
186 cpos = memchr(cur, 0, flen);
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
187 if (cpos) {
11361
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
188 fname = PyBytes_FromStringAndSize(cur, cpos - cur);
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
189 cname = PyBytes_FromStringAndSize(cpos + 1,
7093
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
190 flen - (cpos - cur) - 1);
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
191 if (!fname || !cname ||
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
192 PyDict_SetItem(cmap, fname, cname) == -1 ||
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
193 PyDict_SetItem(dmap, fname, entry) == -1)
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
194 goto quit;
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
195 Py_DECREF(cname);
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
196 } else {
11361
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
197 fname = PyBytes_FromStringAndSize(cur, flen);
7093
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
198 if (!fname ||
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
199 PyDict_SetItem(dmap, fname, entry) == -1)
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
200 goto quit;
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
201 }
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
202 cur += flen;
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
203 Py_DECREF(fname);
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
204 Py_DECREF(entry);
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
205 fname = cname = entry = NULL;
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
206 }
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
207
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
208 ret = parents;
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
209 Py_INCREF(ret);
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
210 quit:
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
211 Py_XDECREF(fname);
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
212 Py_XDECREF(cname);
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
213 Py_XDECREF(entry);
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
214 Py_XDECREF(parents);
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
215 return ret;
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
216 }
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
217
16955
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
218 static inline int getintat(PyObject *tuple, int off, uint32_t *v)
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
219 {
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
220 PyObject *o = PyTuple_GET_ITEM(tuple, off);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
221 long val;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
222
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
223 if (PyInt_Check(o))
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
224 val = PyInt_AS_LONG(o);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
225 else if (PyLong_Check(o)) {
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
226 val = PyLong_AsLong(o);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
227 if (val == -1 && PyErr_Occurred())
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
228 return -1;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
229 } else {
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
230 PyErr_SetString(PyExc_TypeError, "expected an int or long");
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
231 return -1;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
232 }
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
233 if (LONG_MAX > INT_MAX && (val > INT_MAX || val < INT_MIN)) {
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
234 PyErr_SetString(PyExc_OverflowError,
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
235 "Python value to large to convert to uint32_t");
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
236 return -1;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
237 }
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
238 *v = (uint32_t)val;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
239 return 0;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
240 }
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
241
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
242 static PyObject *dirstate_unset;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
243
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
244 /*
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
245 * Efficiently pack a dirstate object into its on-disk format.
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
246 */
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
247 static PyObject *pack_dirstate(PyObject *self, PyObject *args)
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
248 {
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
249 PyObject *packobj = NULL;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
250 PyObject *map, *copymap, *pl;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
251 Py_ssize_t nbytes, pos, l;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
252 PyObject *k, *v, *pn;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
253 char *p, *s;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
254 double now;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
255
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
256 if (!PyArg_ParseTuple(args, "O!O!Od:pack_dirstate",
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
257 &PyDict_Type, &map, &PyDict_Type, &copymap,
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
258 &pl, &now))
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
259 return NULL;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
260
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
261 if (!PySequence_Check(pl) || PySequence_Size(pl) != 2) {
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
262 PyErr_SetString(PyExc_TypeError, "expected 2-element sequence");
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
263 return NULL;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
264 }
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
265
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
266 /* Figure out how much we need to allocate. */
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
267 for (nbytes = 40, pos = 0; PyDict_Next(map, &pos, &k, &v);) {
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
268 PyObject *c;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
269 if (!PyString_Check(k)) {
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
270 PyErr_SetString(PyExc_TypeError, "expected string key");
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
271 goto bail;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
272 }
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
273 nbytes += PyString_GET_SIZE(k) + 17;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
274 c = PyDict_GetItem(copymap, k);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
275 if (c) {
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
276 if (!PyString_Check(c)) {
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
277 PyErr_SetString(PyExc_TypeError,
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
278 "expected string key");
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
279 goto bail;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
280 }
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
281 nbytes += PyString_GET_SIZE(c) + 1;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
282 }
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
283 }
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
284
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
285 packobj = PyString_FromStringAndSize(NULL, nbytes);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
286 if (packobj == NULL)
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
287 goto bail;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
288
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
289 p = PyString_AS_STRING(packobj);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
290
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
291 pn = PySequence_ITEM(pl, 0);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
292 if (PyString_AsStringAndSize(pn, &s, &l) == -1 || l != 20) {
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
293 PyErr_SetString(PyExc_TypeError, "expected a 20-byte hash");
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
294 goto bail;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
295 }
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
296 memcpy(p, s, l);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
297 p += 20;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
298 pn = PySequence_ITEM(pl, 1);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
299 if (PyString_AsStringAndSize(pn, &s, &l) == -1 || l != 20) {
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
300 PyErr_SetString(PyExc_TypeError, "expected a 20-byte hash");
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
301 goto bail;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
302 }
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
303 memcpy(p, s, l);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
304 p += 20;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
305
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
306 for (pos = 0; PyDict_Next(map, &pos, &k, &v); ) {
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
307 uint32_t mode, size, mtime;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
308 Py_ssize_t len, l;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
309 PyObject *o;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
310 char *s, *t;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
311
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
312 if (!PyTuple_Check(v) || PyTuple_GET_SIZE(v) != 4) {
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
313 PyErr_SetString(PyExc_TypeError, "expected a 4-tuple");
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
314 goto bail;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
315 }
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
316 o = PyTuple_GET_ITEM(v, 0);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
317 if (PyString_AsStringAndSize(o, &s, &l) == -1 || l != 1) {
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
318 PyErr_SetString(PyExc_TypeError, "expected one byte");
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
319 goto bail;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
320 }
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
321 *p++ = *s;
17165
249cc4ec4327 parsers.c: remove warning: 'size' may be used uninitialized in this function
Mads Kiilerich <mads@kiilerich.com>
parents: 16955
diff changeset
322 if (getintat(v, 1, &mode) == -1)
249cc4ec4327 parsers.c: remove warning: 'size' may be used uninitialized in this function
Mads Kiilerich <mads@kiilerich.com>
parents: 16955
diff changeset
323 goto bail;
249cc4ec4327 parsers.c: remove warning: 'size' may be used uninitialized in this function
Mads Kiilerich <mads@kiilerich.com>
parents: 16955
diff changeset
324 if (getintat(v, 2, &size) == -1)
249cc4ec4327 parsers.c: remove warning: 'size' may be used uninitialized in this function
Mads Kiilerich <mads@kiilerich.com>
parents: 16955
diff changeset
325 goto bail;
249cc4ec4327 parsers.c: remove warning: 'size' may be used uninitialized in this function
Mads Kiilerich <mads@kiilerich.com>
parents: 16955
diff changeset
326 if (getintat(v, 3, &mtime) == -1)
16955
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
327 goto bail;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
328 if (*s == 'n' && mtime == (uint32_t)now) {
18567
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 18504
diff changeset
329 /* See pure/parsers.py:pack_dirstate for why we do
194e63c1ccb9 dirstate: move pure python dirstate packing to pure/parsers.py
Siddharth Agarwal <sid0@fb.com>
parents: 18504
diff changeset
330 * this. */
16955
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
331 if (PyDict_SetItem(map, k, dirstate_unset) == -1)
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
332 goto bail;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
333 mode = 0, size = -1, mtime = -1;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
334 }
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
335 putbe32(mode, p);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
336 putbe32(size, p + 4);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
337 putbe32(mtime, p + 8);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
338 t = p + 12;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
339 p += 16;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
340 len = PyString_GET_SIZE(k);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
341 memcpy(p, PyString_AS_STRING(k), len);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
342 p += len;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
343 o = PyDict_GetItem(copymap, k);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
344 if (o) {
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
345 *p++ = '\0';
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
346 l = PyString_GET_SIZE(o);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
347 memcpy(p, PyString_AS_STRING(o), l);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
348 p += l;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
349 len += l + 1;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
350 }
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
351 putbe32((uint32_t)len, t);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
352 }
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
353
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
354 pos = p - PyString_AS_STRING(packobj);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
355 if (pos != nbytes) {
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
356 PyErr_Format(PyExc_SystemError, "bad dirstate size: %ld != %ld",
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
357 (long)pos, (long)nbytes);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
358 goto bail;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
359 }
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
360
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
361 return packobj;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
362 bail:
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
363 Py_XDECREF(packobj);
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
364 return NULL;
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
365 }
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
366
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
367 /*
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
368 * A base-16 trie for fast node->rev mapping.
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
369 *
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
370 * Positive value is index of the next node in the trie
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
371 * Negative value is a leaf: -(rev + 1)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
372 * Zero is empty
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
373 */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
374 typedef struct {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
375 int children[16];
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
376 } nodetree;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
377
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
378 /*
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
379 * This class has two behaviours.
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
380 *
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
381 * When used in a list-like way (with integer keys), we decode an
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
382 * entry in a RevlogNG index file on demand. Our last entry is a
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
383 * sentinel, always a nullid. We have limited support for
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
384 * integer-keyed insert and delete, only at elements right before the
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
385 * sentinel.
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
386 *
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
387 * With string keys, we lazily perform a reverse mapping from node to
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
388 * rev, using a base-16 trie.
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
389 */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
390 typedef struct {
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
391 PyObject_HEAD
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
392 /* Type-specific fields go here. */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
393 PyObject *data; /* raw bytes of index */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
394 PyObject **cache; /* cached tuples */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
395 const char **offsets; /* populated on demand */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
396 Py_ssize_t raw_length; /* original number of elements */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
397 Py_ssize_t length; /* current number of elements */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
398 PyObject *added; /* populated on demand */
16787
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
399 PyObject *headrevs; /* cache, invalidated on changes */
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
400 nodetree *nt; /* base-16 trie */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
401 int ntlength; /* # nodes in use */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
402 int ntcapacity; /* # nodes allocated */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
403 int ntdepth; /* maximum depth of tree */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
404 int ntsplits; /* # splits performed */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
405 int ntrev; /* last rev scanned */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
406 int ntlookups; /* # lookups */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
407 int ntmisses; /* # lookups that miss the cache */
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
408 int inlined;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
409 } indexObject;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
410
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
411 static Py_ssize_t index_length(const indexObject *self)
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
412 {
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
413 if (self->added == NULL)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
414 return self->length;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
415 return self->length + PyList_GET_SIZE(self->added);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
416 }
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
417
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
418 static PyObject *nullentry;
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
419 static const char nullid[20];
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
420
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
421 static long inline_scan(indexObject *self, const char **offsets);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
422
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
423 #if LONG_MAX == 0x7fffffffL
16393
ee163a9cf37c util.h: more Python 2.4 fixes
Matt Mackall <mpm@selenic.com>
parents: 16385
diff changeset
424 static char *tuple_format = "Kiiiiiis#";
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
425 #else
16393
ee163a9cf37c util.h: more Python 2.4 fixes
Matt Mackall <mpm@selenic.com>
parents: 16385
diff changeset
426 static char *tuple_format = "kiiiiiis#";
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
427 #endif
7108
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
428
16863
bbedef66c6f3 parsers: replace magic number 64 with symbolic constant
Bryan O'Sullivan <bryano@fb.com>
parents: 16787
diff changeset
429 /* A RevlogNG v1 index entry is 64 bytes long. */
bbedef66c6f3 parsers: replace magic number 64 with symbolic constant
Bryan O'Sullivan <bryano@fb.com>
parents: 16787
diff changeset
430 static const long v1_hdrsize = 64;
bbedef66c6f3 parsers: replace magic number 64 with symbolic constant
Bryan O'Sullivan <bryano@fb.com>
parents: 16787
diff changeset
431
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
432 /*
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
433 * Return a pointer to the beginning of a RevlogNG record.
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
434 */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
435 static const char *index_deref(indexObject *self, Py_ssize_t pos)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
436 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
437 if (self->inlined && pos > 0) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
438 if (self->offsets == NULL) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
439 self->offsets = malloc(self->raw_length *
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
440 sizeof(*self->offsets));
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
441 if (self->offsets == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
442 return (const char *)PyErr_NoMemory();
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
443 inline_scan(self, self->offsets);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
444 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
445 return self->offsets[pos];
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
446 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
447
16863
bbedef66c6f3 parsers: replace magic number 64 with symbolic constant
Bryan O'Sullivan <bryano@fb.com>
parents: 16787
diff changeset
448 return PyString_AS_STRING(self->data) + pos * v1_hdrsize;
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
449 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
450
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
451 /*
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
452 * RevlogNG format (all in big endian, data may be inlined):
7108
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
453 * 6 bytes: offset
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
454 * 2 bytes: flags
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
455 * 4 bytes: compressed length
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
456 * 4 bytes: uncompressed length
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
457 * 4 bytes: base revision
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
458 * 4 bytes: link revision
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
459 * 4 bytes: parent 1 revision
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
460 * 4 bytes: parent 2 revision
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
461 * 32 bytes: nodeid (only 20 bytes used)
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
462 */
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
463 static PyObject *index_get(indexObject *self, Py_ssize_t pos)
7108
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
464 {
7154
7fdf7a0a41b7 index parser: fix refcounting in case of errors, refactor
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7135
diff changeset
465 uint64_t offset_flags;
7108
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
466 int comp_len, uncomp_len, base_rev, link_rev, parent_1, parent_2;
7154
7fdf7a0a41b7 index parser: fix refcounting in case of errors, refactor
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7135
diff changeset
467 const char *c_node_id;
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
468 const char *data;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
469 Py_ssize_t length = index_length(self);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
470 PyObject *entry;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
471
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
472 if (pos < 0)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
473 pos += length;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
474
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
475 if (pos < 0 || pos >= length) {
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
476 PyErr_SetString(PyExc_IndexError, "revlog index out of range");
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
477 return NULL;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
478 }
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
479
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
480 if (pos == length - 1) {
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
481 Py_INCREF(nullentry);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
482 return nullentry;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
483 }
7108
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
484
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
485 if (pos >= self->length - 1) {
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
486 PyObject *obj;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
487 obj = PyList_GET_ITEM(self->added, pos - self->length + 1);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
488 Py_INCREF(obj);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
489 return obj;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
490 }
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
491
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
492 if (self->cache) {
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
493 if (self->cache[pos]) {
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
494 Py_INCREF(self->cache[pos]);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
495 return self->cache[pos];
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
496 }
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
497 } else {
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
498 self->cache = calloc(self->raw_length, sizeof(PyObject *));
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
499 if (self->cache == NULL)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
500 return PyErr_NoMemory();
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
501 }
7190
aecea6934fdd Some additional space/tab cleanups
Thomas Arendsen Hein <thomas@intevation.de>
parents: 7186
diff changeset
502
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
503 data = index_deref(self, pos);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
504 if (data == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
505 return NULL;
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
506
16437
d126a0d16856 util.h: replace ntohl/htonl with get/putbe32
Matt Mackall <mpm@selenic.com>
parents: 16414
diff changeset
507 offset_flags = getbe32(data + 4);
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
508 if (pos == 0) /* mask out version number for the first entry */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
509 offset_flags &= 0xFFFF;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
510 else {
16437
d126a0d16856 util.h: replace ntohl/htonl with get/putbe32
Matt Mackall <mpm@selenic.com>
parents: 16414
diff changeset
511 uint32_t offset_high = getbe32(data);
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
512 offset_flags |= ((uint64_t)offset_high) << 32;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
513 }
7154
7fdf7a0a41b7 index parser: fix refcounting in case of errors, refactor
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7135
diff changeset
514
16437
d126a0d16856 util.h: replace ntohl/htonl with get/putbe32
Matt Mackall <mpm@selenic.com>
parents: 16414
diff changeset
515 comp_len = getbe32(data + 8);
d126a0d16856 util.h: replace ntohl/htonl with get/putbe32
Matt Mackall <mpm@selenic.com>
parents: 16414
diff changeset
516 uncomp_len = getbe32(data + 12);
d126a0d16856 util.h: replace ntohl/htonl with get/putbe32
Matt Mackall <mpm@selenic.com>
parents: 16414
diff changeset
517 base_rev = getbe32(data + 16);
d126a0d16856 util.h: replace ntohl/htonl with get/putbe32
Matt Mackall <mpm@selenic.com>
parents: 16414
diff changeset
518 link_rev = getbe32(data + 20);
d126a0d16856 util.h: replace ntohl/htonl with get/putbe32
Matt Mackall <mpm@selenic.com>
parents: 16414
diff changeset
519 parent_1 = getbe32(data + 24);
d126a0d16856 util.h: replace ntohl/htonl with get/putbe32
Matt Mackall <mpm@selenic.com>
parents: 16414
diff changeset
520 parent_2 = getbe32(data + 28);
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
521 c_node_id = data + 32;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
522
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
523 entry = Py_BuildValue(tuple_format, offset_flags, comp_len,
13254
5ef5eb1f3515 revlog: only build the nodemap on demand
Matt Mackall <mpm@selenic.com>
parents: 11361
diff changeset
524 uncomp_len, base_rev, link_rev,
5ef5eb1f3515 revlog: only build the nodemap on demand
Matt Mackall <mpm@selenic.com>
parents: 11361
diff changeset
525 parent_1, parent_2, c_node_id, 20);
5ef5eb1f3515 revlog: only build the nodemap on demand
Matt Mackall <mpm@selenic.com>
parents: 11361
diff changeset
526
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
527 if (entry)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
528 PyObject_GC_UnTrack(entry);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
529
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
530 self->cache[pos] = entry;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
531 Py_INCREF(entry);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
532
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
533 return entry;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
534 }
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
535
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
536 /*
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
537 * Return the 20-byte SHA of the node corresponding to the given rev.
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
538 */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
539 static const char *index_node(indexObject *self, Py_ssize_t pos)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
540 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
541 Py_ssize_t length = index_length(self);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
542 const char *data;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
543
16664
5bc6edf71b39 parsers: ensure that nullid is always present in the radix tree
Bryan O'Sullivan <bryano@fb.com>
parents: 16663
diff changeset
544 if (pos == length - 1 || pos == INT_MAX)
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
545 return nullid;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
546
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
547 if (pos >= length)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
548 return NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
549
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
550 if (pos >= self->length - 1) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
551 PyObject *tuple, *str;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
552 tuple = PyList_GET_ITEM(self->added, pos - self->length + 1);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
553 str = PyTuple_GetItem(tuple, 7);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
554 return str ? PyString_AS_STRING(str) : NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
555 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
556
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
557 data = index_deref(self, pos);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
558 return data ? data + 32 : NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
559 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
560
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
561 static int nt_insert(indexObject *self, const char *node, int rev);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
562
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
563 static int node_check(PyObject *obj, char **node, Py_ssize_t *nodelen)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
564 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
565 if (PyString_AsStringAndSize(obj, node, nodelen) == -1)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
566 return -1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
567 if (*nodelen == 20)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
568 return 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
569 PyErr_SetString(PyExc_ValueError, "20-byte hash required");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
570 return -1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
571 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
572
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
573 static PyObject *index_insert(indexObject *self, PyObject *args)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
574 {
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
575 PyObject *obj;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
576 char *node;
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
577 long offset;
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
578 Py_ssize_t len, nodelen;
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
579
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
580 if (!PyArg_ParseTuple(args, "lO", &offset, &obj))
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
581 return NULL;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
582
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
583 if (!PyTuple_Check(obj) || PyTuple_GET_SIZE(obj) != 8) {
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
584 PyErr_SetString(PyExc_TypeError, "8-tuple required");
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
585 return NULL;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
586 }
7108
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
587
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
588 if (node_check(PyTuple_GET_ITEM(obj, 7), &node, &nodelen) == -1)
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
589 return NULL;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
590
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
591 len = index_length(self);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
592
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
593 if (offset < 0)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
594 offset += len;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
595
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
596 if (offset != len - 1) {
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
597 PyErr_SetString(PyExc_IndexError,
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
598 "insert only supported at index -1");
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
599 return NULL;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
600 }
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
601
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
602 if (offset > INT_MAX) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
603 PyErr_SetString(PyExc_ValueError,
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
604 "currently only 2**31 revs supported");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
605 return NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
606 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
607
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
608 if (self->added == NULL) {
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
609 self->added = PyList_New(0);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
610 if (self->added == NULL)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
611 return NULL;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
612 }
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
613
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
614 if (PyList_Append(self->added, obj) == -1)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
615 return NULL;
13254
5ef5eb1f3515 revlog: only build the nodemap on demand
Matt Mackall <mpm@selenic.com>
parents: 11361
diff changeset
616
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
617 if (self->nt)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
618 nt_insert(self, node, (int)offset);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
619
16787
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
620 Py_CLEAR(self->headrevs);
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
621 Py_RETURN_NONE;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
622 }
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
623
16370
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
624 static void _index_clearcaches(indexObject *self)
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
625 {
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
626 if (self->cache) {
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
627 Py_ssize_t i;
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
628
16732
277e2acb7e5c parsers: use Py_CLEAR where appropriate
Bryan O'Sullivan <bryano@fb.com>
parents: 16699
diff changeset
629 for (i = 0; i < self->raw_length; i++)
277e2acb7e5c parsers: use Py_CLEAR where appropriate
Bryan O'Sullivan <bryano@fb.com>
parents: 16699
diff changeset
630 Py_CLEAR(self->cache[i]);
16370
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
631 free(self->cache);
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
632 self->cache = NULL;
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
633 }
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
634 if (self->offsets) {
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
635 free(self->offsets);
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
636 self->offsets = NULL;
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
637 }
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
638 if (self->nt) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
639 free(self->nt);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
640 self->nt = NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
641 }
16787
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
642 Py_CLEAR(self->headrevs);
16370
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
643 }
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
644
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
645 static PyObject *index_clearcaches(indexObject *self)
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
646 {
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
647 _index_clearcaches(self);
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
648 self->ntlength = self->ntcapacity = 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
649 self->ntdepth = self->ntsplits = 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
650 self->ntrev = -1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
651 self->ntlookups = self->ntmisses = 0;
16370
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
652 Py_RETURN_NONE;
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
653 }
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
654
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
655 static PyObject *index_stats(indexObject *self)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
656 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
657 PyObject *obj = PyDict_New();
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
658
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
659 if (obj == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
660 return NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
661
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
662 #define istat(__n, __d) \
16629
1435866c1937 parser: use PyInt_FromSsize_t in index_stats
Adrian Buehlmann <adrian@cadifra.com>
parents: 16621
diff changeset
663 if (PyDict_SetItemString(obj, __d, PyInt_FromSsize_t(self->__n)) == -1) \
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
664 goto bail;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
665
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
666 if (self->added) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
667 Py_ssize_t len = PyList_GET_SIZE(self->added);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
668 if (PyDict_SetItemString(obj, "index entries added",
16629
1435866c1937 parser: use PyInt_FromSsize_t in index_stats
Adrian Buehlmann <adrian@cadifra.com>
parents: 16621
diff changeset
669 PyInt_FromSsize_t(len)) == -1)
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
670 goto bail;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
671 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
672
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
673 if (self->raw_length != self->length - 1)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
674 istat(raw_length, "revs on disk");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
675 istat(length, "revs in memory");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
676 istat(ntcapacity, "node trie capacity");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
677 istat(ntdepth, "node trie depth");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
678 istat(ntlength, "node trie count");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
679 istat(ntlookups, "node trie lookups");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
680 istat(ntmisses, "node trie misses");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
681 istat(ntrev, "node trie last rev scanned");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
682 istat(ntsplits, "node trie splits");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
683
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
684 #undef istat
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
685
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
686 return obj;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
687
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
688 bail:
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
689 Py_XDECREF(obj);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
690 return NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
691 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
692
16787
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
693 /*
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
694 * When we cache a list, we want to be sure the caller can't mutate
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
695 * the cached copy.
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
696 */
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
697 static PyObject *list_copy(PyObject *list)
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
698 {
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
699 Py_ssize_t len = PyList_GET_SIZE(list);
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
700 PyObject *newlist = PyList_New(len);
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
701 Py_ssize_t i;
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
702
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
703 if (newlist == NULL)
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
704 return NULL;
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
705
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
706 for (i = 0; i < len; i++) {
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
707 PyObject *obj = PyList_GET_ITEM(list, i);
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
708 Py_INCREF(obj);
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
709 PyList_SET_ITEM(newlist, i, obj);
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
710 }
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
711
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
712 return newlist;
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
713 }
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
714
16786
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
715 static PyObject *index_headrevs(indexObject *self)
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
716 {
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
717 Py_ssize_t i, len, addlen;
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
718 char *nothead = NULL;
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
719 PyObject *heads;
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
720
16787
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
721 if (self->headrevs)
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
722 return list_copy(self->headrevs);
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
723
16786
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
724 len = index_length(self) - 1;
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
725 heads = PyList_New(0);
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
726 if (heads == NULL)
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
727 goto bail;
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
728 if (len == 0) {
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
729 PyObject *nullid = PyInt_FromLong(-1);
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
730 if (nullid == NULL || PyList_Append(heads, nullid) == -1) {
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
731 Py_XDECREF(nullid);
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
732 goto bail;
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
733 }
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
734 goto done;
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
735 }
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
736
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
737 nothead = calloc(len, 1);
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
738 if (nothead == NULL)
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
739 goto bail;
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
740
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
741 for (i = 0; i < self->raw_length; i++) {
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
742 const char *data = index_deref(self, i);
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
743 int parent_1 = getbe32(data + 24);
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
744 int parent_2 = getbe32(data + 28);
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
745 if (parent_1 >= 0)
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
746 nothead[parent_1] = 1;
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
747 if (parent_2 >= 0)
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
748 nothead[parent_2] = 1;
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
749 }
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
750
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
751 addlen = self->added ? PyList_GET_SIZE(self->added) : 0;
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
752
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
753 for (i = 0; i < addlen; i++) {
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
754 PyObject *rev = PyList_GET_ITEM(self->added, i);
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
755 PyObject *p1 = PyTuple_GET_ITEM(rev, 5);
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
756 PyObject *p2 = PyTuple_GET_ITEM(rev, 6);
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
757 long parent_1, parent_2;
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
758
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
759 if (!PyInt_Check(p1) || !PyInt_Check(p2)) {
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
760 PyErr_SetString(PyExc_TypeError,
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
761 "revlog parents are invalid");
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
762 goto bail;
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
763 }
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
764 parent_1 = PyInt_AS_LONG(p1);
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
765 parent_2 = PyInt_AS_LONG(p2);
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
766 if (parent_1 >= 0)
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
767 nothead[parent_1] = 1;
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
768 if (parent_2 >= 0)
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
769 nothead[parent_2] = 1;
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
770 }
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
771
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
772 for (i = 0; i < len; i++) {
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
773 PyObject *head;
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
774
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
775 if (nothead[i])
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
776 continue;
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
777 head = PyInt_FromLong(i);
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
778 if (head == NULL || PyList_Append(heads, head) == -1) {
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
779 Py_XDECREF(head);
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
780 goto bail;
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
781 }
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
782 }
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
783
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
784 done:
16787
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
785 self->headrevs = heads;
16786
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
786 free(nothead);
16787
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
787 return list_copy(self->headrevs);
16786
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
788 bail:
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
789 Py_XDECREF(heads);
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
790 free(nothead);
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
791 return NULL;
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
792 }
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
793
16618
6bae941b58ad parsers: change the type of nt_level
Bryan O'Sullivan <bryano@fb.com>
parents: 16617
diff changeset
794 static inline int nt_level(const char *node, Py_ssize_t level)
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
795 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
796 int v = node[level>>1];
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
797 if (!(level & 1))
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
798 v >>= 4;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
799 return v & 0xf;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
800 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
801
16616
8f79aabd96f6 parsers: allow nt_find to signal an ambiguous match
Bryan O'Sullivan <bryano@fb.com>
parents: 16615
diff changeset
802 /*
8f79aabd96f6 parsers: allow nt_find to signal an ambiguous match
Bryan O'Sullivan <bryano@fb.com>
parents: 16615
diff changeset
803 * Return values:
8f79aabd96f6 parsers: allow nt_find to signal an ambiguous match
Bryan O'Sullivan <bryano@fb.com>
parents: 16615
diff changeset
804 *
8f79aabd96f6 parsers: allow nt_find to signal an ambiguous match
Bryan O'Sullivan <bryano@fb.com>
parents: 16615
diff changeset
805 * -4: match is ambiguous (multiple candidates)
8f79aabd96f6 parsers: allow nt_find to signal an ambiguous match
Bryan O'Sullivan <bryano@fb.com>
parents: 16615
diff changeset
806 * -2: not found
8f79aabd96f6 parsers: allow nt_find to signal an ambiguous match
Bryan O'Sullivan <bryano@fb.com>
parents: 16615
diff changeset
807 * rest: valid rev
8f79aabd96f6 parsers: allow nt_find to signal an ambiguous match
Bryan O'Sullivan <bryano@fb.com>
parents: 16615
diff changeset
808 */
16663
a955e05dd7a0 parsers: allow hex keys
Bryan O'Sullivan <bryano@fb.com>
parents: 16642
diff changeset
809 static int nt_find(indexObject *self, const char *node, Py_ssize_t nodelen,
a955e05dd7a0 parsers: allow hex keys
Bryan O'Sullivan <bryano@fb.com>
parents: 16642
diff changeset
810 int hex)
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
811 {
16663
a955e05dd7a0 parsers: allow hex keys
Bryan O'Sullivan <bryano@fb.com>
parents: 16642
diff changeset
812 int (*getnybble)(const char *, Py_ssize_t) = hex ? hexdigit : nt_level;
16641
e6dfbc5df76f parsers: use the correct maximum radix tree depth
Bryan O'Sullivan <bryano@fb.com>
parents: 16604
diff changeset
813 int level, maxlevel, off;
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
814
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
815 if (nodelen == 20 && node[0] == '\0' && memcmp(node, nullid, 20) == 0)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
816 return -1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
817
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
818 if (self->nt == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
819 return -2;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
820
16663
a955e05dd7a0 parsers: allow hex keys
Bryan O'Sullivan <bryano@fb.com>
parents: 16642
diff changeset
821 if (hex)
16665
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
822 maxlevel = nodelen > 40 ? 40 : (int)nodelen;
16663
a955e05dd7a0 parsers: allow hex keys
Bryan O'Sullivan <bryano@fb.com>
parents: 16642
diff changeset
823 else
a955e05dd7a0 parsers: allow hex keys
Bryan O'Sullivan <bryano@fb.com>
parents: 16642
diff changeset
824 maxlevel = nodelen > 20 ? 40 : ((int)nodelen * 2);
16641
e6dfbc5df76f parsers: use the correct maximum radix tree depth
Bryan O'Sullivan <bryano@fb.com>
parents: 16604
diff changeset
825
e6dfbc5df76f parsers: use the correct maximum radix tree depth
Bryan O'Sullivan <bryano@fb.com>
parents: 16604
diff changeset
826 for (level = off = 0; level < maxlevel; level++) {
16663
a955e05dd7a0 parsers: allow hex keys
Bryan O'Sullivan <bryano@fb.com>
parents: 16642
diff changeset
827 int k = getnybble(node, level);
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
828 nodetree *n = &self->nt[off];
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
829 int v = n->children[k];
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
830
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
831 if (v < 0) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
832 const char *n;
16663
a955e05dd7a0 parsers: allow hex keys
Bryan O'Sullivan <bryano@fb.com>
parents: 16642
diff changeset
833 Py_ssize_t i;
a955e05dd7a0 parsers: allow hex keys
Bryan O'Sullivan <bryano@fb.com>
parents: 16642
diff changeset
834
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
835 v = -v - 1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
836 n = index_node(self, v);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
837 if (n == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
838 return -2;
16663
a955e05dd7a0 parsers: allow hex keys
Bryan O'Sullivan <bryano@fb.com>
parents: 16642
diff changeset
839 for (i = level; i < maxlevel; i++)
a955e05dd7a0 parsers: allow hex keys
Bryan O'Sullivan <bryano@fb.com>
parents: 16642
diff changeset
840 if (getnybble(node, i) != nt_level(n, i))
a955e05dd7a0 parsers: allow hex keys
Bryan O'Sullivan <bryano@fb.com>
parents: 16642
diff changeset
841 return -2;
a955e05dd7a0 parsers: allow hex keys
Bryan O'Sullivan <bryano@fb.com>
parents: 16642
diff changeset
842 return v;
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
843 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
844 if (v == 0)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
845 return -2;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
846 off = v;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
847 }
16616
8f79aabd96f6 parsers: allow nt_find to signal an ambiguous match
Bryan O'Sullivan <bryano@fb.com>
parents: 16615
diff changeset
848 /* multiple matches against an ambiguous prefix */
8f79aabd96f6 parsers: allow nt_find to signal an ambiguous match
Bryan O'Sullivan <bryano@fb.com>
parents: 16615
diff changeset
849 return -4;
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
850 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
851
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
852 static int nt_new(indexObject *self)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
853 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
854 if (self->ntlength == self->ntcapacity) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
855 self->ntcapacity *= 2;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
856 self->nt = realloc(self->nt,
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
857 self->ntcapacity * sizeof(nodetree));
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
858 if (self->nt == NULL) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
859 PyErr_SetString(PyExc_MemoryError, "out of memory");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
860 return -1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
861 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
862 memset(&self->nt[self->ntlength], 0,
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
863 sizeof(nodetree) * (self->ntcapacity - self->ntlength));
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
864 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
865 return self->ntlength++;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
866 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
867
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
868 static int nt_insert(indexObject *self, const char *node, int rev)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
869 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
870 int level = 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
871 int off = 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
872
16641
e6dfbc5df76f parsers: use the correct maximum radix tree depth
Bryan O'Sullivan <bryano@fb.com>
parents: 16604
diff changeset
873 while (level < 40) {
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
874 int k = nt_level(node, level);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
875 nodetree *n;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
876 int v;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
877
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
878 n = &self->nt[off];
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
879 v = n->children[k];
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
880
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
881 if (v == 0) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
882 n->children[k] = -rev - 1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
883 return 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
884 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
885 if (v < 0) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
886 const char *oldnode = index_node(self, -v - 1);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
887 int noff;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
888
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
889 if (!oldnode || !memcmp(oldnode, node, 20)) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
890 n->children[k] = -rev - 1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
891 return 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
892 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
893 noff = nt_new(self);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
894 if (noff == -1)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
895 return -1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
896 /* self->nt may have been changed by realloc */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
897 self->nt[off].children[k] = noff;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
898 off = noff;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
899 n = &self->nt[off];
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
900 n->children[nt_level(oldnode, ++level)] = v;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
901 if (level > self->ntdepth)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
902 self->ntdepth = level;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
903 self->ntsplits += 1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
904 } else {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
905 level += 1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
906 off = v;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
907 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
908 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
909
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
910 return -1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
911 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
912
16615
96fa9dd1db38 parsers: factor out radix tree initialization
Bryan O'Sullivan <bryano@fb.com>
parents: 16614
diff changeset
913 static int nt_init(indexObject *self)
96fa9dd1db38 parsers: factor out radix tree initialization
Bryan O'Sullivan <bryano@fb.com>
parents: 16614
diff changeset
914 {
96fa9dd1db38 parsers: factor out radix tree initialization
Bryan O'Sullivan <bryano@fb.com>
parents: 16614
diff changeset
915 if (self->nt == NULL) {
96fa9dd1db38 parsers: factor out radix tree initialization
Bryan O'Sullivan <bryano@fb.com>
parents: 16614
diff changeset
916 self->ntcapacity = self->raw_length < 4
96fa9dd1db38 parsers: factor out radix tree initialization
Bryan O'Sullivan <bryano@fb.com>
parents: 16614
diff changeset
917 ? 4 : self->raw_length / 2;
96fa9dd1db38 parsers: factor out radix tree initialization
Bryan O'Sullivan <bryano@fb.com>
parents: 16614
diff changeset
918 self->nt = calloc(self->ntcapacity, sizeof(nodetree));
96fa9dd1db38 parsers: factor out radix tree initialization
Bryan O'Sullivan <bryano@fb.com>
parents: 16614
diff changeset
919 if (self->nt == NULL) {
96fa9dd1db38 parsers: factor out radix tree initialization
Bryan O'Sullivan <bryano@fb.com>
parents: 16614
diff changeset
920 PyErr_NoMemory();
96fa9dd1db38 parsers: factor out radix tree initialization
Bryan O'Sullivan <bryano@fb.com>
parents: 16614
diff changeset
921 return -1;
96fa9dd1db38 parsers: factor out radix tree initialization
Bryan O'Sullivan <bryano@fb.com>
parents: 16614
diff changeset
922 }
96fa9dd1db38 parsers: factor out radix tree initialization
Bryan O'Sullivan <bryano@fb.com>
parents: 16614
diff changeset
923 self->ntlength = 1;
96fa9dd1db38 parsers: factor out radix tree initialization
Bryan O'Sullivan <bryano@fb.com>
parents: 16614
diff changeset
924 self->ntrev = (int)index_length(self) - 1;
96fa9dd1db38 parsers: factor out radix tree initialization
Bryan O'Sullivan <bryano@fb.com>
parents: 16614
diff changeset
925 self->ntlookups = 1;
96fa9dd1db38 parsers: factor out radix tree initialization
Bryan O'Sullivan <bryano@fb.com>
parents: 16614
diff changeset
926 self->ntmisses = 0;
16664
5bc6edf71b39 parsers: ensure that nullid is always present in the radix tree
Bryan O'Sullivan <bryano@fb.com>
parents: 16663
diff changeset
927 if (nt_insert(self, nullid, INT_MAX) == -1)
5bc6edf71b39 parsers: ensure that nullid is always present in the radix tree
Bryan O'Sullivan <bryano@fb.com>
parents: 16663
diff changeset
928 return -1;
16615
96fa9dd1db38 parsers: factor out radix tree initialization
Bryan O'Sullivan <bryano@fb.com>
parents: 16614
diff changeset
929 }
96fa9dd1db38 parsers: factor out radix tree initialization
Bryan O'Sullivan <bryano@fb.com>
parents: 16614
diff changeset
930 return 0;
96fa9dd1db38 parsers: factor out radix tree initialization
Bryan O'Sullivan <bryano@fb.com>
parents: 16614
diff changeset
931 }
96fa9dd1db38 parsers: factor out radix tree initialization
Bryan O'Sullivan <bryano@fb.com>
parents: 16614
diff changeset
932
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
933 /*
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
934 * Return values:
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
935 *
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
936 * -3: error (exception set)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
937 * -2: not found (no exception set)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
938 * rest: valid rev
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
939 */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
940 static int index_find_node(indexObject *self,
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
941 const char *node, Py_ssize_t nodelen)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
942 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
943 int rev;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
944
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
945 self->ntlookups++;
16663
a955e05dd7a0 parsers: allow hex keys
Bryan O'Sullivan <bryano@fb.com>
parents: 16642
diff changeset
946 rev = nt_find(self, node, nodelen, 0);
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
947 if (rev >= -1)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
948 return rev;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
949
16615
96fa9dd1db38 parsers: factor out radix tree initialization
Bryan O'Sullivan <bryano@fb.com>
parents: 16614
diff changeset
950 if (nt_init(self) == -1)
96fa9dd1db38 parsers: factor out radix tree initialization
Bryan O'Sullivan <bryano@fb.com>
parents: 16614
diff changeset
951 return -3;
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
952
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
953 /*
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
954 * For the first handful of lookups, we scan the entire index,
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
955 * and cache only the matching nodes. This optimizes for cases
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
956 * like "hg tip", where only a few nodes are accessed.
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
957 *
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
958 * After that, we cache every node we visit, using a single
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
959 * scan amortized over multiple lookups. This gives the best
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
960 * bulk performance, e.g. for "hg log".
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
961 */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
962 if (self->ntmisses++ < 4) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
963 for (rev = self->ntrev - 1; rev >= 0; rev--) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
964 const char *n = index_node(self, rev);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
965 if (n == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
966 return -2;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
967 if (memcmp(node, n, nodelen > 20 ? 20 : nodelen) == 0) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
968 if (nt_insert(self, n, rev) == -1)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
969 return -3;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
970 break;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
971 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
972 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
973 } else {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
974 for (rev = self->ntrev - 1; rev >= 0; rev--) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
975 const char *n = index_node(self, rev);
16614
1d800eb9ba52 parsers: update ntrev when we stop scanning
Bryan O'Sullivan <bryano@fb.com>
parents: 16597
diff changeset
976 if (n == NULL) {
1d800eb9ba52 parsers: update ntrev when we stop scanning
Bryan O'Sullivan <bryano@fb.com>
parents: 16597
diff changeset
977 self->ntrev = rev + 1;
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
978 return -2;
16614
1d800eb9ba52 parsers: update ntrev when we stop scanning
Bryan O'Sullivan <bryano@fb.com>
parents: 16597
diff changeset
979 }
1d800eb9ba52 parsers: update ntrev when we stop scanning
Bryan O'Sullivan <bryano@fb.com>
parents: 16597
diff changeset
980 if (nt_insert(self, n, rev) == -1) {
1d800eb9ba52 parsers: update ntrev when we stop scanning
Bryan O'Sullivan <bryano@fb.com>
parents: 16597
diff changeset
981 self->ntrev = rev + 1;
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
982 return -3;
16614
1d800eb9ba52 parsers: update ntrev when we stop scanning
Bryan O'Sullivan <bryano@fb.com>
parents: 16597
diff changeset
983 }
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
984 if (memcmp(node, n, nodelen > 20 ? 20 : nodelen) == 0) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
985 break;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
986 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
987 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
988 self->ntrev = rev;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
989 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
990
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
991 if (rev >= 0)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
992 return rev;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
993 return -2;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
994 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
995
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
996 static PyObject *raise_revlog_error(void)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
997 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
998 static PyObject *errclass;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
999 PyObject *mod = NULL, *errobj;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1000
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1001 if (errclass == NULL) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1002 PyObject *dict;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1003
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1004 mod = PyImport_ImportModule("mercurial.error");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1005 if (mod == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1006 goto classfail;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1007
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1008 dict = PyModule_GetDict(mod);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1009 if (dict == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1010 goto classfail;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1011
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1012 errclass = PyDict_GetItemString(dict, "RevlogError");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1013 if (errclass == NULL) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1014 PyErr_SetString(PyExc_SystemError,
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1015 "could not find RevlogError");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1016 goto classfail;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1017 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1018 Py_INCREF(errclass);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1019 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1020
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1021 errobj = PyObject_CallFunction(errclass, NULL);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1022 if (errobj == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1023 return NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1024 PyErr_SetObject(errclass, errobj);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1025 return errobj;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1026
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1027 classfail:
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1028 Py_XDECREF(mod);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1029 return NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1030 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1031
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1032 static PyObject *index_getitem(indexObject *self, PyObject *value)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1033 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1034 char *node;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1035 Py_ssize_t nodelen;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1036 int rev;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1037
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1038 if (PyInt_Check(value))
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1039 return index_get(self, PyInt_AS_LONG(value));
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1040
16679
2950d186a927 parsers: strictly check for 20-byte hashes where they're required
Bryan O'Sullivan <bryano@fb.com>
parents: 16641
diff changeset
1041 if (node_check(value, &node, &nodelen) == -1)
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1042 return NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1043 rev = index_find_node(self, node, nodelen);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1044 if (rev >= -1)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1045 return PyInt_FromLong(rev);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1046 if (rev == -2)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1047 raise_revlog_error();
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1048 return NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1049 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1050
16665
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1051 static int nt_partialmatch(indexObject *self, const char *node,
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1052 Py_ssize_t nodelen)
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1053 {
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1054 int rev;
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1055
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1056 if (nt_init(self) == -1)
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1057 return -3;
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1058
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1059 if (self->ntrev > 0) {
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1060 /* ensure that the radix tree is fully populated */
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1061 for (rev = self->ntrev - 1; rev >= 0; rev--) {
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1062 const char *n = index_node(self, rev);
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1063 if (n == NULL)
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1064 return -2;
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1065 if (nt_insert(self, n, rev) == -1)
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1066 return -3;
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1067 }
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1068 self->ntrev = rev;
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1069 }
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1070
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1071 return nt_find(self, node, nodelen, 1);
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1072 }
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1073
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1074 static PyObject *index_partialmatch(indexObject *self, PyObject *args)
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1075 {
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1076 const char *fullnode;
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1077 int nodelen;
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1078 char *node;
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1079 int rev, i;
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1080
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1081 if (!PyArg_ParseTuple(args, "s#", &node, &nodelen))
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1082 return NULL;
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1083
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1084 if (nodelen < 4) {
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1085 PyErr_SetString(PyExc_ValueError, "key too short");
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1086 return NULL;
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1087 }
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1088
17353
bde1185f406c revlog: don't try to partialmatch strings those length > 40
sorcerer
parents: 17165
diff changeset
1089 if (nodelen > 40) {
bde1185f406c revlog: don't try to partialmatch strings those length > 40
sorcerer
parents: 17165
diff changeset
1090 PyErr_SetString(PyExc_ValueError, "key too long");
bde1185f406c revlog: don't try to partialmatch strings those length > 40
sorcerer
parents: 17165
diff changeset
1091 return NULL;
bde1185f406c revlog: don't try to partialmatch strings those length > 40
sorcerer
parents: 17165
diff changeset
1092 }
16665
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1093
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1094 for (i = 0; i < nodelen; i++)
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1095 hexdigit(node, i);
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1096 if (PyErr_Occurred()) {
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1097 /* input contains non-hex characters */
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1098 PyErr_Clear();
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1099 Py_RETURN_NONE;
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1100 }
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1101
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1102 rev = nt_partialmatch(self, node, nodelen);
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1103
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1104 switch (rev) {
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1105 case -4:
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1106 raise_revlog_error();
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1107 case -3:
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1108 return NULL;
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1109 case -2:
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1110 Py_RETURN_NONE;
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1111 case -1:
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1112 return PyString_FromStringAndSize(nullid, 20);
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1113 }
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1114
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1115 fullnode = index_node(self, rev);
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1116 if (fullnode == NULL) {
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1117 PyErr_Format(PyExc_IndexError,
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1118 "could not access rev %d", rev);
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1119 return NULL;
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1120 }
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1121 return PyString_FromStringAndSize(fullnode, 20);
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1122 }
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1123
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1124 static PyObject *index_m_get(indexObject *self, PyObject *args)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1125 {
16679
2950d186a927 parsers: strictly check for 20-byte hashes where they're required
Bryan O'Sullivan <bryano@fb.com>
parents: 16641
diff changeset
1126 Py_ssize_t nodelen;
2950d186a927 parsers: strictly check for 20-byte hashes where they're required
Bryan O'Sullivan <bryano@fb.com>
parents: 16641
diff changeset
1127 PyObject *val;
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1128 char *node;
16679
2950d186a927 parsers: strictly check for 20-byte hashes where they're required
Bryan O'Sullivan <bryano@fb.com>
parents: 16641
diff changeset
1129 int rev;
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1130
16679
2950d186a927 parsers: strictly check for 20-byte hashes where they're required
Bryan O'Sullivan <bryano@fb.com>
parents: 16641
diff changeset
1131 if (!PyArg_ParseTuple(args, "O", &val))
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1132 return NULL;
16679
2950d186a927 parsers: strictly check for 20-byte hashes where they're required
Bryan O'Sullivan <bryano@fb.com>
parents: 16641
diff changeset
1133 if (node_check(val, &node, &nodelen) == -1)
2950d186a927 parsers: strictly check for 20-byte hashes where they're required
Bryan O'Sullivan <bryano@fb.com>
parents: 16641
diff changeset
1134 return NULL;
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1135 rev = index_find_node(self, node, nodelen);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1136 if (rev == -3)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1137 return NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1138 if (rev == -2)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1139 Py_RETURN_NONE;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1140 return PyInt_FromLong(rev);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1141 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1142
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1143 static int index_contains(indexObject *self, PyObject *value)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1144 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1145 char *node;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1146 Py_ssize_t nodelen;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1147
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1148 if (PyInt_Check(value)) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1149 long rev = PyInt_AS_LONG(value);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1150 return rev >= -1 && rev < index_length(self);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1151 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1152
16679
2950d186a927 parsers: strictly check for 20-byte hashes where they're required
Bryan O'Sullivan <bryano@fb.com>
parents: 16641
diff changeset
1153 if (node_check(value, &node, &nodelen) == -1)
2950d186a927 parsers: strictly check for 20-byte hashes where they're required
Bryan O'Sullivan <bryano@fb.com>
parents: 16641
diff changeset
1154 return -1;
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1155
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1156 switch (index_find_node(self, node, nodelen)) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1157 case -3:
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1158 return -1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1159 case -2:
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1160 return 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1161 default:
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1162 return 1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1163 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1164 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1165
18988
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1166 static inline void index_get_parents(indexObject *self, int rev, int *ps)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1167 {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1168 if (rev >= self->length - 1) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1169 PyObject *tuple = PyList_GET_ITEM(self->added,
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1170 rev - self->length + 1);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1171 ps[0] = (int)PyInt_AS_LONG(PyTuple_GET_ITEM(tuple, 5));
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1172 ps[1] = (int)PyInt_AS_LONG(PyTuple_GET_ITEM(tuple, 6));
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1173 } else {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1174 const char *data = index_deref(self, rev);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1175 ps[0] = getbe32(data + 24);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1176 ps[1] = getbe32(data + 28);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1177 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1178 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1179
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1180 typedef uint64_t bitmask;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1181
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1182 /*
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1183 * Given a disjoint set of revs, return all candidates for the
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1184 * greatest common ancestor. In revset notation, this is the set
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1185 * "heads(::a and ::b and ...)"
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1186 */
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1187 static PyObject *find_gca_candidates(indexObject *self, const int *revs,
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1188 int revcount)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1189 {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1190 const bitmask allseen = (1ull << revcount) - 1;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1191 const bitmask poison = 1ull << revcount;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1192 PyObject *gca = PyList_New(0);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1193 int i, v, interesting, left;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1194 int maxrev = -1;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1195 bitmask *seen;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1196
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1197 for (i = 0; i < revcount; i++) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1198 if (revs[i] > maxrev)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1199 maxrev = revs[i];
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1200 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1201
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1202 seen = calloc(sizeof(*seen), maxrev + 1);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1203 if (seen == NULL)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1204 return PyErr_NoMemory();
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1205
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1206 for (i = 0; i < revcount; i++)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1207 seen[revs[i]] = 1ull << i;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1208
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1209 interesting = left = revcount;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1210
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1211 for (v = maxrev; v >= 0 && interesting; v--) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1212 long sv = seen[v];
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1213 int parents[2];
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1214
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1215 if (!sv)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1216 continue;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1217
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1218 if (sv < poison) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1219 interesting -= 1;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1220 if (sv == allseen) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1221 PyObject *obj = PyInt_FromLong(v);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1222 if (obj == NULL)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1223 goto bail;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1224 if (PyList_Append(gca, obj) == -1) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1225 Py_DECREF(obj);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1226 goto bail;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1227 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1228 sv |= poison;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1229 for (i = 0; i < revcount; i++) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1230 if (revs[i] == v) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1231 if (--left <= 1)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1232 goto done;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1233 break;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1234 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1235 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1236 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1237 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1238 index_get_parents(self, v, parents);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1239
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1240 for (i = 0; i < 2; i++) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1241 int p = parents[i];
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1242 if (p == -1)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1243 continue;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1244 const long sp = seen[p];
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1245 if (sv < poison) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1246 if (sp == 0) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1247 seen[p] = sv;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1248 interesting++;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1249 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1250 else if (sp != sv)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1251 seen[p] |= sv;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1252 } else {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1253 if (sp && sp < poison)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1254 interesting--;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1255 seen[p] = sv;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1256 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1257 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1258 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1259
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1260 done:
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1261 free(seen);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1262 return gca;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1263 bail:
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1264 free(seen);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1265 Py_XDECREF(gca);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1266 return NULL;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1267 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1268
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1269 /*
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1270 * Given a disjoint set of revs, return the subset with the longest
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1271 * path to the root.
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1272 */
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1273 static PyObject *find_deepest(indexObject *self, PyObject *revs)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1274 {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1275 const Py_ssize_t revcount = PyList_GET_SIZE(revs);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1276 static const Py_ssize_t capacity = 24;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1277 int *depth, *interesting = NULL;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1278 int i, j, v, ninteresting;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1279 PyObject *dict = NULL, *keys;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1280 long *seen = NULL;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1281 int maxrev = -1;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1282 long final;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1283
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1284 if (revcount > capacity) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1285 PyErr_Format(PyExc_OverflowError,
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1286 "bitset size (%ld) > capacity (%ld)",
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1287 revcount, capacity);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1288 return NULL;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1289 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1290
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1291 for (i = 0; i < revcount; i++) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1292 int n = (int)PyInt_AsLong(PyList_GET_ITEM(revs, i));
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1293 if (n > maxrev)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1294 maxrev = n;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1295 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1296
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1297 depth = calloc(sizeof(*depth), maxrev + 1);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1298 if (depth == NULL)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1299 return PyErr_NoMemory();
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1300
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1301 seen = calloc(sizeof(*seen), maxrev + 1);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1302 if (seen == NULL) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1303 PyErr_NoMemory();
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1304 goto bail;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1305 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1306
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1307 interesting = calloc(sizeof(*interesting), 2 << revcount);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1308 if (interesting == NULL) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1309 PyErr_NoMemory();
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1310 goto bail;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1311 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1312
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1313 for (i = 0; i < revcount; i++) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1314 int n = (int)PyInt_AsLong(PyList_GET_ITEM(revs, i));
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1315 long b = 1l << i;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1316 depth[n] = 1;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1317 seen[n] = b;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1318 interesting[b] = 1;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1319 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1320
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1321 ninteresting = (int)revcount;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1322
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1323 for (v = maxrev; v >= 0 && ninteresting > 1; v--) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1324 int dv = depth[v];
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1325 int parents[2];
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1326 long sv;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1327
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1328 if (dv == 0)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1329 continue;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1330
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1331 sv = seen[v];
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1332 index_get_parents(self, v, parents);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1333
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1334 for (i = 0; i < 2; i++) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1335 int p = parents[i];
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1336 long nsp, sp;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1337 int dp;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1338
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1339 if (p == -1)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1340 continue;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1341
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1342 dp = depth[p];
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1343 nsp = sp = seen[p];
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1344 if (dp <= dv) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1345 depth[p] = dv + 1;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1346 if (sp != sv) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1347 interesting[sv] += 1;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1348 nsp = seen[p] = sv;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1349 if (sp) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1350 interesting[sp] -= 1;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1351 if (interesting[sp] == 0)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1352 ninteresting -= 1;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1353 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1354 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1355 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1356 else if (dv == dp - 1) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1357 nsp = sp | sv;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1358 if (nsp == sp)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1359 continue;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1360 seen[p] = nsp;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1361 interesting[nsp] += 1;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1362 interesting[sp] -= 1;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1363 if (interesting[sp] == 0)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1364 ninteresting -= 1;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1365 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1366 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1367 interesting[sv] -= 1;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1368 if (interesting[sv] == 0)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1369 ninteresting -= 1;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1370 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1371
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1372 final = 0;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1373 j = ninteresting;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1374 for (i = 0; i < (int)(2 << revcount) && j > 0; i++) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1375 if (interesting[i] == 0)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1376 continue;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1377 final |= i;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1378 j -= 1;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1379 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1380 if (final == 0)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1381 return PyList_New(0);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1382
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1383 dict = PyDict_New();
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1384 if (dict == NULL)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1385 goto bail;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1386
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1387 j = ninteresting;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1388 for (i = 0; i < revcount && j > 0; i++) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1389 PyObject *key;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1390
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1391 if ((final & (1 << i)) == 0)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1392 continue;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1393
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1394 key = PyList_GET_ITEM(revs, i);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1395 Py_INCREF(key);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1396 Py_INCREF(Py_None);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1397 if (PyDict_SetItem(dict, key, Py_None) == -1) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1398 Py_DECREF(key);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1399 Py_DECREF(Py_None);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1400 goto bail;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1401 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1402 j -= 1;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1403 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1404
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1405 keys = PyDict_Keys(dict);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1406
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1407 free(depth);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1408 free(seen);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1409 free(interesting);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1410 Py_DECREF(dict);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1411
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1412 return keys;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1413 bail:
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1414 free(depth);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1415 free(seen);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1416 free(interesting);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1417 Py_XDECREF(dict);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1418
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1419 return NULL;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1420 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1421
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1422 /*
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1423 * Given a (possibly overlapping) set of revs, return the greatest
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1424 * common ancestors: those with the longest path to the root.
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1425 */
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1426 static PyObject *index_ancestors(indexObject *self, PyObject *args)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1427 {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1428 PyObject *ret = NULL, *gca = NULL;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1429 Py_ssize_t argcount, i, len;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1430 bitmask repeat = 0;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1431 int revcount = 0;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1432 int *revs;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1433
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1434 argcount = PySequence_Length(args);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1435 revs = malloc(argcount * sizeof(*revs));
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1436 if (argcount > 0 && revs == NULL)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1437 return PyErr_NoMemory();
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1438 len = index_length(self) - 1;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1439
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1440 for (i = 0; i < argcount; i++) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1441 static const int capacity = 24;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1442 PyObject *obj = PySequence_GetItem(args, i);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1443 bitmask x;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1444 long val;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1445
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1446 if (!PyInt_Check(obj)) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1447 PyErr_SetString(PyExc_TypeError,
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1448 "arguments must all be ints");
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1449 goto bail;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1450 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1451 val = PyInt_AsLong(obj);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1452 if (val == -1) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1453 ret = PyList_New(0);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1454 goto done;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1455 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1456 if (val < 0 || val >= len) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1457 PyErr_SetString(PyExc_IndexError,
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1458 "index out of range");
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1459 goto bail;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1460 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1461 /* this cheesy bloom filter lets us avoid some more
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1462 * expensive duplicate checks in the common set-is-disjoint
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1463 * case */
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1464 x = 1ull << (val & 0x3f);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1465 if (repeat & x) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1466 int k;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1467 for (k = 0; k < revcount; k++) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1468 if (val == revs[k])
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1469 goto duplicate;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1470 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1471 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1472 else repeat |= x;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1473 if (revcount >= capacity) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1474 PyErr_Format(PyExc_OverflowError,
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1475 "bitset size (%d) > capacity (%d)",
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1476 revcount, capacity);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1477 goto bail;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1478 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1479 revs[revcount++] = (int)val;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1480 duplicate:;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1481 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1482
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1483 if (revcount == 0) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1484 ret = PyList_New(0);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1485 goto done;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1486 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1487 if (revcount == 1) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1488 PyObject *obj;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1489 ret = PyList_New(1);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1490 if (ret == NULL)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1491 goto bail;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1492 obj = PyInt_FromLong(revs[0]);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1493 if (obj == NULL)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1494 goto bail;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1495 PyList_SET_ITEM(ret, 0, obj);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1496 goto done;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1497 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1498
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1499 gca = find_gca_candidates(self, revs, revcount);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1500 if (gca == NULL)
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1501 goto bail;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1502
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1503 if (PyList_GET_SIZE(gca) <= 1) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1504 ret = gca;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1505 Py_INCREF(gca);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1506 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1507 else if (PyList_GET_SIZE(gca) == 1) {
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1508 ret = PyList_GET_ITEM(gca, 0);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1509 Py_INCREF(ret);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1510 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1511 else ret = find_deepest(self, gca);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1512
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1513 done:
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1514 free(revs);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1515 Py_XDECREF(gca);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1516
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1517 return ret;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1518
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1519 bail:
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1520 free(revs);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1521 Py_XDECREF(gca);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1522 Py_XDECREF(ret);
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1523 return NULL;
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1524 }
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1525
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1526 /*
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1527 * Invalidate any trie entries introduced by added revs.
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1528 */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1529 static void nt_invalidate_added(indexObject *self, Py_ssize_t start)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1530 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1531 Py_ssize_t i, len = PyList_GET_SIZE(self->added);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1532
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1533 for (i = start; i < len; i++) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1534 PyObject *tuple = PyList_GET_ITEM(self->added, i);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1535 PyObject *node = PyTuple_GET_ITEM(tuple, 7);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1536
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1537 nt_insert(self, PyString_AS_STRING(node), -1);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1538 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1539
16732
277e2acb7e5c parsers: use Py_CLEAR where appropriate
Bryan O'Sullivan <bryano@fb.com>
parents: 16699
diff changeset
1540 if (start == 0)
277e2acb7e5c parsers: use Py_CLEAR where appropriate
Bryan O'Sullivan <bryano@fb.com>
parents: 16699
diff changeset
1541 Py_CLEAR(self->added);
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1542 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1543
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1544 /*
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1545 * Delete a numeric range of revs, which must be at the end of the
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1546 * range, but exclude the sentinel nullid entry.
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1547 */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1548 static int index_slice_del(indexObject *self, PyObject *item)
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1549 {
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1550 Py_ssize_t start, stop, step, slicelength;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1551 Py_ssize_t length = index_length(self);
16784
12a852c7c763 parsers: reduce raw_length when truncating
Bryan O'Sullivan <bryano@fb.com>
parents: 16732
diff changeset
1552 int ret = 0;
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1553
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1554 if (PySlice_GetIndicesEx((PySliceObject*)item, length,
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1555 &start, &stop, &step, &slicelength) < 0)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1556 return -1;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1557
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1558 if (slicelength <= 0)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1559 return 0;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1560
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1561 if ((step < 0 && start < stop) || (step > 0 && start > stop))
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1562 stop = start;
7108
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
1563
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1564 if (step < 0) {
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1565 stop = start + 1;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1566 start = stop + step*(slicelength - 1) - 1;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1567 step = -step;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1568 }
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1569
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1570 if (step != 1) {
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1571 PyErr_SetString(PyExc_ValueError,
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1572 "revlog index delete requires step size of 1");
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1573 return -1;
7108
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
1574 }
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1575
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1576 if (stop != length - 1) {
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1577 PyErr_SetString(PyExc_IndexError,
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1578 "revlog index deletion indices are invalid");
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1579 return -1;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1580 }
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1581
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1582 if (start < self->length - 1) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1583 if (self->nt) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1584 Py_ssize_t i;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1585
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1586 for (i = start + 1; i < self->length - 1; i++) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1587 const char *node = index_node(self, i);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1588
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1589 if (node)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1590 nt_insert(self, node, -1);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1591 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1592 if (self->added)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1593 nt_invalidate_added(self, 0);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1594 if (self->ntrev > start)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1595 self->ntrev = (int)start;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1596 }
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1597 self->length = start + 1;
18504
d1d5fdcc2d46 parsers: fix memleak of revlog cache entries on strip
Yuya Nishihara <yuya@tcha.org>
parents: 18430
diff changeset
1598 if (start < self->raw_length) {
d1d5fdcc2d46 parsers: fix memleak of revlog cache entries on strip
Yuya Nishihara <yuya@tcha.org>
parents: 18430
diff changeset
1599 if (self->cache) {
d1d5fdcc2d46 parsers: fix memleak of revlog cache entries on strip
Yuya Nishihara <yuya@tcha.org>
parents: 18430
diff changeset
1600 Py_ssize_t i;
d1d5fdcc2d46 parsers: fix memleak of revlog cache entries on strip
Yuya Nishihara <yuya@tcha.org>
parents: 18430
diff changeset
1601 for (i = start; i < self->raw_length; i++)
d1d5fdcc2d46 parsers: fix memleak of revlog cache entries on strip
Yuya Nishihara <yuya@tcha.org>
parents: 18430
diff changeset
1602 Py_CLEAR(self->cache[i]);
d1d5fdcc2d46 parsers: fix memleak of revlog cache entries on strip
Yuya Nishihara <yuya@tcha.org>
parents: 18430
diff changeset
1603 }
16784
12a852c7c763 parsers: reduce raw_length when truncating
Bryan O'Sullivan <bryano@fb.com>
parents: 16732
diff changeset
1604 self->raw_length = start;
18504
d1d5fdcc2d46 parsers: fix memleak of revlog cache entries on strip
Yuya Nishihara <yuya@tcha.org>
parents: 18430
diff changeset
1605 }
16784
12a852c7c763 parsers: reduce raw_length when truncating
Bryan O'Sullivan <bryano@fb.com>
parents: 16732
diff changeset
1606 goto done;
7108
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
1607 }
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
1608
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1609 if (self->nt) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1610 nt_invalidate_added(self, start - self->length + 1);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1611 if (self->ntrev > start)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1612 self->ntrev = (int)start;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1613 }
16784
12a852c7c763 parsers: reduce raw_length when truncating
Bryan O'Sullivan <bryano@fb.com>
parents: 16732
diff changeset
1614 if (self->added)
12a852c7c763 parsers: reduce raw_length when truncating
Bryan O'Sullivan <bryano@fb.com>
parents: 16732
diff changeset
1615 ret = PyList_SetSlice(self->added, start - self->length + 1,
12a852c7c763 parsers: reduce raw_length when truncating
Bryan O'Sullivan <bryano@fb.com>
parents: 16732
diff changeset
1616 PyList_GET_SIZE(self->added), NULL);
12a852c7c763 parsers: reduce raw_length when truncating
Bryan O'Sullivan <bryano@fb.com>
parents: 16732
diff changeset
1617 done:
16787
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
1618 Py_CLEAR(self->headrevs);
16784
12a852c7c763 parsers: reduce raw_length when truncating
Bryan O'Sullivan <bryano@fb.com>
parents: 16732
diff changeset
1619 return ret;
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1620 }
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1621
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1622 /*
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1623 * Supported ops:
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1624 *
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1625 * slice deletion
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1626 * string assignment (extend node->rev mapping)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1627 * string deletion (shrink node->rev mapping)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1628 */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1629 static int index_assign_subscript(indexObject *self, PyObject *item,
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1630 PyObject *value)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1631 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1632 char *node;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1633 Py_ssize_t nodelen;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1634 long rev;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1635
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1636 if (PySlice_Check(item) && value == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1637 return index_slice_del(self, item);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1638
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1639 if (node_check(item, &node, &nodelen) == -1)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1640 return -1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1641
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1642 if (value == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1643 return self->nt ? nt_insert(self, node, -1) : 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1644 rev = PyInt_AsLong(value);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1645 if (rev > INT_MAX || rev < 0) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1646 if (!PyErr_Occurred())
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1647 PyErr_SetString(PyExc_ValueError, "rev out of range");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1648 return -1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1649 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1650 return nt_insert(self, node, (int)rev);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1651 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1652
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1653 /*
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1654 * Find all RevlogNG entries in an index that has inline data. Update
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1655 * the optional "offsets" table with those entries.
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1656 */
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1657 static long inline_scan(indexObject *self, const char **offsets)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1658 {
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1659 const char *data = PyString_AS_STRING(self->data);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1660 const char *end = data + PyString_GET_SIZE(self->data);
16863
bbedef66c6f3 parsers: replace magic number 64 with symbolic constant
Bryan O'Sullivan <bryano@fb.com>
parents: 16787
diff changeset
1661 long incr = v1_hdrsize;
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1662 Py_ssize_t len = 0;
13254
5ef5eb1f3515 revlog: only build the nodemap on demand
Matt Mackall <mpm@selenic.com>
parents: 11361
diff changeset
1663
16863
bbedef66c6f3 parsers: replace magic number 64 with symbolic constant
Bryan O'Sullivan <bryano@fb.com>
parents: 16787
diff changeset
1664 while (data + v1_hdrsize <= end) {
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1665 uint32_t comp_len;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1666 const char *old_data;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1667 /* 3rd element of header is length of compressed inline data */
16437
d126a0d16856 util.h: replace ntohl/htonl with get/putbe32
Matt Mackall <mpm@selenic.com>
parents: 16414
diff changeset
1668 comp_len = getbe32(data + 8);
16863
bbedef66c6f3 parsers: replace magic number 64 with symbolic constant
Bryan O'Sullivan <bryano@fb.com>
parents: 16787
diff changeset
1669 incr = v1_hdrsize + comp_len;
bbedef66c6f3 parsers: replace magic number 64 with symbolic constant
Bryan O'Sullivan <bryano@fb.com>
parents: 16787
diff changeset
1670 if (incr < v1_hdrsize)
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1671 break;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1672 if (offsets)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1673 offsets[len] = data;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1674 len++;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1675 old_data = data;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1676 data += incr;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1677 if (data <= old_data)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1678 break;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1679 }
13254
5ef5eb1f3515 revlog: only build the nodemap on demand
Matt Mackall <mpm@selenic.com>
parents: 11361
diff changeset
1680
16863
bbedef66c6f3 parsers: replace magic number 64 with symbolic constant
Bryan O'Sullivan <bryano@fb.com>
parents: 16787
diff changeset
1681 if (data != end && data + v1_hdrsize != end) {
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1682 if (!PyErr_Occurred())
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1683 PyErr_SetString(PyExc_ValueError, "corrupt index file");
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1684 return -1;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1685 }
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1686
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1687 return len;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1688 }
13254
5ef5eb1f3515 revlog: only build the nodemap on demand
Matt Mackall <mpm@selenic.com>
parents: 11361
diff changeset
1689
16572
8d44b5a2974f parsers: fix refcount leak, simplify init of index (issue3417)
Bryan O'Sullivan <bryano@fb.com>
parents: 16437
diff changeset
1690 static int index_init(indexObject *self, PyObject *args)
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1691 {
16572
8d44b5a2974f parsers: fix refcount leak, simplify init of index (issue3417)
Bryan O'Sullivan <bryano@fb.com>
parents: 16437
diff changeset
1692 PyObject *data_obj, *inlined_obj;
8d44b5a2974f parsers: fix refcount leak, simplify init of index (issue3417)
Bryan O'Sullivan <bryano@fb.com>
parents: 16437
diff changeset
1693 Py_ssize_t size;
8d44b5a2974f parsers: fix refcount leak, simplify init of index (issue3417)
Bryan O'Sullivan <bryano@fb.com>
parents: 16437
diff changeset
1694
8d44b5a2974f parsers: fix refcount leak, simplify init of index (issue3417)
Bryan O'Sullivan <bryano@fb.com>
parents: 16437
diff changeset
1695 if (!PyArg_ParseTuple(args, "OO", &data_obj, &inlined_obj))
8d44b5a2974f parsers: fix refcount leak, simplify init of index (issue3417)
Bryan O'Sullivan <bryano@fb.com>
parents: 16437
diff changeset
1696 return -1;
8d44b5a2974f parsers: fix refcount leak, simplify init of index (issue3417)
Bryan O'Sullivan <bryano@fb.com>
parents: 16437
diff changeset
1697 if (!PyString_Check(data_obj)) {
8d44b5a2974f parsers: fix refcount leak, simplify init of index (issue3417)
Bryan O'Sullivan <bryano@fb.com>
parents: 16437
diff changeset
1698 PyErr_SetString(PyExc_TypeError, "data is not a string");
8d44b5a2974f parsers: fix refcount leak, simplify init of index (issue3417)
Bryan O'Sullivan <bryano@fb.com>
parents: 16437
diff changeset
1699 return -1;
8d44b5a2974f parsers: fix refcount leak, simplify init of index (issue3417)
Bryan O'Sullivan <bryano@fb.com>
parents: 16437
diff changeset
1700 }
8d44b5a2974f parsers: fix refcount leak, simplify init of index (issue3417)
Bryan O'Sullivan <bryano@fb.com>
parents: 16437
diff changeset
1701 size = PyString_GET_SIZE(data_obj);
8d44b5a2974f parsers: fix refcount leak, simplify init of index (issue3417)
Bryan O'Sullivan <bryano@fb.com>
parents: 16437
diff changeset
1702
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1703 self->inlined = inlined_obj && PyObject_IsTrue(inlined_obj);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1704 self->data = data_obj;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1705 self->cache = NULL;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1706
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1707 self->added = NULL;
16787
bda96ce993f9 parsers: cache the result of index_headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16786
diff changeset
1708 self->headrevs = NULL;
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1709 self->offsets = NULL;
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1710 self->nt = NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1711 self->ntlength = self->ntcapacity = 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1712 self->ntdepth = self->ntsplits = 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1713 self->ntlookups = self->ntmisses = 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1714 self->ntrev = -1;
16597
b767382a8675 parsers: fix refcount bug on corrupt index
Matt Mackall <mpm@selenic.com>
parents: 16572
diff changeset
1715 Py_INCREF(self->data);
7108
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
1716
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1717 if (self->inlined) {
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1718 long len = inline_scan(self, NULL);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1719 if (len == -1)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1720 goto bail;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1721 self->raw_length = len;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1722 self->length = len + 1;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1723 } else {
16863
bbedef66c6f3 parsers: replace magic number 64 with symbolic constant
Bryan O'Sullivan <bryano@fb.com>
parents: 16787
diff changeset
1724 if (size % v1_hdrsize) {
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1725 PyErr_SetString(PyExc_ValueError, "corrupt index file");
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1726 goto bail;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1727 }
16863
bbedef66c6f3 parsers: replace magic number 64 with symbolic constant
Bryan O'Sullivan <bryano@fb.com>
parents: 16787
diff changeset
1728 self->raw_length = size / v1_hdrsize;
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1729 self->length = self->raw_length + 1;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1730 }
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1731
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1732 return 0;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1733 bail:
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1734 return -1;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1735 }
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1736
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1737 static PyObject *index_nodemap(indexObject *self)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1738 {
16572
8d44b5a2974f parsers: fix refcount leak, simplify init of index (issue3417)
Bryan O'Sullivan <bryano@fb.com>
parents: 16437
diff changeset
1739 Py_INCREF(self);
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1740 return (PyObject *)self;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1741 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1742
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1743 static void index_dealloc(indexObject *self)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1744 {
16370
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
1745 _index_clearcaches(self);
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1746 Py_DECREF(self->data);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1747 Py_XDECREF(self->added);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1748 PyObject_Del(self);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1749 }
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1750
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1751 static PySequenceMethods index_sequence_methods = {
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1752 (lenfunc)index_length, /* sq_length */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1753 0, /* sq_concat */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1754 0, /* sq_repeat */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1755 (ssizeargfunc)index_get, /* sq_item */
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1756 0, /* sq_slice */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1757 0, /* sq_ass_item */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1758 0, /* sq_ass_slice */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1759 (objobjproc)index_contains, /* sq_contains */
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1760 };
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1761
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1762 static PyMappingMethods index_mapping_methods = {
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1763 (lenfunc)index_length, /* mp_length */
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1764 (binaryfunc)index_getitem, /* mp_subscript */
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1765 (objobjargproc)index_assign_subscript, /* mp_ass_subscript */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1766 };
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1767
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1768 static PyMethodDef index_methods[] = {
18988
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1769 {"ancestors", (PyCFunction)index_ancestors, METH_VARARGS,
5bae936764bb parsers: a C implementation of the new ancestors algorithm
Bryan O'Sullivan <bryano@fb.com>
parents: 18900
diff changeset
1770 "return the gca set of the given revs"},
16370
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
1771 {"clearcaches", (PyCFunction)index_clearcaches, METH_NOARGS,
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index
Bryan O'Sullivan <bryano@fb.com>
parents: 16363
diff changeset
1772 "clear the index caches"},
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1773 {"get", (PyCFunction)index_m_get, METH_VARARGS,
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1774 "get an index entry"},
16786
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
1775 {"headrevs", (PyCFunction)index_headrevs, METH_NOARGS,
2631cd5dd244 revlog: switch to a C version of headrevs
Bryan O'Sullivan <bryano@fb.com>
parents: 16784
diff changeset
1776 "get head revisions"},
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1777 {"insert", (PyCFunction)index_insert, METH_VARARGS,
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1778 "insert an index entry"},
16665
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1779 {"partialmatch", (PyCFunction)index_partialmatch, METH_VARARGS,
e410be860393 revlog: speed up prefix matching against nodes
Bryan O'Sullivan <bryano@fb.com>
parents: 16664
diff changeset
1780 "match a potentially ambiguous node ID"},
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1781 {"stats", (PyCFunction)index_stats, METH_NOARGS,
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1782 "stats for the index"},
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1783 {NULL} /* Sentinel */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1784 };
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1785
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1786 static PyGetSetDef index_getset[] = {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1787 {"nodemap", (getter)index_nodemap, NULL, "nodemap", NULL},
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1788 {NULL} /* Sentinel */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1789 };
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1790
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1791 static PyTypeObject indexType = {
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1792 PyObject_HEAD_INIT(NULL)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1793 0, /* ob_size */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1794 "parsers.index", /* tp_name */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1795 sizeof(indexObject), /* tp_basicsize */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1796 0, /* tp_itemsize */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1797 (destructor)index_dealloc, /* tp_dealloc */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1798 0, /* tp_print */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1799 0, /* tp_getattr */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1800 0, /* tp_setattr */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1801 0, /* tp_compare */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1802 0, /* tp_repr */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1803 0, /* tp_as_number */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1804 &index_sequence_methods, /* tp_as_sequence */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1805 &index_mapping_methods, /* tp_as_mapping */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1806 0, /* tp_hash */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1807 0, /* tp_call */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1808 0, /* tp_str */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1809 0, /* tp_getattro */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1810 0, /* tp_setattro */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1811 0, /* tp_as_buffer */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1812 Py_TPFLAGS_DEFAULT, /* tp_flags */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1813 "revlog index", /* tp_doc */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1814 0, /* tp_traverse */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1815 0, /* tp_clear */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1816 0, /* tp_richcompare */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1817 0, /* tp_weaklistoffset */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1818 0, /* tp_iter */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1819 0, /* tp_iternext */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1820 index_methods, /* tp_methods */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1821 0, /* tp_members */
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1822 index_getset, /* tp_getset */
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1823 0, /* tp_base */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1824 0, /* tp_dict */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1825 0, /* tp_descr_get */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1826 0, /* tp_descr_set */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1827 0, /* tp_dictoffset */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1828 (initproc)index_init, /* tp_init */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1829 0, /* tp_alloc */
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1830 };
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1831
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1832 /*
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1833 * returns a tuple of the form (index, index, cache) with elements as
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1834 * follows:
7108
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
1835 *
16414
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping
Bryan O'Sullivan <bryano@fb.com>
parents: 16393
diff changeset
1836 * index: an index object that lazily parses RevlogNG records
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1837 * cache: if data is inlined, a tuple (index_file_content, 0), else None
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1838 *
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1839 * added complications are for backwards compatibility
7108
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
1840 */
13254
5ef5eb1f3515 revlog: only build the nodemap on demand
Matt Mackall <mpm@selenic.com>
parents: 11361
diff changeset
1841 static PyObject *parse_index2(PyObject *self, PyObject *args)
7108
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
1842 {
16572
8d44b5a2974f parsers: fix refcount leak, simplify init of index (issue3417)
Bryan O'Sullivan <bryano@fb.com>
parents: 16437
diff changeset
1843 PyObject *tuple = NULL, *cache = NULL;
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1844 indexObject *idx;
16572
8d44b5a2974f parsers: fix refcount leak, simplify init of index (issue3417)
Bryan O'Sullivan <bryano@fb.com>
parents: 16437
diff changeset
1845 int ret;
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1846
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1847 idx = PyObject_New(indexObject, &indexType);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1848 if (idx == NULL)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1849 goto bail;
7108
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
1850
16572
8d44b5a2974f parsers: fix refcount leak, simplify init of index (issue3417)
Bryan O'Sullivan <bryano@fb.com>
parents: 16437
diff changeset
1851 ret = index_init(idx, args);
8d44b5a2974f parsers: fix refcount leak, simplify init of index (issue3417)
Bryan O'Sullivan <bryano@fb.com>
parents: 16437
diff changeset
1852 if (ret == -1)
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1853 goto bail;
7108
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
1854
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1855 if (idx->inlined) {
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1856 cache = Py_BuildValue("iO", 0, idx->data);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1857 if (cache == NULL)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1858 goto bail;
7108
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
1859 } else {
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
1860 cache = Py_None;
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1861 Py_INCREF(cache);
7108
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
1862 }
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
1863
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1864 tuple = Py_BuildValue("NN", idx, cache);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1865 if (!tuple)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1866 goto bail;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1867 return tuple;
7108
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
1868
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1869 bail:
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1870 Py_XDECREF(idx);
7108
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
1871 Py_XDECREF(cache);
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1872 Py_XDECREF(tuple);
7108
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
1873 return NULL;
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
1874 }
1ca878d7b849 C implementation of revlog index parsing
Bernhard Leiner <bleiner@gmail.com>
parents: 7093
diff changeset
1875
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
1876 static char parsers_doc[] = "Efficient content parsing.";
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
1877
17606
318fb32b980e pathencode: new C module with fast encodedir() function
Adrian Buehlmann <adrian@cadifra.com>
parents: 17356
diff changeset
1878 PyObject *encodedir(PyObject *self, PyObject *args);
17616
9535a0dc41f2 store: implement fncache basic path encoding in C
Bryan O'Sullivan <bryano@fb.com>
parents: 17606
diff changeset
1879 PyObject *pathencode(PyObject *self, PyObject *args);
18430
0459c6555f69 store: implement lowerencode in C
Bryan O'Sullivan <bryano@fb.com>
parents: 17616
diff changeset
1880 PyObject *lowerencode(PyObject *self, PyObject *args);
17606
318fb32b980e pathencode: new C module with fast encodedir() function
Adrian Buehlmann <adrian@cadifra.com>
parents: 17356
diff changeset
1881
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
1882 static PyMethodDef methods[] = {
16955
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
1883 {"pack_dirstate", pack_dirstate, METH_VARARGS, "pack a dirstate\n"},
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
1884 {"parse_manifest", parse_manifest, METH_VARARGS, "parse a manifest\n"},
7093
16bafcebd3d1 dirstate: C parsing extension
Matt Mackall <mpm@selenic.com>
parents: 7092
diff changeset
1885 {"parse_dirstate", parse_dirstate, METH_VARARGS, "parse a dirstate\n"},
13254
5ef5eb1f3515 revlog: only build the nodemap on demand
Matt Mackall <mpm@selenic.com>
parents: 11361
diff changeset
1886 {"parse_index2", parse_index2, METH_VARARGS, "parse a revlog index\n"},
17606
318fb32b980e pathencode: new C module with fast encodedir() function
Adrian Buehlmann <adrian@cadifra.com>
parents: 17356
diff changeset
1887 {"encodedir", encodedir, METH_VARARGS, "encodedir a path\n"},
17616
9535a0dc41f2 store: implement fncache basic path encoding in C
Bryan O'Sullivan <bryano@fb.com>
parents: 17606
diff changeset
1888 {"pathencode", pathencode, METH_VARARGS, "fncache-encode a path\n"},
18430
0459c6555f69 store: implement lowerencode in C
Bryan O'Sullivan <bryano@fb.com>
parents: 17616
diff changeset
1889 {"lowerencode", lowerencode, METH_VARARGS, "lower-encode a path\n"},
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
1890 {NULL, NULL}
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
1891 };
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
1892
18900
02ee846b246a scmutil: rewrite dirs in C, use if available
Bryan O'Sullivan <bryano@fb.com>
parents: 18567
diff changeset
1893 void dirs_module_init(PyObject *mod);
02ee846b246a scmutil: rewrite dirs in C, use if available
Bryan O'Sullivan <bryano@fb.com>
parents: 18567
diff changeset
1894
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1895 static void module_init(PyObject *mod)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1896 {
18900
02ee846b246a scmutil: rewrite dirs in C, use if available
Bryan O'Sullivan <bryano@fb.com>
parents: 18567
diff changeset
1897 dirs_module_init(mod);
02ee846b246a scmutil: rewrite dirs in C, use if available
Bryan O'Sullivan <bryano@fb.com>
parents: 18567
diff changeset
1898
16604
48e42f984074 parsers: statically initializing tp_new to PyType_GenericNew is not portable
Adrian Buehlmann <adrian@cadifra.com>
parents: 16597
diff changeset
1899 indexType.tp_new = PyType_GenericNew;
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1900 if (PyType_Ready(&indexType) < 0)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1901 return;
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1902 Py_INCREF(&indexType);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1903
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1904 PyModule_AddObject(mod, "index", (PyObject *)&indexType);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1905
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1906 nullentry = Py_BuildValue("iiiiiiis#", 0, 0, 0,
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1907 -1, -1, -1, -1, nullid, 20);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1908 if (nullentry)
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1909 PyObject_GC_UnTrack(nullentry);
16955
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
1910
92e1c64ba0d4 parsers: add a C function to pack the dirstate
Bryan O'Sullivan <bryano@fb.com>
parents: 16863
diff changeset
1911 dirstate_unset = Py_BuildValue("ciii", 'n', 0, -1, -1);
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1912 }
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1913
11361
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
1914 #ifdef IS_PY3K
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
1915 static struct PyModuleDef parsers_module = {
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
1916 PyModuleDef_HEAD_INIT,
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
1917 "parsers",
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
1918 parsers_doc,
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
1919 -1,
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
1920 methods
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
1921 };
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
1922
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
1923 PyMODINIT_FUNC PyInit_parsers(void)
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
1924 {
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1925 PyObject *mod = PyModule_Create(&parsers_module);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1926 module_init(mod);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1927 return mod;
11361
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
1928 }
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
1929 #else
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
1930 PyMODINIT_FUNC initparsers(void)
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
1931 {
16363
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1932 PyObject *mod = Py_InitModule3("parsers", methods, parsers_doc);
2cdd7e63211b parsers: incrementally parse the revlog index in C
Bryan O'Sullivan <bryano@fb.com>
parents: 15033
diff changeset
1933 module_init(mod);
6389
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
1934 }
11361
3de3d670d2b6 parsers.c: Added support for py3k.
Renato Cunha <renatoc@gmail.com>
parents: 10449
diff changeset
1935 #endif