mercurial-scm/hg-stable: mercurial/parsers.c annotate

annotate mercurial/parsers.c @ 16414:e8d37b78acfb

parsers: use base-16 trie for faster node->rev mapping This greatly speeds up node->rev lookups, with results that are often user-perceptible: for instance, "hg --time log" of the node associated with rev 1000 on a linux-2.6 repo improves from 0.3 seconds to 0.03. I have not found any instances of slowdowns. The new perfnodelookup command in contrib/perf.py demonstrates the speedup more dramatically, since it performs no I/O. For a single lookup, the new code is about 40x faster. These changes also prepare the ground for the possibility of further improving the performance of prefix-based node lookups.

author	Bryan O'Sullivan <bryano@fb.com>
date	Thu, 12 Apr 2012 14:05:59 -0700
parents	ee163a9cf37c
children	d126a0d16856

rev	line source
6389 0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	1 /*
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	2 parsers.c - efficient content parsing
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	3
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	4 Copyright 2008 Matt Mackall <mpm@selenic.com> and others
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	5
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	6 This software may be used and distributed according to the terms of
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	7 the GNU General Public License, incorporated herein by reference.
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	8 */
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	9
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	10 #include <Python.h>
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	11 #include <ctype.h>
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	12 #include <string.h>
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	13
11361 3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	14 #include "util.h"
3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	15
6389 0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	16 static int hexdigit(char c)
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	17 {
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	18 if (c >= '0' && c <= '9')
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	19 return c - '0';
7092 fb3fc27617a2 parsers: speed up hex decoding for manifests Matt Mackall <mpm@selenic.com> parents: 7091 diff changeset	20 if (c >= 'a' && c <= 'f')
fb3fc27617a2 parsers: speed up hex decoding for manifests Matt Mackall <mpm@selenic.com> parents: 7091 diff changeset	21 return c - 'a' + 10;
6389 0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	22 if (c >= 'A' && c <= 'F')
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	23 return c - 'A' + 10;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	24
7092 fb3fc27617a2 parsers: speed up hex decoding for manifests Matt Mackall <mpm@selenic.com> parents: 7091 diff changeset	25 PyErr_SetString(PyExc_ValueError, "input contains non-hex character");
fb3fc27617a2 parsers: speed up hex decoding for manifests Matt Mackall <mpm@selenic.com> parents: 7091 diff changeset	26 return 0;
6389 0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	27 }
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	28
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	29 /*
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	30 * Turn a hex-encoded string into binary.
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	31 */
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	32 static PyObject unhexlify(const char str, int len)
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	33 {
7092 fb3fc27617a2 parsers: speed up hex decoding for manifests Matt Mackall <mpm@selenic.com> parents: 7091 diff changeset	34 PyObject *ret;
6395 3f0294536b24 fix const annotation warning Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 6389 diff changeset	35 const char *c;
3f0294536b24 fix const annotation warning Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 6389 diff changeset	36 char *d;
6389 0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	37
11361 3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	38 ret = PyBytes_FromStringAndSize(NULL, len / 2);
3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	39
6389 0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	40 if (!ret)
7092 fb3fc27617a2 parsers: speed up hex decoding for manifests Matt Mackall <mpm@selenic.com> parents: 7091 diff changeset	41 return NULL;
6389 0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	42
11361 3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	43 d = PyBytes_AsString(ret);
3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	44
6389 0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	45 for (c = str; c < str + len;) {
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	46 int hi = hexdigit(*c++);
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	47 int lo = hexdigit(*c++);
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	48 *d++ = (hi << 4) \| lo;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	49 }
7091 12b35ae03365 parsers: clean up whitespace Matt Mackall <mpm@selenic.com> parents: 6395 diff changeset	50
6389 0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	51 return ret;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	52 }
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	53
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	54 /*
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	55 * This code assumes that a manifest is stitched together with newline
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	56 * ('\n') characters.
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	57 */
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	58 static PyObject parse_manifest(PyObject self, PyObject *args)
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	59 {
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	60 PyObject mfdict, fdict;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	61 char str, cur, start, zero;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	62 int len;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	63
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	64 if (!PyArg_ParseTuple(args, "O!O!s#:parse_manifest",
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	65 &PyDict_Type, &mfdict,
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	66 &PyDict_Type, &fdict,
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	67 &str, &len))
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	68 goto quit;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	69
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	70 for (start = cur = str, zero = NULL; cur < str + len; cur++) {
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	71 PyObject file = NULL, node = NULL;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	72 PyObject *flags = NULL;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	73 int nlen;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	74
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	75 if (!*cur) {
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	76 zero = cur;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	77 continue;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	78 }
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	79 else if (*cur != '\n')
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	80 continue;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	81
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	82 if (!zero) {
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	83 PyErr_SetString(PyExc_ValueError,
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	84 "manifest entry has no separator");
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	85 goto quit;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	86 }
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	87
11361 3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	88 file = PyBytes_FromStringAndSize(start, zero - start);
3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	89
6389 0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	90 if (!file)
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	91 goto bail;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	92
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	93 nlen = cur - zero - 1;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	94
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	95 node = unhexlify(zero + 1, nlen > 40 ? 40 : nlen);
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	96 if (!node)
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	97 goto bail;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	98
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	99 if (nlen > 40) {
11361 3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	100 flags = PyBytes_FromStringAndSize(zero + 41,
6389 0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	101 nlen - 40);
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	102 if (!flags)
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	103 goto bail;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	104
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	105 if (PyDict_SetItem(fdict, file, flags) == -1)
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	106 goto bail;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	107 }
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	108
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	109 if (PyDict_SetItem(mfdict, file, node) == -1)
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	110 goto bail;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	111
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	112 start = cur + 1;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	113 zero = NULL;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	114
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	115 Py_XDECREF(flags);
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	116 Py_XDECREF(node);
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	117 Py_XDECREF(file);
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	118 continue;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	119 bail:
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	120 Py_XDECREF(flags);
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	121 Py_XDECREF(node);
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	122 Py_XDECREF(file);
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	123 goto quit;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	124 }
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	125
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	126 if (len > 0 && *(cur - 1) != '\n') {
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	127 PyErr_SetString(PyExc_ValueError,
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	128 "manifest contains trailing garbage");
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	129 goto quit;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	130 }
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	131
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	132 Py_INCREF(Py_None);
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	133 return Py_None;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	134 quit:
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	135 return NULL;
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	136 }
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	137
7093 16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	138 static PyObject parse_dirstate(PyObject self, PyObject *args)
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	139 {
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	140 PyObject dmap, cmap, parents = NULL, ret = NULL;
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	141 PyObject fname = NULL, cname = NULL, *entry = NULL;
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	142 char str, cur, end, cpos;
7174 4da87407b845 parsers.c: fix integer overflows Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 7168 diff changeset	143 int state, mode, size, mtime;
4da87407b845 parsers.c: fix integer overflows Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 7168 diff changeset	144 unsigned int flen;
7093 16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	145 int len;
15033 2ef2d3a5cd2d parsers: avoid pointer aliasing Matt Mackall <mpm@selenic.com> parents: 13302 diff changeset	146 uint32_t decode[4]; /* for alignment */
7093 16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	147
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	148 if (!PyArg_ParseTuple(args, "O!O!s#:parse_dirstate",
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	149 &PyDict_Type, &dmap,
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	150 &PyDict_Type, &cmap,
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	151 &str, &len))
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	152 goto quit;
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	153
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	154 /* read parents */
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	155 if (len < 40)
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	156 goto quit;
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	157
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	158 parents = Py_BuildValue("s#s#", str, 20, str + 20, 20);
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	159 if (!parents)
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	160 goto quit;
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	161
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	162 /* read filenames */
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	163 cur = str + 40;
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	164 end = str + len;
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	165
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	166 while (cur < end - 17) {
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	167 /* unpack header */
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	168 state = *cur;
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	169 memcpy(decode, cur + 1, 16);
15033 2ef2d3a5cd2d parsers: avoid pointer aliasing Matt Mackall <mpm@selenic.com> parents: 13302 diff changeset	170 mode = ntohl(decode[0]);
2ef2d3a5cd2d parsers: avoid pointer aliasing Matt Mackall <mpm@selenic.com> parents: 13302 diff changeset	171 size = ntohl(decode[1]);
2ef2d3a5cd2d parsers: avoid pointer aliasing Matt Mackall <mpm@selenic.com> parents: 13302 diff changeset	172 mtime = ntohl(decode[2]);
2ef2d3a5cd2d parsers: avoid pointer aliasing Matt Mackall <mpm@selenic.com> parents: 13302 diff changeset	173 flen = ntohl(decode[3]);
7093 16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	174 cur += 17;
10449 7c8266c1d15a parsers: fix some signed comparison issues Matt Mackall <mpm@selenic.com> parents: 10282 diff changeset	175 if (cur + flen > end \|\| cur + flen < cur) {
7174 4da87407b845 parsers.c: fix integer overflows Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 7168 diff changeset	176 PyErr_SetString(PyExc_ValueError, "overflow in dirstate");
7093 16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	177 goto quit;
7174 4da87407b845 parsers.c: fix integer overflows Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 7168 diff changeset	178 }
7093 16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	179
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	180 entry = Py_BuildValue("ciii", state, mode, size, mtime);
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	181 if (!entry)
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	182 goto quit;
7175 5d8626b2c1db parsers.c: do not try to untrack after a failure Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 7174 diff changeset	183 PyObject_GC_UnTrack(entry); /* don't waste time with this */
7093 16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	184
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	185 cpos = memchr(cur, 0, flen);
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	186 if (cpos) {
11361 3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	187 fname = PyBytes_FromStringAndSize(cur, cpos - cur);
3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	188 cname = PyBytes_FromStringAndSize(cpos + 1,
7093 16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	189 flen - (cpos - cur) - 1);
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	190 if (!fname \|\| !cname \|\|
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	191 PyDict_SetItem(cmap, fname, cname) == -1 \|\|
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	192 PyDict_SetItem(dmap, fname, entry) == -1)
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	193 goto quit;
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	194 Py_DECREF(cname);
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	195 } else {
11361 3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	196 fname = PyBytes_FromStringAndSize(cur, flen);
7093 16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	197 if (!fname \|\|
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	198 PyDict_SetItem(dmap, fname, entry) == -1)
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	199 goto quit;
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	200 }
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	201 cur += flen;
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	202 Py_DECREF(fname);
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	203 Py_DECREF(entry);
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	204 fname = cname = entry = NULL;
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	205 }
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	206
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	207 ret = parents;
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	208 Py_INCREF(ret);
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	209 quit:
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	210 Py_XDECREF(fname);
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	211 Py_XDECREF(cname);
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	212 Py_XDECREF(entry);
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	213 Py_XDECREF(parents);
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	214 return ret;
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	215 }
16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	216
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	217 /*
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	218 * A base-16 trie for fast node->rev mapping.
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	219 *
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	220 * Positive value is index of the next node in the trie
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	221 * Negative value is a leaf: -(rev + 1)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	222 * Zero is empty
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	223 */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	224 typedef struct {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	225 int children[16];
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	226 } nodetree;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	227
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	228 /*
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	229 * This class has two behaviours.
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	230 *
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	231 * When used in a list-like way (with integer keys), we decode an
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	232 * entry in a RevlogNG index file on demand. Our last entry is a
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	233 * sentinel, always a nullid. We have limited support for
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	234 * integer-keyed insert and delete, only at elements right before the
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	235 * sentinel.
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	236 *
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	237 * With string keys, we lazily perform a reverse mapping from node to
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	238 * rev, using a base-16 trie.
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	239 */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	240 typedef struct {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	241 PyObject_HEAD
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	242 /* Type-specific fields go here. */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	243 PyObject data; / raw bytes of index */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	244 PyObject *cache; / cached tuples */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	245 const char *offsets; / populated on demand */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	246 Py_ssize_t raw_length; /* original number of elements */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	247 Py_ssize_t length; /* current number of elements */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	248 PyObject added; / populated on demand */
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	249 nodetree nt; / base-16 trie */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	250 int ntlength; /* # nodes in use */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	251 int ntcapacity; /* # nodes allocated */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	252 int ntdepth; /* maximum depth of tree */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	253 int ntsplits; /* # splits performed */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	254 int ntrev; /* last rev scanned */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	255 int ntlookups; /* # lookups */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	256 int ntmisses; /* # lookups that miss the cache */
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	257 int inlined;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	258 } indexObject;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	259
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	260 static Py_ssize_t index_length(const indexObject *self)
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	261 {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	262 if (self->added == NULL)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	263 return self->length;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	264 return self->length + PyList_GET_SIZE(self->added);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	265 }
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	266
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	267 static PyObject *nullentry;
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	268 static const char nullid[20];
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	269
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	270 static long inline_scan(indexObject self, const char *offsets);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	271
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	272 #if LONG_MAX == 0x7fffffffL
16393 ee163a9cf37c util.h: more Python 2.4 fixes Matt Mackall <mpm@selenic.com> parents: 16385 diff changeset	273 static char *tuple_format = "Kiiiiiis#";
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	274 #else
16393 ee163a9cf37c util.h: more Python 2.4 fixes Matt Mackall <mpm@selenic.com> parents: 16385 diff changeset	275 static char *tuple_format = "kiiiiiis#";
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	276 #endif
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	277
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	278 /*
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	279 * Return a pointer to the beginning of a RevlogNG record.
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	280 */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	281 static const char index_deref(indexObject self, Py_ssize_t pos)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	282 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	283 if (self->inlined && pos > 0) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	284 if (self->offsets == NULL) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	285 self->offsets = malloc(self->raw_length *
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	286 sizeof(*self->offsets));
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	287 if (self->offsets == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	288 return (const char *)PyErr_NoMemory();
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	289 inline_scan(self, self->offsets);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	290 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	291 return self->offsets[pos];
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	292 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	293
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	294 return PyString_AS_STRING(self->data) + pos * 64;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	295 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	296
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	297 /*
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	298 * RevlogNG format (all in big endian, data may be inlined):
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	299 * 6 bytes: offset
1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	300 * 2 bytes: flags
1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	301 * 4 bytes: compressed length
1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	302 * 4 bytes: uncompressed length
1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	303 * 4 bytes: base revision
1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	304 * 4 bytes: link revision
1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	305 * 4 bytes: parent 1 revision
1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	306 * 4 bytes: parent 2 revision
1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	307 * 32 bytes: nodeid (only 20 bytes used)
1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	308 */
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	309 static PyObject index_get(indexObject self, Py_ssize_t pos)
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	310 {
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	311 uint32_t decode[8]; /* to enforce alignment with inline data */
7154 7fdf7a0a41b7 index parser: fix refcounting in case of errors, refactor Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 7135 diff changeset	312 uint64_t offset_flags;
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	313 int comp_len, uncomp_len, base_rev, link_rev, parent_1, parent_2;
7154 7fdf7a0a41b7 index parser: fix refcounting in case of errors, refactor Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 7135 diff changeset	314 const char *c_node_id;
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	315 const char *data;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	316 Py_ssize_t length = index_length(self);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	317 PyObject *entry;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	318
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	319 if (pos < 0)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	320 pos += length;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	321
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	322 if (pos < 0 \|\| pos >= length) {
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	323 PyErr_SetString(PyExc_IndexError, "revlog index out of range");
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	324 return NULL;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	325 }
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	326
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	327 if (pos == length - 1) {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	328 Py_INCREF(nullentry);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	329 return nullentry;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	330 }
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	331
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	332 if (pos >= self->length - 1) {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	333 PyObject *obj;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	334 obj = PyList_GET_ITEM(self->added, pos - self->length + 1);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	335 Py_INCREF(obj);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	336 return obj;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	337 }
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	338
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	339 if (self->cache) {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	340 if (self->cache[pos]) {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	341 Py_INCREF(self->cache[pos]);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	342 return self->cache[pos];
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	343 }
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	344 } else {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	345 self->cache = calloc(self->raw_length, sizeof(PyObject *));
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	346 if (self->cache == NULL)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	347 return PyErr_NoMemory();
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	348 }
7190 aecea6934fdd Some additional space/tab cleanups Thomas Arendsen Hein <thomas@intevation.de> parents: 7186 diff changeset	349
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	350 data = index_deref(self, pos);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	351 if (data == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	352 return NULL;
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	353
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	354 memcpy(decode, data, 8 * sizeof(uint32_t));
7133 42db22108d85 revlog parser: use ntohl() instead of ntohll() (fix endianness issues) Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 7108 diff changeset	355
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	356 offset_flags = ntohl(decode[1]);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	357 if (pos == 0) /* mask out version number for the first entry */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	358 offset_flags &= 0xFFFF;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	359 else {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	360 uint32_t offset_high = ntohl(decode[0]);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	361 offset_flags \|= ((uint64_t)offset_high) << 32;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	362 }
7154 7fdf7a0a41b7 index parser: fix refcounting in case of errors, refactor Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 7135 diff changeset	363
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	364 comp_len = ntohl(decode[2]);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	365 uncomp_len = ntohl(decode[3]);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	366 base_rev = ntohl(decode[4]);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	367 link_rev = ntohl(decode[5]);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	368 parent_1 = ntohl(decode[6]);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	369 parent_2 = ntohl(decode[7]);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	370 c_node_id = data + 32;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	371
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	372 entry = Py_BuildValue(tuple_format, offset_flags, comp_len,
13254 5ef5eb1f3515 revlog: only build the nodemap on demand Matt Mackall <mpm@selenic.com> parents: 11361 diff changeset	373 uncomp_len, base_rev, link_rev,
5ef5eb1f3515 revlog: only build the nodemap on demand Matt Mackall <mpm@selenic.com> parents: 11361 diff changeset	374 parent_1, parent_2, c_node_id, 20);
5ef5eb1f3515 revlog: only build the nodemap on demand Matt Mackall <mpm@selenic.com> parents: 11361 diff changeset	375
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	376 if (entry)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	377 PyObject_GC_UnTrack(entry);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	378
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	379 self->cache[pos] = entry;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	380 Py_INCREF(entry);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	381
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	382 return entry;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	383 }
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	384
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	385 /*
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	386 * Return the 20-byte SHA of the node corresponding to the given rev.
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	387 */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	388 static const char index_node(indexObject self, Py_ssize_t pos)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	389 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	390 Py_ssize_t length = index_length(self);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	391 const char *data;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	392
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	393 if (pos == length - 1)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	394 return nullid;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	395
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	396 if (pos >= length)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	397 return NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	398
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	399 if (pos >= self->length - 1) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	400 PyObject tuple, str;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	401 tuple = PyList_GET_ITEM(self->added, pos - self->length + 1);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	402 str = PyTuple_GetItem(tuple, 7);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	403 return str ? PyString_AS_STRING(str) : NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	404 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	405
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	406 data = index_deref(self, pos);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	407 return data ? data + 32 : NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	408 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	409
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	410 static int nt_insert(indexObject self, const char node, int rev);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	411
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	412 static int node_check(PyObject obj, char node, Py_ssize_t nodelen)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	413 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	414 if (PyString_AsStringAndSize(obj, node, nodelen) == -1)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	415 return -1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	416 if (*nodelen == 20)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	417 return 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	418 PyErr_SetString(PyExc_ValueError, "20-byte hash required");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	419 return -1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	420 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	421
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	422 static PyObject index_insert(indexObject self, PyObject *args)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	423 {
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	424 PyObject *obj;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	425 char *node;
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	426 long offset;
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	427 Py_ssize_t len, nodelen;
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	428
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	429 if (!PyArg_ParseTuple(args, "lO", &offset, &obj))
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	430 return NULL;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	431
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	432 if (!PyTuple_Check(obj) \|\| PyTuple_GET_SIZE(obj) != 8) {
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	433 PyErr_SetString(PyExc_TypeError, "8-tuple required");
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	434 return NULL;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	435 }
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	436
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	437 if (node_check(PyTuple_GET_ITEM(obj, 7), &node, &nodelen) == -1)
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	438 return NULL;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	439
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	440 len = index_length(self);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	441
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	442 if (offset < 0)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	443 offset += len;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	444
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	445 if (offset != len - 1) {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	446 PyErr_SetString(PyExc_IndexError,
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	447 "insert only supported at index -1");
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	448 return NULL;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	449 }
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	450
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	451 if (offset > INT_MAX) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	452 PyErr_SetString(PyExc_ValueError,
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	453 "currently only 2**31 revs supported");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	454 return NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	455 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	456
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	457 if (self->added == NULL) {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	458 self->added = PyList_New(0);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	459 if (self->added == NULL)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	460 return NULL;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	461 }
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	462
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	463 if (PyList_Append(self->added, obj) == -1)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	464 return NULL;
13254 5ef5eb1f3515 revlog: only build the nodemap on demand Matt Mackall <mpm@selenic.com> parents: 11361 diff changeset	465
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	466 if (self->nt)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	467 nt_insert(self, node, (int)offset);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	468
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	469 Py_RETURN_NONE;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	470 }
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	471
16370 28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	472 static void _index_clearcaches(indexObject *self)
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	473 {
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	474 if (self->cache) {
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	475 Py_ssize_t i;
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	476
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	477 for (i = 0; i < self->raw_length; i++) {
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	478 Py_XDECREF(self->cache[i]);
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	479 self->cache[i] = NULL;
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	480 }
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	481 free(self->cache);
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	482 self->cache = NULL;
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	483 }
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	484 if (self->offsets) {
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	485 free(self->offsets);
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	486 self->offsets = NULL;
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	487 }
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	488 if (self->nt) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	489 free(self->nt);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	490 self->nt = NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	491 }
16370 28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	492 }
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	493
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	494 static PyObject index_clearcaches(indexObject self)
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	495 {
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	496 _index_clearcaches(self);
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	497 self->ntlength = self->ntcapacity = 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	498 self->ntdepth = self->ntsplits = 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	499 self->ntrev = -1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	500 self->ntlookups = self->ntmisses = 0;
16370 28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	501 Py_RETURN_NONE;
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	502 }
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	503
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	504 static PyObject index_stats(indexObject self)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	505 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	506 PyObject *obj = PyDict_New();
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	507
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	508 if (obj == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	509 return NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	510
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	511 #define istat(__n, __d) \
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	512 if (PyDict_SetItemString(obj, __d, PyInt_FromLong(self->__n)) == -1) \
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	513 goto bail;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	514
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	515 if (self->added) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	516 Py_ssize_t len = PyList_GET_SIZE(self->added);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	517 if (PyDict_SetItemString(obj, "index entries added",
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	518 PyInt_FromLong(len)) == -1)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	519 goto bail;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	520 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	521
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	522 if (self->raw_length != self->length - 1)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	523 istat(raw_length, "revs on disk");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	524 istat(length, "revs in memory");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	525 istat(ntcapacity, "node trie capacity");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	526 istat(ntdepth, "node trie depth");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	527 istat(ntlength, "node trie count");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	528 istat(ntlookups, "node trie lookups");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	529 istat(ntmisses, "node trie misses");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	530 istat(ntrev, "node trie last rev scanned");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	531 istat(ntsplits, "node trie splits");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	532
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	533 #undef istat
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	534
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	535 return obj;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	536
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	537 bail:
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	538 Py_XDECREF(obj);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	539 return NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	540 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	541
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	542 static inline int nt_level(const char *node, int level)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	543 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	544 int v = node[level>>1];
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	545 if (!(level & 1))
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	546 v >>= 4;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	547 return v & 0xf;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	548 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	549
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	550 static int nt_find(indexObject self, const char node, Py_ssize_t nodelen)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	551 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	552 int level, off;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	553
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	554 if (nodelen == 20 && node[0] == '\0' && memcmp(node, nullid, 20) == 0)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	555 return -1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	556
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	557 if (self->nt == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	558 return -2;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	559
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	560 for (level = off = 0; level < nodelen; level++) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	561 int k = nt_level(node, level);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	562 nodetree *n = &self->nt[off];
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	563 int v = n->children[k];
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	564
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	565 if (v < 0) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	566 const char *n;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	567 v = -v - 1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	568 n = index_node(self, v);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	569 if (n == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	570 return -2;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	571 return memcmp(node, n, nodelen > 20 ? 20 : nodelen)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	572 ? -2 : v;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	573 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	574 if (v == 0)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	575 return -2;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	576 off = v;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	577 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	578 return -2;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	579 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	580
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	581 static int nt_new(indexObject *self)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	582 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	583 if (self->ntlength == self->ntcapacity) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	584 self->ntcapacity *= 2;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	585 self->nt = realloc(self->nt,
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	586 self->ntcapacity * sizeof(nodetree));
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	587 if (self->nt == NULL) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	588 PyErr_SetString(PyExc_MemoryError, "out of memory");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	589 return -1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	590 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	591 memset(&self->nt[self->ntlength], 0,
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	592 sizeof(nodetree) * (self->ntcapacity - self->ntlength));
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	593 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	594 return self->ntlength++;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	595 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	596
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	597 static int nt_insert(indexObject self, const char node, int rev)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	598 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	599 int level = 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	600 int off = 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	601
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	602 while (level < 20) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	603 int k = nt_level(node, level);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	604 nodetree *n;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	605 int v;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	606
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	607 n = &self->nt[off];
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	608 v = n->children[k];
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	609
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	610 if (v == 0) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	611 n->children[k] = -rev - 1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	612 return 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	613 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	614 if (v < 0) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	615 const char *oldnode = index_node(self, -v - 1);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	616 int noff;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	617
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	618 if (!oldnode \|\| !memcmp(oldnode, node, 20)) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	619 n->children[k] = -rev - 1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	620 return 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	621 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	622 noff = nt_new(self);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	623 if (noff == -1)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	624 return -1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	625 /* self->nt may have been changed by realloc */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	626 self->nt[off].children[k] = noff;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	627 off = noff;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	628 n = &self->nt[off];
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	629 n->children[nt_level(oldnode, ++level)] = v;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	630 if (level > self->ntdepth)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	631 self->ntdepth = level;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	632 self->ntsplits += 1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	633 } else {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	634 level += 1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	635 off = v;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	636 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	637 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	638
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	639 return -1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	640 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	641
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	642 /*
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	643 * Return values:
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	644 *
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	645 * -3: error (exception set)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	646 * -2: not found (no exception set)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	647 * rest: valid rev
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	648 */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	649 static int index_find_node(indexObject *self,
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	650 const char *node, Py_ssize_t nodelen)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	651 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	652 int rev;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	653
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	654 self->ntlookups++;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	655 rev = nt_find(self, node, nodelen);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	656 if (rev >= -1)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	657 return rev;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	658
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	659 if (self->nt == NULL) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	660 self->ntcapacity = self->raw_length < 4
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	661 ? 4 : self->raw_length / 2;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	662 self->nt = calloc(self->ntcapacity, sizeof(nodetree));
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	663 if (self->nt == NULL) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	664 PyErr_SetString(PyExc_MemoryError, "out of memory");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	665 return -3;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	666 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	667 self->ntlength = 1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	668 self->ntrev = (int)index_length(self) - 1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	669 self->ntlookups = 1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	670 self->ntmisses = 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	671 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	672
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	673 /*
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	674 * For the first handful of lookups, we scan the entire index,
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	675 * and cache only the matching nodes. This optimizes for cases
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	676 * like "hg tip", where only a few nodes are accessed.
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	677 *
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	678 * After that, we cache every node we visit, using a single
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	679 * scan amortized over multiple lookups. This gives the best
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	680 * bulk performance, e.g. for "hg log".
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	681 */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	682 if (self->ntmisses++ < 4) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	683 for (rev = self->ntrev - 1; rev >= 0; rev--) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	684 const char *n = index_node(self, rev);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	685 if (n == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	686 return -2;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	687 if (memcmp(node, n, nodelen > 20 ? 20 : nodelen) == 0) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	688 if (nt_insert(self, n, rev) == -1)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	689 return -3;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	690 break;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	691 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	692 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	693 } else {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	694 for (rev = self->ntrev - 1; rev >= 0; rev--) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	695 const char *n = index_node(self, rev);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	696 if (n == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	697 return -2;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	698 if (nt_insert(self, n, rev) == -1)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	699 return -3;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	700 if (memcmp(node, n, nodelen > 20 ? 20 : nodelen) == 0) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	701 break;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	702 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	703 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	704 self->ntrev = rev;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	705 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	706
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	707 if (rev >= 0)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	708 return rev;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	709 return -2;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	710 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	711
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	712 static PyObject *raise_revlog_error(void)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	713 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	714 static PyObject *errclass;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	715 PyObject mod = NULL, errobj;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	716
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	717 if (errclass == NULL) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	718 PyObject *dict;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	719
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	720 mod = PyImport_ImportModule("mercurial.error");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	721 if (mod == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	722 goto classfail;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	723
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	724 dict = PyModule_GetDict(mod);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	725 if (dict == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	726 goto classfail;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	727
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	728 errclass = PyDict_GetItemString(dict, "RevlogError");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	729 if (errclass == NULL) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	730 PyErr_SetString(PyExc_SystemError,
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	731 "could not find RevlogError");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	732 goto classfail;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	733 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	734 Py_INCREF(errclass);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	735 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	736
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	737 errobj = PyObject_CallFunction(errclass, NULL);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	738 if (errobj == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	739 return NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	740 PyErr_SetObject(errclass, errobj);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	741 return errobj;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	742
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	743 classfail:
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	744 Py_XDECREF(mod);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	745 return NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	746 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	747
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	748 static PyObject index_getitem(indexObject self, PyObject *value)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	749 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	750 char *node;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	751 Py_ssize_t nodelen;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	752 int rev;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	753
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	754 if (PyInt_Check(value))
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	755 return index_get(self, PyInt_AS_LONG(value));
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	756
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	757 if (PyString_AsStringAndSize(value, &node, &nodelen) == -1)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	758 return NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	759 rev = index_find_node(self, node, nodelen);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	760 if (rev >= -1)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	761 return PyInt_FromLong(rev);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	762 if (rev == -2)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	763 raise_revlog_error();
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	764 return NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	765 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	766
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	767 static PyObject index_m_get(indexObject self, PyObject *args)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	768 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	769 char *node;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	770 int nodelen, rev;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	771
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	772 if (!PyArg_ParseTuple(args, "s#", &node, &nodelen))
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	773 return NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	774
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	775 rev = index_find_node(self, node, nodelen);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	776 if (rev == -3)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	777 return NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	778 if (rev == -2)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	779 Py_RETURN_NONE;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	780 return PyInt_FromLong(rev);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	781 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	782
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	783 static int index_contains(indexObject self, PyObject value)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	784 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	785 char *node;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	786 Py_ssize_t nodelen;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	787
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	788 if (PyInt_Check(value)) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	789 long rev = PyInt_AS_LONG(value);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	790 return rev >= -1 && rev < index_length(self);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	791 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	792
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	793 if (!PyString_Check(value))
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	794 return 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	795
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	796 node = PyString_AS_STRING(value);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	797 nodelen = PyString_GET_SIZE(value);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	798
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	799 switch (index_find_node(self, node, nodelen)) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	800 case -3:
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	801 return -1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	802 case -2:
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	803 return 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	804 default:
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	805 return 1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	806 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	807 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	808
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	809 /*
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	810 * Invalidate any trie entries introduced by added revs.
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	811 */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	812 static void nt_invalidate_added(indexObject *self, Py_ssize_t start)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	813 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	814 Py_ssize_t i, len = PyList_GET_SIZE(self->added);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	815
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	816 for (i = start; i < len; i++) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	817 PyObject *tuple = PyList_GET_ITEM(self->added, i);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	818 PyObject *node = PyTuple_GET_ITEM(tuple, 7);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	819
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	820 nt_insert(self, PyString_AS_STRING(node), -1);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	821 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	822
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	823 if (start == 0) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	824 Py_DECREF(self->added);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	825 self->added = NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	826 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	827 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	828
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	829 /*
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	830 * Delete a numeric range of revs, which must be at the end of the
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	831 * range, but exclude the sentinel nullid entry.
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	832 */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	833 static int index_slice_del(indexObject self, PyObject item)
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	834 {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	835 Py_ssize_t start, stop, step, slicelength;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	836 Py_ssize_t length = index_length(self);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	837
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	838 if (PySlice_GetIndicesEx((PySliceObject*)item, length,
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	839 &start, &stop, &step, &slicelength) < 0)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	840 return -1;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	841
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	842 if (slicelength <= 0)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	843 return 0;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	844
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	845 if ((step < 0 && start < stop) \|\| (step > 0 && start > stop))
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	846 stop = start;
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	847
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	848 if (step < 0) {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	849 stop = start + 1;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	850 start = stop + step*(slicelength - 1) - 1;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	851 step = -step;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	852 }
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	853
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	854 if (step != 1) {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	855 PyErr_SetString(PyExc_ValueError,
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	856 "revlog index delete requires step size of 1");
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	857 return -1;
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	858 }
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	859
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	860 if (stop != length - 1) {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	861 PyErr_SetString(PyExc_IndexError,
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	862 "revlog index deletion indices are invalid");
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	863 return -1;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	864 }
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	865
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	866 if (start < self->length - 1) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	867 if (self->nt) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	868 Py_ssize_t i;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	869
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	870 for (i = start + 1; i < self->length - 1; i++) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	871 const char *node = index_node(self, i);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	872
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	873 if (node)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	874 nt_insert(self, node, -1);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	875 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	876 if (self->added)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	877 nt_invalidate_added(self, 0);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	878 if (self->ntrev > start)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	879 self->ntrev = (int)start;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	880 }
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	881 self->length = start + 1;
7154 7fdf7a0a41b7 index parser: fix refcounting in case of errors, refactor Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 7135 diff changeset	882 return 0;
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	883 }
1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	884
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	885 if (self->nt) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	886 nt_invalidate_added(self, start - self->length + 1);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	887 if (self->ntrev > start)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	888 self->ntrev = (int)start;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	889 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	890 return self->added
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	891 ? PyList_SetSlice(self->added, start - self->length + 1,
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	892 PyList_GET_SIZE(self->added), NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	893 : 0;
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	894 }
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	895
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	896 /*
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	897 * Supported ops:
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	898 *
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	899 * slice deletion
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	900 * string assignment (extend node->rev mapping)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	901 * string deletion (shrink node->rev mapping)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	902 */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	903 static int index_assign_subscript(indexObject self, PyObject item,
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	904 PyObject *value)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	905 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	906 char *node;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	907 Py_ssize_t nodelen;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	908 long rev;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	909
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	910 if (PySlice_Check(item) && value == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	911 return index_slice_del(self, item);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	912
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	913 if (node_check(item, &node, &nodelen) == -1)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	914 return -1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	915
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	916 if (value == NULL)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	917 return self->nt ? nt_insert(self, node, -1) : 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	918 rev = PyInt_AsLong(value);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	919 if (rev > INT_MAX \|\| rev < 0) {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	920 if (!PyErr_Occurred())
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	921 PyErr_SetString(PyExc_ValueError, "rev out of range");
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	922 return -1;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	923 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	924 return nt_insert(self, node, (int)rev);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	925 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	926
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	927 /*
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	928 * Find all RevlogNG entries in an index that has inline data. Update
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	929 * the optional "offsets" table with those entries.
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	930 */
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	931 static long inline_scan(indexObject self, const char *offsets)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	932 {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	933 const char *data = PyString_AS_STRING(self->data);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	934 const char *end = data + PyString_GET_SIZE(self->data);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	935 const long hdrsize = 64;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	936 long incr = hdrsize;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	937 Py_ssize_t len = 0;
13254 5ef5eb1f3515 revlog: only build the nodemap on demand Matt Mackall <mpm@selenic.com> parents: 11361 diff changeset	938
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	939 while (data + hdrsize <= end) {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	940 uint32_t comp_len;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	941 const char *old_data;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	942 /* 3rd element of header is length of compressed inline data */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	943 memcpy(&comp_len, data + 8, sizeof(uint32_t));
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	944 incr = hdrsize + ntohl(comp_len);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	945 if (incr < hdrsize)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	946 break;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	947 if (offsets)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	948 offsets[len] = data;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	949 len++;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	950 old_data = data;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	951 data += incr;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	952 if (data <= old_data)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	953 break;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	954 }
13254 5ef5eb1f3515 revlog: only build the nodemap on demand Matt Mackall <mpm@selenic.com> parents: 11361 diff changeset	955
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	956 if (data != end && data + hdrsize != end) {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	957 if (!PyErr_Occurred())
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	958 PyErr_SetString(PyExc_ValueError, "corrupt index file");
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	959 return -1;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	960 }
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	961
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	962 return len;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	963 }
13254 5ef5eb1f3515 revlog: only build the nodemap on demand Matt Mackall <mpm@selenic.com> parents: 11361 diff changeset	964
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	965 static int index_real_init(indexObject self, const char data, int size,
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	966 PyObject inlined_obj, PyObject data_obj)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	967 {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	968 self->inlined = inlined_obj && PyObject_IsTrue(inlined_obj);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	969 self->data = data_obj;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	970 self->cache = NULL;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	971
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	972 self->added = NULL;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	973 self->offsets = NULL;
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	974 self->nt = NULL;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	975 self->ntlength = self->ntcapacity = 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	976 self->ntdepth = self->ntsplits = 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	977 self->ntlookups = self->ntmisses = 0;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	978 self->ntrev = -1;
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	979 Py_INCREF(self->data);
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	980
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	981 if (self->inlined) {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	982 long len = inline_scan(self, NULL);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	983 if (len == -1)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	984 goto bail;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	985 self->raw_length = len;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	986 self->length = len + 1;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	987 } else {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	988 if (size % 64) {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	989 PyErr_SetString(PyExc_ValueError, "corrupt index file");
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	990 goto bail;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	991 }
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	992 self->raw_length = size / 64;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	993 self->length = self->raw_length + 1;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	994 }
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	995
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	996 return 0;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	997 bail:
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	998 return -1;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	999 }
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1000
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1001 static int index_init(indexObject self, PyObject args, PyObject *kwds)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1002 {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1003 const char *data;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1004 int size;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1005 PyObject *inlined_obj;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1006
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1007 if (!PyArg_ParseTuple(args, "s#O", &data, &size, &inlined_obj))
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1008 return -1;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1009
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1010 return index_real_init(self, data, size, inlined_obj,
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1011 PyTuple_GET_ITEM(args, 0));
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	1012 }
1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	1013
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1014 static PyObject index_nodemap(indexObject self)
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1015 {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1016 return (PyObject *)self;
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1017 }
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1018
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1019 static void index_dealloc(indexObject *self)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1020 {
16370 28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	1021 _index_clearcaches(self);
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1022 Py_DECREF(self->data);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1023 Py_XDECREF(self->added);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1024 PyObject_Del(self);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1025 }
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1026
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1027 static PySequenceMethods index_sequence_methods = {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1028 (lenfunc)index_length, /* sq_length */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1029 0, /* sq_concat */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1030 0, /* sq_repeat */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1031 (ssizeargfunc)index_get, /* sq_item */
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1032 0, /* sq_slice */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1033 0, /* sq_ass_item */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1034 0, /* sq_ass_slice */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1035 (objobjproc)index_contains, /* sq_contains */
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1036 };
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1037
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1038 static PyMappingMethods index_mapping_methods = {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1039 (lenfunc)index_length, /* mp_length */
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1040 (binaryfunc)index_getitem, /* mp_subscript */
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1041 (objobjargproc)index_assign_subscript, /* mp_ass_subscript */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1042 };
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1043
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1044 static PyMethodDef index_methods[] = {
16370 28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	1045 {"clearcaches", (PyCFunction)index_clearcaches, METH_NOARGS,
28bb4daf070c parsers: fix a memleak, and add a clearcaches method to the index Bryan O'Sullivan <bryano@fb.com> parents: 16363 diff changeset	1046 "clear the index caches"},
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1047 {"get", (PyCFunction)index_m_get, METH_VARARGS,
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1048 "get an index entry"},
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1049 {"insert", (PyCFunction)index_insert, METH_VARARGS,
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1050 "insert an index entry"},
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1051 {"stats", (PyCFunction)index_stats, METH_NOARGS,
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1052 "stats for the index"},
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1053 {NULL} /* Sentinel */
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1054 };
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1055
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1056 static PyGetSetDef index_getset[] = {
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1057 {"nodemap", (getter)index_nodemap, NULL, "nodemap", NULL},
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1058 {NULL} /* Sentinel */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1059 };
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1060
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1061 static PyTypeObject indexType = {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1062 PyObject_HEAD_INIT(NULL)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1063 0, /* ob_size */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1064 "parsers.index", /* tp_name */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1065 sizeof(indexObject), /* tp_basicsize */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1066 0, /* tp_itemsize */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1067 (destructor)index_dealloc, /* tp_dealloc */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1068 0, /* tp_print */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1069 0, /* tp_getattr */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1070 0, /* tp_setattr */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1071 0, /* tp_compare */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1072 0, /* tp_repr */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1073 0, /* tp_as_number */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1074 &index_sequence_methods, /* tp_as_sequence */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1075 &index_mapping_methods, /* tp_as_mapping */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1076 0, /* tp_hash */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1077 0, /* tp_call */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1078 0, /* tp_str */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1079 0, /* tp_getattro */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1080 0, /* tp_setattro */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1081 0, /* tp_as_buffer */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1082 Py_TPFLAGS_DEFAULT, /* tp_flags */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1083 "revlog index", /* tp_doc */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1084 0, /* tp_traverse */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1085 0, /* tp_clear */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1086 0, /* tp_richcompare */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1087 0, /* tp_weaklistoffset */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1088 0, /* tp_iter */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1089 0, /* tp_iternext */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1090 index_methods, /* tp_methods */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1091 0, /* tp_members */
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1092 index_getset, /* tp_getset */
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1093 0, /* tp_base */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1094 0, /* tp_dict */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1095 0, /* tp_descr_get */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1096 0, /* tp_descr_set */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1097 0, /* tp_dictoffset */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1098 (initproc)index_init, /* tp_init */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1099 0, /* tp_alloc */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1100 PyType_GenericNew, /* tp_new */
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1101 };
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1102
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1103 /*
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1104 * returns a tuple of the form (index, index, cache) with elements as
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1105 * follows:
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	1106 *
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1107 * index: an index object that lazily parses RevlogNG records
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1108 * cache: if data is inlined, a tuple (index_file_content, 0), else None
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1109 *
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1110 * added complications are for backwards compatibility
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	1111 */
13254 5ef5eb1f3515 revlog: only build the nodemap on demand Matt Mackall <mpm@selenic.com> parents: 11361 diff changeset	1112 static PyObject parse_index2(PyObject self, PyObject *args)
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	1113 {
1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	1114 const char *data;
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1115 int size, ret;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1116 PyObject inlined_obj, tuple = NULL, *cache = NULL;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1117 indexObject *idx;
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	1118
1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	1119 if (!PyArg_ParseTuple(args, "s#O", &data, &size, &inlined_obj))
1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	1120 return NULL;
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1121
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1122 idx = PyObject_New(indexObject, &indexType);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1123
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1124 if (idx == NULL)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1125 goto bail;
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	1126
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1127 ret = index_real_init(idx, data, size, inlined_obj,
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1128 PyTuple_GET_ITEM(args, 0));
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1129 if (ret)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1130 goto bail;
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	1131
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1132 if (idx->inlined) {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1133 Py_INCREF(idx->data);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1134 cache = Py_BuildValue("iO", 0, idx->data);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1135 if (cache == NULL)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1136 goto bail;
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	1137 } else {
1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	1138 cache = Py_None;
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1139 Py_INCREF(cache);
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	1140 }
1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	1141
16414 e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1142 Py_INCREF(idx);
e8d37b78acfb parsers: use base-16 trie for faster node->rev mapping Bryan O'Sullivan <bryano@fb.com> parents: 16393 diff changeset	1143
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1144 tuple = Py_BuildValue("NN", idx, cache);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1145 if (!tuple)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1146 goto bail;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1147 return tuple;
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	1148
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1149 bail:
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1150 Py_XDECREF(idx);
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	1151 Py_XDECREF(cache);
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1152 Py_XDECREF(tuple);
7108 1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	1153 return NULL;
1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	1154 }
1ca878d7b849 C implementation of revlog index parsing Bernhard Leiner <bleiner@gmail.com> parents: 7093 diff changeset	1155
6389 0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	1156 static char parsers_doc[] = "Efficient content parsing.";
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	1157
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	1158 static PyMethodDef methods[] = {
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	1159 {"parse_manifest", parse_manifest, METH_VARARGS, "parse a manifest\n"},
7093 16bafcebd3d1 dirstate: C parsing extension Matt Mackall <mpm@selenic.com> parents: 7092 diff changeset	1160 {"parse_dirstate", parse_dirstate, METH_VARARGS, "parse a dirstate\n"},
13254 5ef5eb1f3515 revlog: only build the nodemap on demand Matt Mackall <mpm@selenic.com> parents: 11361 diff changeset	1161 {"parse_index2", parse_index2, METH_VARARGS, "parse a revlog index\n"},
6389 0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	1162 {NULL, NULL}
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	1163 };
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	1164
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1165 static void module_init(PyObject *mod)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1166 {
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1167 if (PyType_Ready(&indexType) < 0)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1168 return;
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1169 Py_INCREF(&indexType);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1170
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1171 PyModule_AddObject(mod, "index", (PyObject *)&indexType);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1172
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1173 nullentry = Py_BuildValue("iiiiiiis#", 0, 0, 0,
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1174 -1, -1, -1, -1, nullid, 20);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1175 if (nullentry)
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1176 PyObject_GC_UnTrack(nullentry);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1177 }
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1178
11361 3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	1179 #ifdef IS_PY3K
3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	1180 static struct PyModuleDef parsers_module = {
3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	1181 PyModuleDef_HEAD_INIT,
3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	1182 "parsers",
3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	1183 parsers_doc,
3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	1184 -1,
3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	1185 methods
3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	1186 };
3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	1187
3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	1188 PyMODINIT_FUNC PyInit_parsers(void)
3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	1189 {
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1190 PyObject *mod = PyModule_Create(&parsers_module);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1191 module_init(mod);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1192 return mod;
11361 3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	1193 }
3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	1194 #else
6389 0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	1195 PyMODINIT_FUNC initparsers(void)
0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	1196 {
16363 2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1197 PyObject *mod = Py_InitModule3("parsers", methods, parsers_doc);
2cdd7e63211b parsers: incrementally parse the revlog index in C Bryan O'Sullivan <bryano@fb.com> parents: 15033 diff changeset	1198 module_init(mod);
6389 0231f763ebc8 manifest: improve parsing performance by 8x via a new C extension Bryan O'Sullivan <bos@serpentine.com> parents: diff changeset	1199 }
11361 3de3d670d2b6 parsers.c: Added support for py3k. Renato Cunha <renatoc@gmail.com> parents: 10449 diff changeset	1200 #endif

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/parsers.c @ 16414:e8d37b78acfb