annotate rust/hg-core/examples/nodemap/index.rs @ 44417:8f7c6656ac79

rust-nodemap: pure Rust example To run, use `cargo run --release --example nodemap` This demonstrates that simple scenarios entirely written in Rust can content themselves with `NodeTree<T>`. The example mmaps both the nodemap file and the changelog index. We had of course to include an implementation of `RevlogIndex` directly, which isn't much at this stage. It felt a bit prematurate to include it in the lib. Here are some first performance measurements, obtained with this example, on a clone of mozilla-central with 440000 changesets: (create) Nodemap constructed in RAM in 153.638305ms (query CAE63161B68962) found in 22.362us: Ok(Some(269489)) (bench) Did 3 queries in 36.418?s (mean 12.139?s) (bench) Did 50 queries in 184.318?s (mean 3.686?s) (bench) Did 100000 queries in 31.053461ms (mean 310ns) To be fair, even between bench runs, results tend to depend whether the file is still in kernel caches, and it's not so easy to get back to a real cold start. The worst we've seen was in the 50us ballpark. In any busy server setting, the pages would always be in RAM. We hope it's good enough not to be significantly slower on any concrete Mercurial operation than the C nodetree when fully in RAM, and of course this implementation has the serious headstart advantage of persistence. Differential Revision: https://phab.mercurial-scm.org/D7797
author Georges Racinet <georges.racinet@octobus.net>
date Tue, 18 Feb 2020 19:11:15 +0100
parents
children e834b79def74
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
44417
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
1 // Copyright 2019-2020 Georges Racinet <georges.racinet@octobus.net>
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
2 //
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
3 // This software may be used and distributed according to the terms of the
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
4 // GNU General Public License version 2 or any later version.
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
5
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
6 //! Minimal `RevlogIndex`, readable from standard Mercurial file format
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
7 use hg::*;
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
8 use memmap::*;
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
9 use std::fs::File;
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
10 use std::ops::Deref;
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
11 use std::path::Path;
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
12 use std::slice;
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
13
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
14 pub struct Index {
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
15 data: Box<dyn Deref<Target = [IndexEntry]> + Send>,
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
16 }
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
17
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
18 /// A fixed sized index entry. All numbers are big endian
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
19 #[repr(C)]
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
20 pub struct IndexEntry {
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
21 not_used_yet: [u8; 24],
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
22 p1: Revision,
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
23 p2: Revision,
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
24 node: Node,
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
25 unused_node: [u8; 12],
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
26 }
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
27
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
28 pub const INDEX_ENTRY_SIZE: usize = 64;
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
29
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
30 impl IndexEntry {
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
31 fn parents(&self) -> [Revision; 2] {
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
32 [Revision::from_be(self.p1), Revision::from_be(self.p1)]
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
33 }
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
34 }
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
35
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
36 impl RevlogIndex for Index {
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
37 fn len(&self) -> usize {
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
38 self.data.len()
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
39 }
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
40
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
41 fn node(&self, rev: Revision) -> Option<&Node> {
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
42 if rev == NULL_REVISION {
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
43 return None;
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
44 }
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
45 let i = rev as usize;
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
46 if i >= self.len() {
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
47 None
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
48 } else {
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
49 Some(&self.data[i].node)
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
50 }
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
51 }
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
52 }
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
53
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
54 impl Graph for &Index {
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
55 fn parents(&self, rev: Revision) -> Result<[Revision; 2], GraphError> {
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
56 let [p1, p2] = (*self).data[rev as usize].parents();
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
57 let len = (*self).len();
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
58 if p1 < NULL_REVISION
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
59 || p2 < NULL_REVISION
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
60 || p1 as usize >= len
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
61 || p2 as usize >= len
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
62 {
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
63 return Err(GraphError::ParentOutOfRange(rev));
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
64 }
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
65 Ok([p1, p2])
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
66 }
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
67 }
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
68
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
69 struct IndexMmap(Mmap);
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
70
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
71 impl Deref for IndexMmap {
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
72 type Target = [IndexEntry];
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
73
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
74 fn deref(&self) -> &[IndexEntry] {
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
75 let ptr = self.0.as_ptr() as *const IndexEntry;
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
76 // Any misaligned data will be ignored.
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
77 debug_assert_eq!(
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
78 self.0.len() % std::mem::align_of::<IndexEntry>(),
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
79 0,
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
80 "Misaligned data in mmap"
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
81 );
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
82 unsafe { slice::from_raw_parts(ptr, self.0.len() / INDEX_ENTRY_SIZE) }
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
83 }
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
84 }
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
85
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
86 impl Index {
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
87 pub fn load_mmap(path: impl AsRef<Path>) -> Self {
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
88 let file = File::open(path).unwrap();
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
89 let msg = "Index file is missing, or missing permission";
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
90 let mmap = unsafe { MmapOptions::new().map(&file) }.expect(msg);
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
91 Self {
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
92 data: Box::new(IndexMmap(mmap)),
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
93 }
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
94 }
8f7c6656ac79 rust-nodemap: pure Rust example
Georges Racinet <georges.racinet@octobus.net>
parents:
diff changeset
95 }