annotate rust/hg-core/src/dirstate.rs @ 48178:f12a19d03d2c

fix: reduce number of tool executions By grouping together (path, ctx) pairs according to the inputs they would provide to fixer tools, we can deduplicate executions of fixer tools to significantly reduce the amount of time spent running slow tools. This change does not handle clean files in the working copy, which could still be deduplicated against the files in the checked out commit. It's a little harder to do that because the filerev is not available in the workingfilectx (and it doesn't exist for added files). Anecdotally, this change makes some real uses cases at Google 10x faster. I think we were originally hesitant to do this because the benefits weren't obvious, and implementing it efficiently is kind of tricky. If we simply memoized the formatter execution function, we would be keeping tons of file content in memory. Also included is a regression test for a corner case that I broke with my first attempt at optimizing this code. Differential Revision: https://phab.mercurial-scm.org/D11280
author Danny Hooper <hooper@google.com>
date Thu, 02 Sep 2021 14:08:45 -0700
parents bf8837e3d7ce
children b07465adbcc8
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
42748
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42609
diff changeset
1 // dirstate module
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42609
diff changeset
2 //
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42609
diff changeset
3 // Copyright 2019 Raphaël Gomès <rgomes@octobus.net>
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42609
diff changeset
4 //
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42609
diff changeset
5 // This software may be used and distributed according to the terms of the
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42609
diff changeset
6 // GNU General Public License version 2 or any later version.
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42609
diff changeset
7
47335
ed1583a845d2 dirstate-v2: Make more APIs fallible, returning Result
Simon Sapin <simon.sapin@octobus.net>
parents: 47332
diff changeset
8 use crate::dirstate_tree::on_disk::DirstateV2ParseError;
47374
bd88b6bfd8da rhg: Add support for dirstate-v2
Simon Sapin <simon.sapin@octobus.net>
parents: 47335
diff changeset
9 use crate::revlog::node::NULL_NODE;
46595
98a455a62699 rust: Make `DirstateParents`?s fields typed `Node`s
Simon Sapin <simon.sapin@octobus.net>
parents: 46594
diff changeset
10 use crate::revlog::Node;
48068
bf8837e3d7ce dirstate: Remove the flat Rust DirstateMap implementation
Simon Sapin <simon.sapin@octobus.net>
parents: 48018
diff changeset
11 use crate::utils::hg_path::HgPath;
48018
08efe5945d2b rust: Move DirstateEntry to its own module
Simon Sapin <simon.sapin@octobus.net>
parents: 47956
diff changeset
12 use bytes_cast::BytesCast;
42748
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42609
diff changeset
13
42536
2dcee6497b0b rust-dirstate: add "dirs" Rust implementation
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42424
diff changeset
14 pub mod dirs_multiset;
48018
08efe5945d2b rust: Move DirstateEntry to its own module
Simon Sapin <simon.sapin@octobus.net>
parents: 47956
diff changeset
15 pub mod entry;
42424
d3b5cbe311d9 rust-dirstate: create dirstate submodule
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
16 pub mod parsers;
43271
99394e6c5d12 rust-dirstate-status: add first Rust implementation of `dirstate.status`
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42957
diff changeset
17 pub mod status;
42424
d3b5cbe311d9 rust-dirstate: create dirstate submodule
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
18
48018
08efe5945d2b rust: Move DirstateEntry to its own module
Simon Sapin <simon.sapin@octobus.net>
parents: 47956
diff changeset
19 pub use self::entry::*;
08efe5945d2b rust: Move DirstateEntry to its own module
Simon Sapin <simon.sapin@octobus.net>
parents: 47956
diff changeset
20
47956
81aedf1fc897 rust: Add Repo::dirstate_map and use it in `rhg status`
Simon Sapin <simon.sapin@octobus.net>
parents: 47683
diff changeset
21 #[derive(Debug, PartialEq, Copy, Clone, BytesCast)]
46594
f88e8ae0aa8f rust: Rewrite dirstate parsing usin the `bytes-cast` crate
Simon Sapin <simon.sapin@octobus.net>
parents: 46440
diff changeset
22 #[repr(C)]
42748
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42609
diff changeset
23 pub struct DirstateParents {
46595
98a455a62699 rust: Make `DirstateParents`?s fields typed `Node`s
Simon Sapin <simon.sapin@octobus.net>
parents: 46594
diff changeset
24 pub p1: Node,
98a455a62699 rust: Make `DirstateParents`?s fields typed `Node`s
Simon Sapin <simon.sapin@octobus.net>
parents: 46594
diff changeset
25 pub p2: Node,
42424
d3b5cbe311d9 rust-dirstate: create dirstate submodule
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
26 }
d3b5cbe311d9 rust-dirstate: create dirstate submodule
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
27
47374
bd88b6bfd8da rhg: Add support for dirstate-v2
Simon Sapin <simon.sapin@octobus.net>
parents: 47335
diff changeset
28 impl DirstateParents {
bd88b6bfd8da rhg: Add support for dirstate-v2
Simon Sapin <simon.sapin@octobus.net>
parents: 47335
diff changeset
29 pub const NULL: Self = Self {
bd88b6bfd8da rhg: Add support for dirstate-v2
Simon Sapin <simon.sapin@octobus.net>
parents: 47335
diff changeset
30 p1: NULL_NODE,
bd88b6bfd8da rhg: Add support for dirstate-v2
Simon Sapin <simon.sapin@octobus.net>
parents: 47335
diff changeset
31 p2: NULL_NODE,
bd88b6bfd8da rhg: Add support for dirstate-v2
Simon Sapin <simon.sapin@octobus.net>
parents: 47335
diff changeset
32 };
bd88b6bfd8da rhg: Add support for dirstate-v2
Simon Sapin <simon.sapin@octobus.net>
parents: 47335
diff changeset
33 }
bd88b6bfd8da rhg: Add support for dirstate-v2
Simon Sapin <simon.sapin@octobus.net>
parents: 47335
diff changeset
34
47335
ed1583a845d2 dirstate-v2: Make more APIs fallible, returning Result
Simon Sapin <simon.sapin@octobus.net>
parents: 47332
diff changeset
35 pub type StateMapIter<'a> = Box<
ed1583a845d2 dirstate-v2: Make more APIs fallible, returning Result
Simon Sapin <simon.sapin@octobus.net>
parents: 47332
diff changeset
36 dyn Iterator<
ed1583a845d2 dirstate-v2: Make more APIs fallible, returning Result
Simon Sapin <simon.sapin@octobus.net>
parents: 47332
diff changeset
37 Item = Result<(&'a HgPath, DirstateEntry), DirstateV2ParseError>,
ed1583a845d2 dirstate-v2: Make more APIs fallible, returning Result
Simon Sapin <simon.sapin@octobus.net>
parents: 47332
diff changeset
38 > + Send
ed1583a845d2 dirstate-v2: Make more APIs fallible, returning Result
Simon Sapin <simon.sapin@octobus.net>
parents: 47332
diff changeset
39 + 'a,
ed1583a845d2 dirstate-v2: Make more APIs fallible, returning Result
Simon Sapin <simon.sapin@octobus.net>
parents: 47332
diff changeset
40 >;
45610
496537c9c1b4 rust: start plugging the dirstate tree behind a feature gate
Rapha?l Gom?s <rgomes@octobus.net>
parents: 45609
diff changeset
41
47335
ed1583a845d2 dirstate-v2: Make more APIs fallible, returning Result
Simon Sapin <simon.sapin@octobus.net>
parents: 47332
diff changeset
42 pub type CopyMapIter<'a> = Box<
ed1583a845d2 dirstate-v2: Make more APIs fallible, returning Result
Simon Sapin <simon.sapin@octobus.net>
parents: 47332
diff changeset
43 dyn Iterator<Item = Result<(&'a HgPath, &'a HgPath), DirstateV2ParseError>>
ed1583a845d2 dirstate-v2: Make more APIs fallible, returning Result
Simon Sapin <simon.sapin@octobus.net>
parents: 47332
diff changeset
44 + Send
ed1583a845d2 dirstate-v2: Make more APIs fallible, returning Result
Simon Sapin <simon.sapin@octobus.net>
parents: 47332
diff changeset
45 + 'a,
ed1583a845d2 dirstate-v2: Make more APIs fallible, returning Result
Simon Sapin <simon.sapin@octobus.net>
parents: 47332
diff changeset
46 >;