Mercurial > public > mercurial-scm > hg
annotate rust/chg/src/attachio.rs @ 48178:f12a19d03d2c
fix: reduce number of tool executions
By grouping together (path, ctx) pairs according to the inputs they would
provide to fixer tools, we can deduplicate executions of fixer tools to
significantly reduce the amount of time spent running slow tools.
This change does not handle clean files in the working copy, which could still
be deduplicated against the files in the checked out commit. It's a little
harder to do that because the filerev is not available in the workingfilectx
(and it doesn't exist for added files).
Anecdotally, this change makes some real uses cases at Google 10x faster. I
think we were originally hesitant to do this because the benefits weren't
obvious, and implementing it efficiently is kind of tricky. If we simply
memoized the formatter execution function, we would be keeping tons of file
content in memory.
Also included is a regression test for a corner case that I broke with my first
attempt at optimizing this code.
Differential Revision: https://phab.mercurial-scm.org/D11280
author | Danny Hooper <hooper@google.com> |
---|---|
date | Thu, 02 Sep 2021 14:08:45 -0700 |
parents | 27fe8cc1338f |
children |
rev | line source |
---|---|
39972
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
1 // Copyright 2018 Yuya Nishihara <yuya@tcha.org> |
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
2 // |
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
3 // This software may be used and distributed according to the terms of the |
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
4 // GNU General Public License version 2 or any later version. |
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
5 |
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
6 //! Functions to send client-side fds over the command server channel. |
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
7 |
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
8 use std::io; |
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
9 use std::os::unix::io::AsRawFd; |
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
10 use tokio_hglib::codec::ChannelMessage; |
44738
1be605526c34
rust-chg: reimplement attach_io operation as async function
Yuya Nishihara <yuya@tcha.org>
parents:
44689
diff
changeset
|
11 use tokio_hglib::{Connection, Protocol}; |
39972
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
12 |
44689
6bef9d43cc55
rust-chg: use "crate::" to import local modules
Yuya Nishihara <yuya@tcha.org>
parents:
44688
diff
changeset
|
13 use crate::message; |
6bef9d43cc55
rust-chg: use "crate::" to import local modules
Yuya Nishihara <yuya@tcha.org>
parents:
44688
diff
changeset
|
14 use crate::procutil; |
39972
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
15 |
44738
1be605526c34
rust-chg: reimplement attach_io operation as async function
Yuya Nishihara <yuya@tcha.org>
parents:
44689
diff
changeset
|
16 /// Sends client-side fds over the command server channel. |
39972
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
17 /// |
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
18 /// This works as follows: |
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
19 /// 1. Client sends "attachio" request. |
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
20 /// 2. Server sends back 1-byte input request. |
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
21 /// 3. Client sends fds with 1-byte dummy payload in response. |
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
22 /// 4. Server returns the number of the fds received. |
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
23 /// |
44749
cb5822e6e545
rust-chg: have attach_io() simply take reference of AsRawFd object
Yuya Nishihara <yuya@tcha.org>
parents:
44738
diff
changeset
|
24 /// The client-side fds may be dropped once duplicated to the server. |
44738
1be605526c34
rust-chg: reimplement attach_io operation as async function
Yuya Nishihara <yuya@tcha.org>
parents:
44689
diff
changeset
|
25 pub async fn attach_io( |
1be605526c34
rust-chg: reimplement attach_io operation as async function
Yuya Nishihara <yuya@tcha.org>
parents:
44689
diff
changeset
|
26 proto: &mut Protocol<impl Connection + AsRawFd>, |
44749
cb5822e6e545
rust-chg: have attach_io() simply take reference of AsRawFd object
Yuya Nishihara <yuya@tcha.org>
parents:
44738
diff
changeset
|
27 stdin: &impl AsRawFd, |
cb5822e6e545
rust-chg: have attach_io() simply take reference of AsRawFd object
Yuya Nishihara <yuya@tcha.org>
parents:
44738
diff
changeset
|
28 stdout: &impl AsRawFd, |
cb5822e6e545
rust-chg: have attach_io() simply take reference of AsRawFd object
Yuya Nishihara <yuya@tcha.org>
parents:
44738
diff
changeset
|
29 stderr: &impl AsRawFd, |
44738
1be605526c34
rust-chg: reimplement attach_io operation as async function
Yuya Nishihara <yuya@tcha.org>
parents:
44689
diff
changeset
|
30 ) -> io::Result<()> { |
44756
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
31 proto.send_command("attachio").await?; |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
32 loop { |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
33 match proto.fetch_response().await? { |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
34 ChannelMessage::Data(b'r', data) => { |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
35 let fd_cnt = message::parse_result_code(data)?; |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
36 if fd_cnt == 3 { |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
37 return Ok(()); |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
38 } else { |
43818
ce088b38f92b
rust: run rustfmt
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39972
diff
changeset
|
39 return Err(io::Error::new( |
ce088b38f92b
rust: run rustfmt
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39972
diff
changeset
|
40 io::ErrorKind::InvalidData, |
44756
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
41 "unexpected attachio result", |
43818
ce088b38f92b
rust: run rustfmt
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39972
diff
changeset
|
42 )); |
39972
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
43 } |
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
44 } |
44756
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
45 ChannelMessage::Data(..) => { |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
46 // just ignore data sent to uninteresting (optional) channel |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
47 } |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
48 ChannelMessage::InputRequest(1) => { |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
49 // this may fail with EWOULDBLOCK in theory, but the |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
50 // payload is quite small, and the send buffer should |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
51 // be empty so the operation will complete immediately |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
52 let sock_fd = proto.as_raw_fd(); |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
53 let ifd = stdin.as_raw_fd(); |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
54 let ofd = stdout.as_raw_fd(); |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
55 let efd = stderr.as_raw_fd(); |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
56 procutil::send_raw_fds(sock_fd, &[ifd, ofd, efd])?; |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
57 } |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
58 ChannelMessage::InputRequest(..) |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
59 | ChannelMessage::LineRequest(..) |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
60 | ChannelMessage::SystemRequest(..) => { |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
61 return Err(io::Error::new( |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
62 io::ErrorKind::InvalidData, |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
63 "unsupported request while attaching io", |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
64 )); |
27fe8cc1338f
rust-chg: clean up excessive indents
Yuya Nishihara <yuya@tcha.org>
parents:
44749
diff
changeset
|
65 } |
39972
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
66 } |
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
67 } |
7a0ffdd4af78
rust-chg: add future that handles "attachio" request
Yuya Nishihara <yuya@tcha.org>
parents:
diff
changeset
|
68 } |