rust/hg-cpython/src/lib.rs
author Rapha?l Gom?s <rgomes@octobus.net>
Wed, 19 Jun 2024 19:10:49 +0200
changeset 52163 7346f93be7a4
parent 52056 8b7123c8947b
child 52292 4361d787e6cf
permissions -rw-r--r--
revlog: add the glue to use the Rust `InnerRevlog` from Python The performance of this has been looked at for quite some time, and some workflows are actually quite a bit faster than with the Python + C code. However, we are still (up to 20%) slower in some crucial places like cloning certain repos, log, cat, which makes this an incomplete rewrite. This is mostly due to the high amount of overhead in Python <-> Rust FFI, especially around the VFS code. A future patch series will rewrite the VFS code in pure Rust, which should hopefully get us up to par with current perfomance, if not better in all important cases. This is a "save state" of sorts, as this is a ton of code, and I don't want to pile up even more things in a single review. Continuing to try to match the current performance will take an extremely long time, if it's not impossible, without the aforementioned VFS work.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
40965
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
     1
// lib.rs
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
     2
//
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
     3
// Copyright 2018 Georges Racinet <gracinet@anybox.fr>
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
     4
//
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
     5
// This software may be used and distributed according to the terms of the
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
     6
// GNU General Public License version 2 or any later version.
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
     7
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
     8
//! Python bindings of `hg-core` objects using the `cpython` crate.
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
     9
//! Once compiled, the resulting single shared library object can be placed in
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    10
//! the `mercurial` package directly as `rustext.so` or `rustext.dll`.
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    11
//! It holds several modules, so that from the point of view of Python,
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    12
//! it behaves as the `cext` package.
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    13
//!
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    14
//! Example:
41184
dcf818267bc1 rust-cpython: rustdoc improvements
Georges Racinet <georges.racinet@octobus.net>
parents: 41130
diff changeset
    15
//!
dcf818267bc1 rust-cpython: rustdoc improvements
Georges Racinet <georges.racinet@octobus.net>
parents: 41130
diff changeset
    16
//! ```text
40965
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    17
//! >>> from mercurial.rustext import ancestor
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    18
//! >>> ancestor.__doc__
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    19
//! 'Generic DAG ancestor algorithms - Rust implementation'
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    20
//! ```
49932
136aa80aa8b2 rust-clippy: disable some lints crate-wide for `hg-cpython`
Rapha?l Gom?s <rgomes@octobus.net>
parents: 48839
diff changeset
    21
#![allow(clippy::too_many_arguments)] // rust-cpython macros
136aa80aa8b2 rust-clippy: disable some lints crate-wide for `hg-cpython`
Rapha?l Gom?s <rgomes@octobus.net>
parents: 48839
diff changeset
    22
#![allow(clippy::zero_ptr)] // rust-cpython macros
136aa80aa8b2 rust-clippy: disable some lints crate-wide for `hg-cpython`
Rapha?l Gom?s <rgomes@octobus.net>
parents: 48839
diff changeset
    23
#![allow(clippy::needless_update)] // rust-cpython macros
136aa80aa8b2 rust-clippy: disable some lints crate-wide for `hg-cpython`
Rapha?l Gom?s <rgomes@octobus.net>
parents: 48839
diff changeset
    24
#![allow(clippy::manual_strip)] // rust-cpython macros
136aa80aa8b2 rust-clippy: disable some lints crate-wide for `hg-cpython`
Rapha?l Gom?s <rgomes@octobus.net>
parents: 48839
diff changeset
    25
#![allow(clippy::type_complexity)] // rust-cpython macros
40965
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    26
50976
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    27
use cpython::{FromPyObject, PyInt, Python, ToPyObject};
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    28
use hg::{BaseRevision, Revision};
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    29
42609
326fdce22fb2 rust: switch hg-core and hg-cpython to rust 2018 edition
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42557
diff changeset
    30
/// This crate uses nested private macros, `extern crate` is still needed in
326fdce22fb2 rust: switch hg-core and hg-cpython to rust 2018 edition
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42557
diff changeset
    31
/// 2018 edition.
40965
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    32
#[macro_use]
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    33
extern crate cpython;
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    34
41184
dcf818267bc1 rust-cpython: rustdoc improvements
Georges Racinet <georges.racinet@octobus.net>
parents: 41130
diff changeset
    35
pub mod ancestors;
41052
4c25038c112c rust-cpython: implement Graph using C parents function
Georges Racinet <gracinet@anybox.fr>
parents: 40965
diff changeset
    36
mod cindex;
41240
ff333620a4cc rust-cpython: moved generic conversion fn out of ancestors module
Georges Racinet <georges.racinet@octobus.net>
parents: 41184
diff changeset
    37
mod conversion;
42752
30320c7bf79f rust-cpython: add macro for sharing references
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42747
diff changeset
    38
#[macro_use]
30320c7bf79f rust-cpython: add macro for sharing references
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42747
diff changeset
    39
pub mod ref_sharing;
45945
50c5ee3bdf9a copies: introduce the hg-cpython wrapper for `combine_changeset_copies`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44531
diff changeset
    40
pub mod copy_tracing;
41694
0c7b353ce100 rust-cpython: binding for headrevs()
Georges Racinet <georges.racinet@octobus.net>
parents: 41240
diff changeset
    41
pub mod dagops;
44531
d4f19eb471ca rust-cpython: add `debug` module to expose debug information to Python
Rapha?l Gom?s <rgomes@octobus.net>
parents: 44137
diff changeset
    42
pub mod debug;
42557
d26e4a434fe5 rust: run rfmt on all hg-core/hg-cpython code
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42357
diff changeset
    43
pub mod dirstate;
42179
13b64247f48f rust-discovery: cpython bindings for the core logic
Georges Racinet <georges.racinet@octobus.net>
parents: 41694
diff changeset
    44
pub mod discovery;
41184
dcf818267bc1 rust-cpython: rustdoc improvements
Georges Racinet <georges.racinet@octobus.net>
parents: 41130
diff changeset
    45
pub mod exceptions;
47953
8f031a274cd6 rust: Move PyBytesWithData out of copy-tracing code
Simon Sapin <simon.sapin@octobus.net>
parents: 45945
diff changeset
    46
mod pybytes_deref;
43945
f98f0e3ddaa1 rust-index: add a function to convert PyObject index for hg-core
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43289
diff changeset
    47
pub mod revlog;
52056
8b7123c8947b update: add a Rust fast-path when updating from null (and clean)
Rapha?l Gom?s <rgomes@octobus.net>
parents: 50976
diff changeset
    48
pub mod update;
43251
970978975574 rust-utils: introduce a debug util to print the python stack trace
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42819
diff changeset
    49
pub mod utils;
52163
7346f93be7a4 revlog: add the glue to use the Rust `InnerRevlog` from Python
Rapha?l Gom?s <rgomes@octobus.net>
parents: 52056
diff changeset
    50
pub mod vfs;
40965
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    51
50976
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    52
/// Revision as exposed to/from the Python layer.
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    53
///
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    54
/// We need this indirection because of the orphan rule, meaning we can't
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    55
/// implement a foreign trait (like [`cpython::ToPyObject`])
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    56
/// for a foreign type (like [`hg::UncheckedRevision`]).
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    57
///
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    58
/// This also acts as a deterrent against blindly trusting Python to send
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    59
/// us valid revision numbers.
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    60
#[derive(Debug, Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    61
pub struct PyRevision(BaseRevision);
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    62
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    63
impl From<Revision> for PyRevision {
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    64
    fn from(r: Revision) -> Self {
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    65
        PyRevision(r.0)
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    66
    }
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    67
}
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    68
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    69
impl<'s> FromPyObject<'s> for PyRevision {
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    70
    fn extract(
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    71
        py: Python,
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    72
        obj: &'s cpython::PyObject,
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    73
    ) -> cpython::PyResult<Self> {
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    74
        Ok(Self(obj.extract::<BaseRevision>(py)?))
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    75
    }
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    76
}
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    77
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    78
impl ToPyObject for PyRevision {
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    79
    type ObjectType = PyInt;
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    80
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    81
    fn to_py_object(&self, py: Python) -> Self::ObjectType {
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    82
        self.0.to_py_object(py)
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    83
    }
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    84
}
4c5f6e95df84 rust: make `Revision` a newtype
Rapha?l Gom?s <rgomes@octobus.net>
parents: 49932
diff changeset
    85
40965
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    86
py_module_initializer!(rustext, initrustext, PyInit_rustext, |py, m| {
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    87
    m.add(
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    88
        py,
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    89
        "__doc__",
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    90
        "Mercurial core concepts - Rust implementation",
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    91
    )?;
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    92
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    93
    let dotted_name: String = m.get(py, "__name__")?.extract(py)?;
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
    94
    m.add(py, "ancestor", ancestors::init_module(py, &dotted_name)?)?;
41694
0c7b353ce100 rust-cpython: binding for headrevs()
Georges Racinet <georges.racinet@octobus.net>
parents: 41240
diff changeset
    95
    m.add(py, "dagop", dagops::init_module(py, &dotted_name)?)?;
44531
d4f19eb471ca rust-cpython: add `debug` module to expose debug information to Python
Rapha?l Gom?s <rgomes@octobus.net>
parents: 44137
diff changeset
    96
    m.add(py, "debug", debug::init_module(py, &dotted_name)?)?;
45945
50c5ee3bdf9a copies: introduce the hg-cpython wrapper for `combine_changeset_copies`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44531
diff changeset
    97
    m.add(
50c5ee3bdf9a copies: introduce the hg-cpython wrapper for `combine_changeset_copies`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44531
diff changeset
    98
        py,
50c5ee3bdf9a copies: introduce the hg-cpython wrapper for `combine_changeset_copies`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44531
diff changeset
    99
        "copy_tracing",
50c5ee3bdf9a copies: introduce the hg-cpython wrapper for `combine_changeset_copies`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44531
diff changeset
   100
        copy_tracing::init_module(py, &dotted_name)?,
50c5ee3bdf9a copies: introduce the hg-cpython wrapper for `combine_changeset_copies`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44531
diff changeset
   101
    )?;
42179
13b64247f48f rust-discovery: cpython bindings for the core logic
Georges Racinet <georges.racinet@octobus.net>
parents: 41694
diff changeset
   102
    m.add(py, "discovery", discovery::init_module(py, &dotted_name)?)?;
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42179
diff changeset
   103
    m.add(py, "dirstate", dirstate::init_module(py, &dotted_name)?)?;
43961
b69d5f3a41d0 rust-index: add a struct wrapping the C index
Georges Racinet <georges.racinet@octobus.net>
parents: 43945
diff changeset
   104
    m.add(py, "revlog", revlog::init_module(py, &dotted_name)?)?;
52056
8b7123c8947b update: add a Rust fast-path when updating from null (and clean)
Rapha?l Gom?s <rgomes@octobus.net>
parents: 50976
diff changeset
   105
    m.add(py, "update", update::init_module(py, &dotted_name)?)?;
40965
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
   106
    m.add(py, "GraphError", py.get_type::<exceptions::GraphError>())?;
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
   107
    Ok(())
5532823e8c18 rust-cpython: start cpython crate bindings
Georges Racinet <gracinet@anybox.fr>
parents:
diff changeset
   108
});
43289
8d432d3a2d7c rust-cpython: prepare for writing tests that require libpython
Yuya Nishihara <yuya@tcha.org>
parents: 43251
diff changeset
   109
48839
7b068abe4aa2 rust: jettison Python 2 support
Augie Fackler <augie@google.com>
parents: 48020
diff changeset
   110
#[cfg(not(feature = "python3-bin"))]
43289
8d432d3a2d7c rust-cpython: prepare for writing tests that require libpython
Yuya Nishihara <yuya@tcha.org>
parents: 43251
diff changeset
   111
#[test]
8d432d3a2d7c rust-cpython: prepare for writing tests that require libpython
Yuya Nishihara <yuya@tcha.org>
parents: 43251
diff changeset
   112
#[ignore]
8d432d3a2d7c rust-cpython: prepare for writing tests that require libpython
Yuya Nishihara <yuya@tcha.org>
parents: 43251
diff changeset
   113
fn libpython_must_be_linked_to_run_tests() {
8d432d3a2d7c rust-cpython: prepare for writing tests that require libpython
Yuya Nishihara <yuya@tcha.org>
parents: 43251
diff changeset
   114
    // stub function to tell that some tests wouldn't run
8d432d3a2d7c rust-cpython: prepare for writing tests that require libpython
Yuya Nishihara <yuya@tcha.org>
parents: 43251
diff changeset
   115
}