rust/hg-cpython/src/dirstate.rs
author Rapha?l Gom?s <rgomes@octobus.net>
Tue, 09 Jul 2019 15:15:54 +0200
changeset 42752 30320c7bf79f
parent 42749 7ceded4419a3
child 42754 4e8f504424f3
permissions -rw-r--r--
rust-cpython: add macro for sharing references Following an experiment done by Georges Racinet, we now have a working way of sharing references between Python and Rust. This is needed in many points of the codebase, for example every time we need to expose an iterator to a Rust-backed Python class. In a few words, references are (unsafely) marked as `'static` and coupled with manual reference counting; we are doing manual borrow-checking. This changes introduces two declarative macro to help reduce boilerplate. While it is better than not using macros, they are not perfect. They need to: - Integrate with the garbage collector for container types (not needed as of yet), as stated in the docstring - Allow for leaking multiple attributes at the same time - Inject the `py_shared_state` data attribute in `py_class`-generated structs - Automatically namespace the functions and attributes they generate For at least the last two points, we will need to write a procedural macro instead of a declarative one. While this reference-sharing mechanism is being ironed out I thought it best not to implement it yet. Lastly, and implementation detail renders our Rust-backed Python iterators too strict to be proper drop-in replacements, as will be illustrated in a future patch: if the data structure referenced by a non-depleted iterator is mutated, an `AlreadyBorrowed` exception is raised, whereas Python would allow it, only to raise a `RuntimeError` if `next` is called on said iterator. This will have to be addressed at some point. Differential Revision: https://phab.mercurial-scm.org/D6631
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
     1
// dirstate.rs
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
     2
//
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
     3
// Copyright 2019 Raphaël Gomès <rgomes@octobus.net>
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
     4
//
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
     5
// This software may be used and distributed according to the terms of the
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
     6
// GNU General Public License version 2 or any later version.
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
     7
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
     8
//! Bindings for the `hg::dirstate` module provided by the
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
     9
//! `hg-core` package.
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    10
//!
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    11
//! From Python, this will be seen as `mercurial.rustext.dirstate`
42746
b3518b0baa47 rust-dirstate: create dirstate submodule in hg-cpython
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42609
diff changeset
    12
mod dirs_multiset;
b3518b0baa47 rust-dirstate: create dirstate submodule in hg-cpython
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42609
diff changeset
    13
use crate::dirstate::dirs_multiset::Dirs;
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    14
use cpython::{
42749
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42748
diff changeset
    15
    exc, PyBytes, PyDict, PyErr, PyModule, PyObject, PyResult, PySequence,
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42748
diff changeset
    16
    Python,
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    17
};
42749
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42748
diff changeset
    18
use hg::{DirstateEntry, DirstateParseError, EntryState, StateMap};
42609
326fdce22fb2 rust: switch hg-core and hg-cpython to rust 2018 edition
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42558
diff changeset
    19
use libc::{c_char, c_int};
326fdce22fb2 rust: switch hg-core and hg-cpython to rust 2018 edition
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42558
diff changeset
    20
#[cfg(feature = "python27")]
326fdce22fb2 rust: switch hg-core and hg-cpython to rust 2018 edition
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42558
diff changeset
    21
use python27_sys::PyCapsule_Import;
326fdce22fb2 rust: switch hg-core and hg-cpython to rust 2018 edition
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42558
diff changeset
    22
#[cfg(feature = "python3")]
326fdce22fb2 rust: switch hg-core and hg-cpython to rust 2018 edition
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42558
diff changeset
    23
use python3_sys::PyCapsule_Import;
42749
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42748
diff changeset
    24
use std::convert::TryFrom;
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    25
use std::ffi::CStr;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    26
use std::mem::transmute;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    27
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    28
/// C code uses a custom `dirstate_tuple` type, checks in multiple instances
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    29
/// for this type, and raises a Python `Exception` if the check does not pass.
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    30
/// Because this type differs only in name from the regular Python tuple, it
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    31
/// would be a good idea in the near future to remove it entirely to allow
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    32
/// for a pure Python tuple of the same effective structure to be used,
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    33
/// rendering this type and the capsule below useless.
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    34
type MakeDirstateTupleFn = extern "C" fn(
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    35
    state: c_char,
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    36
    mode: c_int,
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    37
    size: c_int,
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    38
    mtime: c_int,
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    39
) -> PyObject;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    40
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    41
/// This is largely a copy/paste from cindex.rs, pending the merge of a
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    42
/// `py_capsule_fn!` macro in the rust-cpython project:
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    43
/// https://github.com/dgrunwald/rust-cpython/pull/169
42747
760a7851e9ba rust-parsers: move parser bindings to their own file and Python module
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42746
diff changeset
    44
pub fn decapsule_make_dirstate_tuple(
760a7851e9ba rust-parsers: move parser bindings to their own file and Python module
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42746
diff changeset
    45
    py: Python,
760a7851e9ba rust-parsers: move parser bindings to their own file and Python module
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42746
diff changeset
    46
) -> PyResult<MakeDirstateTupleFn> {
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    47
    unsafe {
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    48
        let caps_name = CStr::from_bytes_with_nul_unchecked(
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    49
            b"mercurial.cext.parsers.make_dirstate_tuple_CAPI\0",
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    50
        );
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    51
        let from_caps = PyCapsule_Import(caps_name.as_ptr(), 0);
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    52
        if from_caps.is_null() {
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    53
            return Err(PyErr::fetch(py));
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    54
        }
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    55
        Ok(transmute(from_caps))
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    56
    }
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    57
}
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    58
42748
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42747
diff changeset
    59
pub fn extract_dirstate(py: Python, dmap: &PyDict) -> Result<StateMap, PyErr> {
42537
ce94f9622acd rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42408
diff changeset
    60
    dmap.items(py)
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    61
        .iter()
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    62
        .map(|(filename, stats)| {
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    63
            let stats = stats.extract::<PySequence>(py)?;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    64
            let state = stats.get_item(py, 0)?.extract::<PyBytes>(py)?;
42749
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42748
diff changeset
    65
            let state = EntryState::try_from(state.data(py)[0]).map_err(
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42748
diff changeset
    66
                |e: DirstateParseError| {
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42748
diff changeset
    67
                    PyErr::new::<exc::ValueError, _>(py, e.to_string())
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42748
diff changeset
    68
                },
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42748
diff changeset
    69
            )?;
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    70
            let mode = stats.get_item(py, 1)?.extract(py)?;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    71
            let size = stats.get_item(py, 2)?.extract(py)?;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    72
            let mtime = stats.get_item(py, 3)?.extract(py)?;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    73
            let filename = filename.extract::<PyBytes>(py)?;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    74
            let filename = filename.data(py);
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    75
            Ok((
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    76
                filename.to_owned(),
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    77
                DirstateEntry {
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    78
                    state,
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    79
                    mode,
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    80
                    size,
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    81
                    mtime,
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    82
                },
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    83
            ))
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    84
        })
42537
ce94f9622acd rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42408
diff changeset
    85
        .collect()
ce94f9622acd rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42408
diff changeset
    86
}
ce94f9622acd rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42408
diff changeset
    87
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    88
/// Create the module, with `__package__` given from parent
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    89
pub fn init_module(py: Python, package: &str) -> PyResult<PyModule> {
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    90
    let dotted_name = &format!("{}.dirstate", package);
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    91
    let m = PyModule::new(py, dotted_name)?;
42537
ce94f9622acd rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42408
diff changeset
    92
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    93
    m.add(py, "__package__", package)?;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    94
    m.add(py, "__doc__", "Dirstate - Rust implementation")?;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    95
42537
ce94f9622acd rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42408
diff changeset
    96
    m.add_class::<Dirs>(py)?;
ce94f9622acd rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42408
diff changeset
    97
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    98
    let sys = PyModule::import(py, "sys")?;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
    99
    let sys_modules: PyDict = sys.get(py, "modules")?.extract(py)?;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
   100
    sys_modules.set_item(py, dotted_name, &m)?;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
   101
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
   102
    Ok(m)
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
   103
}