annotate rust/hg-cpython/src/dirstate.rs @ 42752:30320c7bf79f

rust-cpython: add macro for sharing references Following an experiment done by Georges Racinet, we now have a working way of sharing references between Python and Rust. This is needed in many points of the codebase, for example every time we need to expose an iterator to a Rust-backed Python class. In a few words, references are (unsafely) marked as `'static` and coupled with manual reference counting; we are doing manual borrow-checking. This changes introduces two declarative macro to help reduce boilerplate. While it is better than not using macros, they are not perfect. They need to: - Integrate with the garbage collector for container types (not needed as of yet), as stated in the docstring - Allow for leaking multiple attributes at the same time - Inject the `py_shared_state` data attribute in `py_class`-generated structs - Automatically namespace the functions and attributes they generate For at least the last two points, we will need to write a procedural macro instead of a declarative one. While this reference-sharing mechanism is being ironed out I thought it best not to implement it yet. Lastly, and implementation detail renders our Rust-backed Python iterators too strict to be proper drop-in replacements, as will be illustrated in a future patch: if the data structure referenced by a non-depleted iterator is mutated, an `AlreadyBorrowed` exception is raised, whereas Python would allow it, only to raise a `RuntimeError` if `next` is called on said iterator. This will have to be addressed at some point. Differential Revision: https://phab.mercurial-scm.org/D6631
author Rapha?l Gom?s <rgomes@octobus.net>
date Tue, 09 Jul 2019 15:15:54 +0200
parents 7ceded4419a3
children 4e8f504424f3
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
1 // dirstate.rs
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
2 //
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
3 // Copyright 2019 Raphaël Gomès <rgomes@octobus.net>
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
4 //
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
5 // This software may be used and distributed according to the terms of the
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
6 // GNU General Public License version 2 or any later version.
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
7
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
8 //! Bindings for the `hg::dirstate` module provided by the
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
9 //! `hg-core` package.
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
10 //!
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
11 //! From Python, this will be seen as `mercurial.rustext.dirstate`
42746
b3518b0baa47 rust-dirstate: create dirstate submodule in hg-cpython
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42609
diff changeset
12 mod dirs_multiset;
b3518b0baa47 rust-dirstate: create dirstate submodule in hg-cpython
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42609
diff changeset
13 use crate::dirstate::dirs_multiset::Dirs;
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
14 use cpython::{
42749
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42748
diff changeset
15 exc, PyBytes, PyDict, PyErr, PyModule, PyObject, PyResult, PySequence,
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42748
diff changeset
16 Python,
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
17 };
42749
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42748
diff changeset
18 use hg::{DirstateEntry, DirstateParseError, EntryState, StateMap};
42609
326fdce22fb2 rust: switch hg-core and hg-cpython to rust 2018 edition
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42558
diff changeset
19 use libc::{c_char, c_int};
326fdce22fb2 rust: switch hg-core and hg-cpython to rust 2018 edition
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42558
diff changeset
20 #[cfg(feature = "python27")]
326fdce22fb2 rust: switch hg-core and hg-cpython to rust 2018 edition
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42558
diff changeset
21 use python27_sys::PyCapsule_Import;
326fdce22fb2 rust: switch hg-core and hg-cpython to rust 2018 edition
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42558
diff changeset
22 #[cfg(feature = "python3")]
326fdce22fb2 rust: switch hg-core and hg-cpython to rust 2018 edition
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42558
diff changeset
23 use python3_sys::PyCapsule_Import;
42749
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42748
diff changeset
24 use std::convert::TryFrom;
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
25 use std::ffi::CStr;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
26 use std::mem::transmute;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
27
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
28 /// C code uses a custom `dirstate_tuple` type, checks in multiple instances
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
29 /// for this type, and raises a Python `Exception` if the check does not pass.
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
30 /// Because this type differs only in name from the regular Python tuple, it
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
31 /// would be a good idea in the near future to remove it entirely to allow
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
32 /// for a pure Python tuple of the same effective structure to be used,
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
33 /// rendering this type and the capsule below useless.
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
34 type MakeDirstateTupleFn = extern "C" fn(
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
35 state: c_char,
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
36 mode: c_int,
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
37 size: c_int,
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
38 mtime: c_int,
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
39 ) -> PyObject;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
40
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
41 /// This is largely a copy/paste from cindex.rs, pending the merge of a
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
42 /// `py_capsule_fn!` macro in the rust-cpython project:
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
43 /// https://github.com/dgrunwald/rust-cpython/pull/169
42747
760a7851e9ba rust-parsers: move parser bindings to their own file and Python module
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42746
diff changeset
44 pub fn decapsule_make_dirstate_tuple(
760a7851e9ba rust-parsers: move parser bindings to their own file and Python module
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42746
diff changeset
45 py: Python,
760a7851e9ba rust-parsers: move parser bindings to their own file and Python module
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42746
diff changeset
46 ) -> PyResult<MakeDirstateTupleFn> {
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
47 unsafe {
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
48 let caps_name = CStr::from_bytes_with_nul_unchecked(
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
49 b"mercurial.cext.parsers.make_dirstate_tuple_CAPI\0",
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
50 );
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
51 let from_caps = PyCapsule_Import(caps_name.as_ptr(), 0);
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
52 if from_caps.is_null() {
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
53 return Err(PyErr::fetch(py));
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
54 }
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
55 Ok(transmute(from_caps))
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
56 }
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
57 }
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
58
42748
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42747
diff changeset
59 pub fn extract_dirstate(py: Python, dmap: &PyDict) -> Result<StateMap, PyErr> {
42537
ce94f9622acd rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42408
diff changeset
60 dmap.items(py)
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
61 .iter()
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
62 .map(|(filename, stats)| {
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
63 let stats = stats.extract::<PySequence>(py)?;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
64 let state = stats.get_item(py, 0)?.extract::<PyBytes>(py)?;
42749
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42748
diff changeset
65 let state = EntryState::try_from(state.data(py)[0]).map_err(
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42748
diff changeset
66 |e: DirstateParseError| {
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42748
diff changeset
67 PyErr::new::<exc::ValueError, _>(py, e.to_string())
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42748
diff changeset
68 },
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42748
diff changeset
69 )?;
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
70 let mode = stats.get_item(py, 1)?.extract(py)?;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
71 let size = stats.get_item(py, 2)?.extract(py)?;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
72 let mtime = stats.get_item(py, 3)?.extract(py)?;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
73 let filename = filename.extract::<PyBytes>(py)?;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
74 let filename = filename.data(py);
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
75 Ok((
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
76 filename.to_owned(),
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
77 DirstateEntry {
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
78 state,
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
79 mode,
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
80 size,
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
81 mtime,
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
82 },
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
83 ))
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
84 })
42537
ce94f9622acd rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42408
diff changeset
85 .collect()
ce94f9622acd rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42408
diff changeset
86 }
ce94f9622acd rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42408
diff changeset
87
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
88 /// Create the module, with `__package__` given from parent
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
89 pub fn init_module(py: Python, package: &str) -> PyResult<PyModule> {
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
90 let dotted_name = &format!("{}.dirstate", package);
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
91 let m = PyModule::new(py, dotted_name)?;
42537
ce94f9622acd rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42408
diff changeset
92
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
93 m.add(py, "__package__", package)?;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
94 m.add(py, "__doc__", "Dirstate - Rust implementation")?;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
95
42537
ce94f9622acd rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42408
diff changeset
96 m.add_class::<Dirs>(py)?;
ce94f9622acd rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents: 42408
diff changeset
97
42303
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
98 let sys = PyModule::import(py, "sys")?;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
99 let sys_modules: PyDict = sys.get(py, "modules")?.extract(py)?;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
100 sys_modules.set_item(py, dotted_name, &m)?;
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
101
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
102 Ok(m)
e240bec26626 rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff changeset
103 }