Mercurial > public > mercurial-scm > hg
annotate rust/hg-cpython/src/dirstate.rs @ 42752:30320c7bf79f
rust-cpython: add macro for sharing references
Following an experiment done by Georges Racinet, we now have a working way of
sharing references between Python and Rust. This is needed in many points of
the codebase, for example every time we need to expose an iterator to a
Rust-backed Python class.
In a few words, references are (unsafely) marked as `'static` and coupled
with manual reference counting; we are doing manual borrow-checking.
This changes introduces two declarative macro to help reduce boilerplate.
While it is better than not using macros, they are not perfect. They need to:
- Integrate with the garbage collector for container types (not needed
as of yet), as stated in the docstring
- Allow for leaking multiple attributes at the same time
- Inject the `py_shared_state` data attribute in `py_class`-generated
structs
- Automatically namespace the functions and attributes they generate
For at least the last two points, we will need to write a procedural macro
instead of a declarative one.
While this reference-sharing mechanism is being ironed out I thought it best
not to implement it yet.
Lastly, and implementation detail renders our Rust-backed Python iterators too
strict to be proper drop-in replacements, as will be illustrated in a future
patch: if the data structure referenced by a non-depleted iterator is mutated,
an `AlreadyBorrowed` exception is raised, whereas Python would allow it, only
to raise a `RuntimeError` if `next` is called on said iterator. This will have
to be addressed at some point.
Differential Revision: https://phab.mercurial-scm.org/D6631
author | Rapha?l Gom?s <rgomes@octobus.net> |
---|---|
date | Tue, 09 Jul 2019 15:15:54 +0200 |
parents | 7ceded4419a3 |
children | 4e8f504424f3 |
rev | line source |
---|---|
42303
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
1 // dirstate.rs |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
2 // |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
3 // Copyright 2019 Raphaël Gomès <rgomes@octobus.net> |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
4 // |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
5 // This software may be used and distributed according to the terms of the |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
6 // GNU General Public License version 2 or any later version. |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
7 |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
8 //! Bindings for the `hg::dirstate` module provided by the |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
9 //! `hg-core` package. |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
10 //! |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
11 //! From Python, this will be seen as `mercurial.rustext.dirstate` |
42746
b3518b0baa47
rust-dirstate: create dirstate submodule in hg-cpython
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42609
diff
changeset
|
12 mod dirs_multiset; |
b3518b0baa47
rust-dirstate: create dirstate submodule in hg-cpython
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42609
diff
changeset
|
13 use crate::dirstate::dirs_multiset::Dirs; |
42303
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
14 use cpython::{ |
42749
7ceded4419a3
rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42748
diff
changeset
|
15 exc, PyBytes, PyDict, PyErr, PyModule, PyObject, PyResult, PySequence, |
7ceded4419a3
rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42748
diff
changeset
|
16 Python, |
42303
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
17 }; |
42749
7ceded4419a3
rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42748
diff
changeset
|
18 use hg::{DirstateEntry, DirstateParseError, EntryState, StateMap}; |
42609
326fdce22fb2
rust: switch hg-core and hg-cpython to rust 2018 edition
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42558
diff
changeset
|
19 use libc::{c_char, c_int}; |
326fdce22fb2
rust: switch hg-core and hg-cpython to rust 2018 edition
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42558
diff
changeset
|
20 #[cfg(feature = "python27")] |
326fdce22fb2
rust: switch hg-core and hg-cpython to rust 2018 edition
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42558
diff
changeset
|
21 use python27_sys::PyCapsule_Import; |
326fdce22fb2
rust: switch hg-core and hg-cpython to rust 2018 edition
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42558
diff
changeset
|
22 #[cfg(feature = "python3")] |
326fdce22fb2
rust: switch hg-core and hg-cpython to rust 2018 edition
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42558
diff
changeset
|
23 use python3_sys::PyCapsule_Import; |
42749
7ceded4419a3
rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42748
diff
changeset
|
24 use std::convert::TryFrom; |
42303
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
25 use std::ffi::CStr; |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
26 use std::mem::transmute; |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
27 |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
28 /// C code uses a custom `dirstate_tuple` type, checks in multiple instances |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
29 /// for this type, and raises a Python `Exception` if the check does not pass. |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
30 /// Because this type differs only in name from the regular Python tuple, it |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
31 /// would be a good idea in the near future to remove it entirely to allow |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
32 /// for a pure Python tuple of the same effective structure to be used, |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
33 /// rendering this type and the capsule below useless. |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
34 type MakeDirstateTupleFn = extern "C" fn( |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
35 state: c_char, |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
36 mode: c_int, |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
37 size: c_int, |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
38 mtime: c_int, |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
39 ) -> PyObject; |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
40 |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
41 /// This is largely a copy/paste from cindex.rs, pending the merge of a |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
42 /// `py_capsule_fn!` macro in the rust-cpython project: |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
43 /// https://github.com/dgrunwald/rust-cpython/pull/169 |
42747
760a7851e9ba
rust-parsers: move parser bindings to their own file and Python module
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42746
diff
changeset
|
44 pub fn decapsule_make_dirstate_tuple( |
760a7851e9ba
rust-parsers: move parser bindings to their own file and Python module
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42746
diff
changeset
|
45 py: Python, |
760a7851e9ba
rust-parsers: move parser bindings to their own file and Python module
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42746
diff
changeset
|
46 ) -> PyResult<MakeDirstateTupleFn> { |
42303
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
47 unsafe { |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
48 let caps_name = CStr::from_bytes_with_nul_unchecked( |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
49 b"mercurial.cext.parsers.make_dirstate_tuple_CAPI\0", |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
50 ); |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
51 let from_caps = PyCapsule_Import(caps_name.as_ptr(), 0); |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
52 if from_caps.is_null() { |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
53 return Err(PyErr::fetch(py)); |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
54 } |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
55 Ok(transmute(from_caps)) |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
56 } |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
57 } |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
58 |
42748
7cae6bc29ff9
rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42747
diff
changeset
|
59 pub fn extract_dirstate(py: Python, dmap: &PyDict) -> Result<StateMap, PyErr> { |
42537
ce94f9622acd
rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42408
diff
changeset
|
60 dmap.items(py) |
42303
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
61 .iter() |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
62 .map(|(filename, stats)| { |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
63 let stats = stats.extract::<PySequence>(py)?; |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
64 let state = stats.get_item(py, 0)?.extract::<PyBytes>(py)?; |
42749
7ceded4419a3
rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42748
diff
changeset
|
65 let state = EntryState::try_from(state.data(py)[0]).map_err( |
7ceded4419a3
rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42748
diff
changeset
|
66 |e: DirstateParseError| { |
7ceded4419a3
rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42748
diff
changeset
|
67 PyErr::new::<exc::ValueError, _>(py, e.to_string()) |
7ceded4419a3
rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42748
diff
changeset
|
68 }, |
7ceded4419a3
rust-dirstate: use EntryState enum instead of literals
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42748
diff
changeset
|
69 )?; |
42303
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
70 let mode = stats.get_item(py, 1)?.extract(py)?; |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
71 let size = stats.get_item(py, 2)?.extract(py)?; |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
72 let mtime = stats.get_item(py, 3)?.extract(py)?; |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
73 let filename = filename.extract::<PyBytes>(py)?; |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
74 let filename = filename.data(py); |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
75 Ok(( |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
76 filename.to_owned(), |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
77 DirstateEntry { |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
78 state, |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
79 mode, |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
80 size, |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
81 mtime, |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
82 }, |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
83 )) |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
84 }) |
42537
ce94f9622acd
rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42408
diff
changeset
|
85 .collect() |
ce94f9622acd
rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42408
diff
changeset
|
86 } |
ce94f9622acd
rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42408
diff
changeset
|
87 |
42303
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
88 /// Create the module, with `__package__` given from parent |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
89 pub fn init_module(py: Python, package: &str) -> PyResult<PyModule> { |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
90 let dotted_name = &format!("{}.dirstate", package); |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
91 let m = PyModule::new(py, dotted_name)?; |
42537
ce94f9622acd
rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42408
diff
changeset
|
92 |
42303
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
93 m.add(py, "__package__", package)?; |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
94 m.add(py, "__doc__", "Dirstate - Rust implementation")?; |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
95 |
42537
ce94f9622acd
rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42408
diff
changeset
|
96 m.add_class::<Dirs>(py)?; |
ce94f9622acd
rust-dirstate: add "dirs" rust-cpython binding
Rapha?l Gom?s <rgomes@octobus.net>
parents:
42408
diff
changeset
|
97 |
42303
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
98 let sys = PyModule::import(py, "sys")?; |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
99 let sys_modules: PyDict = sys.get(py, "modules")?.extract(py)?; |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
100 sys_modules.set_item(py, dotted_name, &m)?; |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
101 |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
102 Ok(m) |
e240bec26626
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
Rapha?l Gom?s <rgomes@octobus.net>
parents:
diff
changeset
|
103 } |