annotate mercurial/obsolete.py @ 37060:2ec1fb9de638

wireproto: add request IDs to frames One of my primary goals with the new wire protocol is to make operations faster and enable both client and server-side operations to scale to multiple CPU cores. One of the ways we make server interactions faster is by reducing the number of round trips to that server. With the existing wire protocol, the "batch" command facilitates executing multiple commands from a single request payload. The way it works is the requests for multiple commands are serialized. The server executes those commands sequentially then serializes all their results. As an optimization for reducing round trips, this is very effective. The technical implementation, however, is pretty bad and suffers from a number of deficiencies. For example, it creates a new place where authorization to run a command must be checked. (The lack of this checking in older Mercurial releases was CVE-2018-1000132.) The principles behind the "batch" command are sound. However, the execution is not. Therefore, I want to ditch "batch" in the new wire protocol and have protocol level support for issuing multiple requests in a single round trip. This commit introduces support in the frame-based wire protocol to facilitate this. We do this by adding a "request ID" to each frame. If a server sees frames associated with different "request IDs," it handles them as separate requests. All of this happening possibly as part of the same message from client to server (the same request body in the case of HTTP). We /could/ model the exchange the way pipelined HTTP requests do, where the server processes requests in order they are issued and received. But this artifically constrains scalability. A better model is to allow multi-requests to be executed concurrently and for responses to be sent and handled concurrently. So the specification explicitly allows this. There is some work to be done around specifying dependencies between multi-requests. We take the easy road for now and punt on this problem, declaring that if order is important, clients must not issue the request until responses to dependent requests have been received. This commit focuses on the boilerplate of implementing the request ID. The server reactor still can't manage multiple, in-flight request IDs. This will be addressed in a subsequent commit. Because the wire semantics have changed, we bump the version of the media type. Differential Revision: https://phab.mercurial-scm.org/D2869
author Gregory Szorc <gregory.szorc@gmail.com>
date Wed, 14 Mar 2018 16:51:34 -0700
parents 98c14e857e71
children d30810d09d6f
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
1 # obsolete.py - obsolete markers handling
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
2 #
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
3 # Copyright 2012 Pierre-Yves David <pierre-yves.david@ens-lyon.org>
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
4 # Logilab SA <contact@logilab.fr>
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
5 #
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
6 # This software may be used and distributed according to the terms of the
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
7 # GNU General Public License version 2 or any later version.
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
8
21164
2efcef493aa2 obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents: 21098
diff changeset
9 """Obsolete marker handling
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
10
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
11 An obsolete marker maps an old changeset to a list of new
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
12 changesets. If the list of new changesets is empty, the old changeset
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
13 is said to be "killed". Otherwise, the old changeset is being
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
14 "replaced" by the new changesets.
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
15
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
16 Obsolete markers can be used to record and distribute changeset graph
21164
2efcef493aa2 obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents: 21098
diff changeset
17 transformations performed by history rewrite operations, and help
2efcef493aa2 obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents: 21098
diff changeset
18 building new tools to reconcile conflicting rewrite actions. To
2efcef493aa2 obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents: 21098
diff changeset
19 facilitate conflict resolution, markers include various annotations
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
20 besides old and news changeset identifiers, such as creation date or
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
21 author name.
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
22
33760
0c3112f17610 obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents: 33759
diff changeset
23 The old obsoleted changeset is called a "predecessor" and possible
21164
2efcef493aa2 obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents: 21098
diff changeset
24 replacements are called "successors". Markers that used changeset X as
33760
0c3112f17610 obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents: 33759
diff changeset
25 a predecessor are called "successor markers of X" because they hold
21164
2efcef493aa2 obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents: 21098
diff changeset
26 information about the successors of X. Markers that use changeset Y as
33760
0c3112f17610 obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents: 33759
diff changeset
27 a successors are call "predecessor markers of Y" because they hold
0c3112f17610 obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents: 33759
diff changeset
28 information about the predecessors of Y.
17776
072812e9f570 obsolete: flip `obstore.successors` and `obsolete.precursors`
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17775
diff changeset
29
17775
13744acc4ad7 obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17774
diff changeset
30 Examples:
13744acc4ad7 obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17774
diff changeset
31
21164
2efcef493aa2 obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents: 21098
diff changeset
32 - When changeset A is replaced by changeset A', one marker is stored:
17775
13744acc4ad7 obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17774
diff changeset
33
21166
bf2891877378 obsolete: fix one-element tuple in module docstring
Martin Geisler <martin@geisler.net>
parents: 21165
diff changeset
34 (A, (A',))
17775
13744acc4ad7 obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17774
diff changeset
35
21164
2efcef493aa2 obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents: 21098
diff changeset
36 - When changesets A and B are folded into a new changeset C, two markers are
17775
13744acc4ad7 obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17774
diff changeset
37 stored:
13744acc4ad7 obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17774
diff changeset
38
13744acc4ad7 obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17774
diff changeset
39 (A, (C,)) and (B, (C,))
13744acc4ad7 obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17774
diff changeset
40
21164
2efcef493aa2 obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents: 21098
diff changeset
41 - When changeset A is simply "pruned" from the graph, a marker is created:
17775
13744acc4ad7 obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17774
diff changeset
42
13744acc4ad7 obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17774
diff changeset
43 (A, ())
13744acc4ad7 obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17774
diff changeset
44
29898
02c150850e16 obsolete: fix minor module documentation issues
liscju <piotr.listkiewicz@gmail.com>
parents: 29389
diff changeset
45 - When changeset A is split into B and C, a single marker is used:
17775
13744acc4ad7 obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17774
diff changeset
46
29898
02c150850e16 obsolete: fix minor module documentation issues
liscju <piotr.listkiewicz@gmail.com>
parents: 29389
diff changeset
47 (A, (B, C))
17775
13744acc4ad7 obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17774
diff changeset
48
21164
2efcef493aa2 obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents: 21098
diff changeset
49 We use a single marker to distinguish the "split" case from the "divergence"
2efcef493aa2 obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents: 21098
diff changeset
50 case. If two independent operations rewrite the same changeset A in to A' and
2efcef493aa2 obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents: 21098
diff changeset
51 A'', we have an error case: divergent rewriting. We can detect it because
17775
13744acc4ad7 obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17774
diff changeset
52 two markers will be created independently:
13744acc4ad7 obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17774
diff changeset
53
13744acc4ad7 obsolete: add example of marker usage in the documentation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17774
diff changeset
54 (A, (B,)) and (A, (C,))
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
55
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
56 Format
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
57 ------
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
58
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
59 Markers are stored in an append-only file stored in
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
60 '.hg/store/obsstore'.
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
61
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
62 The file starts with a version header:
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
63
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
64 - 1 unsigned byte: version number, starting at zero.
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
65
22612
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
66 The header is followed by the markers. Marker format depend of the version. See
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
67 comment associated with each format for details.
21164
2efcef493aa2 obsolete: fix language and grammar in module docstring
Martin Geisler <martin@geisler.net>
parents: 21098
diff changeset
68
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
69 """
27332
04f346b8768f obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26684
diff changeset
70 from __future__ import absolute_import
04f346b8768f obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26684
diff changeset
71
04f346b8768f obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26684
diff changeset
72 import errno
04f346b8768f obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26684
diff changeset
73 import struct
04f346b8768f obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26684
diff changeset
74
04f346b8768f obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26684
diff changeset
75 from .i18n import _
04f346b8768f obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26684
diff changeset
76 from . import (
04f346b8768f obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26684
diff changeset
77 error,
04f346b8768f obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26684
diff changeset
78 node,
33148
4f49810a1011 obsutil: move 'successorssets' to the new modules
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33137
diff changeset
79 obsutil,
27332
04f346b8768f obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26684
diff changeset
80 phases,
32411
df448de7cf3b parsers: switch to policy importer
Yuya Nishihara <yuya@tcha.org>
parents: 32393
diff changeset
81 policy,
27332
04f346b8768f obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26684
diff changeset
82 util,
04f346b8768f obsolete: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26684
diff changeset
83 )
36636
c6061cadb400 util: extract all date-related utils in utils/dateutil module
Boris Feld <boris.feld@octobus.net>
parents: 36453
diff changeset
84 from .utils import dateutil
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
85
32411
df448de7cf3b parsers: switch to policy importer
Yuya Nishihara <yuya@tcha.org>
parents: 32393
diff changeset
86 parsers = policy.importmod(r'parsers')
df448de7cf3b parsers: switch to policy importer
Yuya Nishihara <yuya@tcha.org>
parents: 32393
diff changeset
87
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
88 _pack = struct.pack
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
89 _unpack = struct.unpack
23498
ac910b1f5658 obsstore: prefetch struct.calcsize
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23497
diff changeset
90 _calcsize = struct.calcsize
24046
6e1d9f9932a9 obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents: 24045
diff changeset
91 propertycache = util.propertycache
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
92
17424
e7cfe3587ea4 fix trivial spelling errors
Mads Kiilerich <mads@kiilerich.com>
parents: 17405
diff changeset
93 # the obsolete feature is not mature enough to be enabled by default.
17296
a1f8869f2eee obsolete: introduce an `_enabled` switch to disable the feature by default
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17295
diff changeset
94 # you have to rely on third party extension extension to enable this.
a1f8869f2eee obsolete: introduce an `_enabled` switch to disable the feature by default
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17295
diff changeset
95 _enabled = False
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
96
22951
6c86c673dde6 obsolete: add createmarkers option
Durham Goode <durham@fb.com>
parents: 22950
diff changeset
97 # Options for obsolescence
6c86c673dde6 obsolete: add createmarkers option
Durham Goode <durham@fb.com>
parents: 22950
diff changeset
98 createmarkersopt = 'createmarkers'
22952
8792ac090e3b obsolete: add allowunstable option
Durham Goode <durham@fb.com>
parents: 22951
diff changeset
99 allowunstableopt = 'allowunstable'
22953
b1d694d3975e obsolete: add exchange option
Durham Goode <durham@fb.com>
parents: 22952
diff changeset
100 exchangeopt = 'exchange'
22951
6c86c673dde6 obsolete: add createmarkers option
Durham Goode <durham@fb.com>
parents: 22950
diff changeset
101
34864
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
102 def _getoptionvalue(repo, option):
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
103 """Returns True if the given repository has the given obsolete option
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
104 enabled.
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
105 """
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
106 configkey = 'evolution.%s' % option
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
107 newconfig = repo.ui.configbool('experimental', configkey)
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
108
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
109 # Return the value only if defined
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
110 if newconfig is not None:
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
111 return newconfig
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
112
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
113 # Fallback on generic option
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
114 try:
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
115 return repo.ui.configbool('experimental', 'evolution')
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
116 except (error.ConfigError, AttributeError):
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
117 # Fallback on old-fashion config
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
118 # inconsistent config: experimental.evolution
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
119 result = set(repo.ui.configlist('experimental', 'evolution'))
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
120
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
121 if 'all' in result:
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
122 return True
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
123
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
124 # For migration purposes, temporarily return true if the config hasn't
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
125 # been set but _enabled is true.
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
126 if len(result) == 0 and _enabled:
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
127 return True
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
128
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
129 # Temporary hack for next check
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
130 newconfig = repo.ui.config('experimental', 'evolution.createmarkers')
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
131 if newconfig:
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
132 result.add('createmarkers')
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
133
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
134 return option in result
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
135
32372
566cfe9cbbb9 obsolete: move the 'isenabled' function at the top of the file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32366
diff changeset
136 def isenabled(repo, option):
566cfe9cbbb9 obsolete: move the 'isenabled' function at the top of the file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32366
diff changeset
137 """Returns True if the given repository has the given obsolete option
566cfe9cbbb9 obsolete: move the 'isenabled' function at the top of the file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32366
diff changeset
138 enabled.
566cfe9cbbb9 obsolete: move the 'isenabled' function at the top of the file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32366
diff changeset
139 """
34864
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
140 createmarkersvalue = _getoptionvalue(repo, createmarkersopt)
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
141 unstabluevalue = _getoptionvalue(repo, allowunstableopt)
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
142 exchangevalue = _getoptionvalue(repo, exchangeopt)
32372
566cfe9cbbb9 obsolete: move the 'isenabled' function at the top of the file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32366
diff changeset
143
566cfe9cbbb9 obsolete: move the 'isenabled' function at the top of the file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32366
diff changeset
144 # createmarkers must be enabled if other options are enabled
34864
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
145 if ((unstabluevalue or exchangevalue) and not createmarkersvalue):
32372
566cfe9cbbb9 obsolete: move the 'isenabled' function at the top of the file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32366
diff changeset
146 raise error.Abort(_("'createmarkers' obsolete option must be enabled "
34864
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
147 "if other obsolete options are enabled"))
32372
566cfe9cbbb9 obsolete: move the 'isenabled' function at the top of the file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32366
diff changeset
148
34864
fec79a3f250f config: update evolution-related config
Boris Feld <boris.feld@octobus.net>
parents: 34863
diff changeset
149 return _getoptionvalue(repo, option)
32372
566cfe9cbbb9 obsolete: move the 'isenabled' function at the top of the file
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32366
diff changeset
150
37020
98c14e857e71 obsolete: add a note that explains creating aliases for marker flags
Anton Shestakov <av6@dwimlabs.net>
parents: 36959
diff changeset
151 # Creating aliases for marker flags because evolve extension looks for
98c14e857e71 obsolete: add a note that explains creating aliases for marker flags
Anton Shestakov <av6@dwimlabs.net>
parents: 36959
diff changeset
152 # bumpedfix in obsolete.py
36959
b9bbcf9ffac1 obsolete: move marker flags to obsutil
Anton Shestakov <av6@dwimlabs.net>
parents: 36636
diff changeset
153 bumpedfix = obsutil.bumpedfix
b9bbcf9ffac1 obsolete: move marker flags to obsutil
Anton Shestakov <av6@dwimlabs.net>
parents: 36636
diff changeset
154 usingsha256 = obsutil.usingsha256
17831
70b08df24fef obsolete: add a flag that allows fixing "bumped" changeset
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17828
diff changeset
155
22612
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
156 ## Parsing and writing of version "0"
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
157 #
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
158 # The header is followed by the markers. Each marker is made of:
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
159 #
22849
fd759142c6e5 obsolete: use uint## in the format documention
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22848
diff changeset
160 # - 1 uint8 : number of new changesets "N", can be zero.
22612
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
161 #
22849
fd759142c6e5 obsolete: use uint## in the format documention
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22848
diff changeset
162 # - 1 uint32: metadata size "M" in bytes.
22612
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
163 #
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
164 # - 1 byte: a bit field. It is reserved for flags used in common
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
165 # obsolete marker operations, to avoid repeated decoding of metadata
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
166 # entries.
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
167 #
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
168 # - 20 bytes: obsoleted changeset identifier.
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
169 #
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
170 # - N*20 bytes: new changesets identifiers.
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
171 #
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
172 # - M bytes: metadata as a sequence of nul-terminated strings. Each
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
173 # string contains a key and a value, separated by a colon ':', without
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
174 # additional encoding. Keys cannot contain '\0' or ':' and values
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
175 # cannot contain '\0'.
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
176 _fm0version = 0
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
177 _fm0fixed = '>BIB20s'
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
178 _fm0node = '20s'
23498
ac910b1f5658 obsstore: prefetch struct.calcsize
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23497
diff changeset
179 _fm0fsize = _calcsize(_fm0fixed)
ac910b1f5658 obsstore: prefetch struct.calcsize
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23497
diff changeset
180 _fm0fnodesize = _calcsize(_fm0node)
22334
f2c3cfc30e9c obsolete: move _encodemarkers next to _readmarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22333
diff changeset
181
33504
5d3ba4395288 obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents: 33499
diff changeset
182 def _fm0readmarkers(data, off, stop):
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
183 # Loop on markers
33504
5d3ba4395288 obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents: 33499
diff changeset
184 while off < stop:
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
185 # read fixed part
22327
f737631a9f0a obsolete: rename all _fm to _fm0
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22326
diff changeset
186 cur = data[off:off + _fm0fsize]
f737631a9f0a obsolete: rename all _fm to _fm0
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22326
diff changeset
187 off += _fm0fsize
22685
753515d9e274 obsolete: replace "nb" notation with "num" (for "number")
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22614
diff changeset
188 numsuc, mdsize, flags, pre = _unpack(_fm0fixed, cur)
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
189 # read replacement
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
190 sucs = ()
22685
753515d9e274 obsolete: replace "nb" notation with "num" (for "number")
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22614
diff changeset
191 if numsuc:
753515d9e274 obsolete: replace "nb" notation with "num" (for "number")
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22614
diff changeset
192 s = (_fm0fnodesize * numsuc)
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
193 cur = data[off:off + s]
22685
753515d9e274 obsolete: replace "nb" notation with "num" (for "number")
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22614
diff changeset
194 sucs = _unpack(_fm0node * numsuc, cur)
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
195 off += s
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
196 # read metadata
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
197 # (metadata will be decoded on demand)
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
198 metadata = data[off:off + mdsize]
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
199 if len(metadata) != mdsize:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26310
diff changeset
200 raise error.Abort(_('parsing obsolete marker: metadata is too '
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
201 'short, %d bytes expected, got %d')
17253
67f56ff5afcd obsolete: fix decoding error message arguments
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17220
diff changeset
202 % (mdsize, len(metadata)))
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
203 off += mdsize
22847
37460ee2003c obsolete: _rename decodemeta to _fm0decodemeta
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22846
diff changeset
204 metadata = _fm0decodemeta(metadata)
22222
883e8b6bd461 obsmarker: add `date` as an explicit field
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22221
diff changeset
205 try:
22845
ef880e28e56a obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22685
diff changeset
206 when, offset = metadata.pop('date', '0 0').split(' ')
22309
a65697c3f20e obsolete: avoid slow, generic date parsing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22274
diff changeset
207 date = float(when), int(offset)
a65697c3f20e obsolete: avoid slow, generic date parsing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22274
diff changeset
208 except ValueError:
22222
883e8b6bd461 obsmarker: add `date` as an explicit field
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22221
diff changeset
209 date = (0., 0)
22258
cf414981978a obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22257
diff changeset
210 parents = None
22845
ef880e28e56a obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22685
diff changeset
211 if 'p2' in metadata:
ef880e28e56a obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22685
diff changeset
212 parents = (metadata.pop('p1', None), metadata.pop('p2', None))
ef880e28e56a obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22685
diff changeset
213 elif 'p1' in metadata:
ef880e28e56a obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22685
diff changeset
214 parents = (metadata.pop('p1', None),)
ef880e28e56a obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22685
diff changeset
215 elif 'p0' in metadata:
22258
cf414981978a obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22257
diff changeset
216 parents = ()
cf414981978a obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22257
diff changeset
217 if parents is not None:
cf414981978a obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22257
diff changeset
218 try:
cf414981978a obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22257
diff changeset
219 parents = tuple(node.bin(p) for p in parents)
cf414981978a obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22257
diff changeset
220 # if parent content is not a nodeid, drop the data
cf414981978a obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22257
diff changeset
221 for p in parents:
cf414981978a obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22257
diff changeset
222 if len(p) != 20:
cf414981978a obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22257
diff changeset
223 parents = None
cf414981978a obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22257
diff changeset
224 break
cf414981978a obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22257
diff changeset
225 except TypeError:
cf414981978a obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22257
diff changeset
226 # if content cannot be translated to nodeid drop the data.
cf414981978a obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22257
diff changeset
227 parents = None
cf414981978a obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22257
diff changeset
228
22845
ef880e28e56a obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22685
diff changeset
229 metadata = tuple(sorted(metadata.iteritems()))
22222
883e8b6bd461 obsmarker: add `date` as an explicit field
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22221
diff changeset
230
22258
cf414981978a obsstore: also store the 'parents' field on disk
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22257
diff changeset
231 yield (pre, sucs, flags, metadata, date, parents)
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
232
22330
e74f8a65252d obsolete: move _fm0encodeonemarker next to _fm0readmarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22329
diff changeset
233 def _fm0encodeonemarker(marker):
e74f8a65252d obsolete: move _fm0encodeonemarker next to _fm0readmarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22329
diff changeset
234 pre, sucs, flags, metadata, date, parents = marker
22850
b078e4dc9f9a obsstore: add a flag for sha256 hashes
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22849
diff changeset
235 if flags & usingsha256:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26310
diff changeset
236 raise error.Abort(_('cannot handle sha256 with old obsstore format'))
22845
ef880e28e56a obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22685
diff changeset
237 metadata = dict(metadata)
23002
2920a96f5839 obsstore: record data as floating point in fm0 format
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22954
diff changeset
238 time, tz = date
2920a96f5839 obsstore: record data as floating point in fm0 format
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22954
diff changeset
239 metadata['date'] = '%r %i' % (time, tz)
22330
e74f8a65252d obsolete: move _fm0encodeonemarker next to _fm0readmarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22329
diff changeset
240 if parents is not None:
e74f8a65252d obsolete: move _fm0encodeonemarker next to _fm0readmarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22329
diff changeset
241 if not parents:
e74f8a65252d obsolete: move _fm0encodeonemarker next to _fm0readmarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22329
diff changeset
242 # mark that we explicitly recorded no parents
e74f8a65252d obsolete: move _fm0encodeonemarker next to _fm0readmarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22329
diff changeset
243 metadata['p0'] = ''
32318
7c3ef55dedbe obsolete: use 2 argument form of enumerate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32245
diff changeset
244 for i, p in enumerate(parents, 1):
7c3ef55dedbe obsolete: use 2 argument form of enumerate()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32245
diff changeset
245 metadata['p%i' % i] = node.hex(p)
22846
b1efc4893da4 obsolete: _rename encodemeta to _fm0encodemeta
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22845
diff changeset
246 metadata = _fm0encodemeta(metadata)
22685
753515d9e274 obsolete: replace "nb" notation with "num" (for "number")
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22614
diff changeset
247 numsuc = len(sucs)
753515d9e274 obsolete: replace "nb" notation with "num" (for "number")
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22614
diff changeset
248 format = _fm0fixed + (_fm0node * numsuc)
753515d9e274 obsolete: replace "nb" notation with "num" (for "number")
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22614
diff changeset
249 data = [numsuc, len(metadata), flags, pre]
22330
e74f8a65252d obsolete: move _fm0encodeonemarker next to _fm0readmarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22329
diff changeset
250 data.extend(sucs)
e74f8a65252d obsolete: move _fm0encodeonemarker next to _fm0readmarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22329
diff changeset
251 return _pack(format, *data) + metadata
e74f8a65252d obsolete: move _fm0encodeonemarker next to _fm0readmarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22329
diff changeset
252
22848
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
253 def _fm0encodemeta(meta):
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
254 """Return encoded metadata string to string mapping.
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
255
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
256 Assume no ':' in key and no '\0' in both key and value."""
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
257 for key, value in meta.iteritems():
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
258 if ':' in key or '\0' in key:
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
259 raise ValueError("':' and '\0' are forbidden in metadata key'")
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
260 if '\0' in value:
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
261 raise ValueError("':' is forbidden in metadata value'")
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
262 return '\0'.join(['%s:%s' % (k, meta[k]) for k in sorted(meta)])
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
263
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
264 def _fm0decodemeta(data):
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
265 """Return string to string dictionary from encoded version."""
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
266 d = {}
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
267 for l in data.split('\0'):
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
268 if l:
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
269 key, value = l.split(':')
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
270 d[key] = value
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
271 return d
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
272
22851
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
273 ## Parsing and writing of version "1"
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
274 #
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
275 # The header is followed by the markers. Each marker is made of:
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
276 #
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
277 # - uint32: total size of the marker (including this field)
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
278 #
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
279 # - float64: date in seconds since epoch
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
280 #
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
281 # - int16: timezone offset in minutes
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
282 #
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
283 # - uint16: a bit field. It is reserved for flags used in common
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
284 # obsolete marker operations, to avoid repeated decoding of metadata
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
285 # entries.
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
286 #
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
287 # - uint8: number of successors "N", can be zero.
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
288 #
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
289 # - uint8: number of parents "P", can be zero.
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
290 #
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
291 # 0: parents data stored but no parent,
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
292 # 1: one parent stored,
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
293 # 2: two parents stored,
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
294 # 3: no parent data stored
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
295 #
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
296 # - uint8: number of metadata entries M
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
297 #
33760
0c3112f17610 obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents: 33759
diff changeset
298 # - 20 or 32 bytes: predecessor changeset identifier.
22851
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
299 #
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
300 # - N*(20 or 32) bytes: successors changesets identifiers.
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
301 #
33760
0c3112f17610 obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents: 33759
diff changeset
302 # - P*(20 or 32) bytes: parents of the predecessors changesets.
22851
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
303 #
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
304 # - M*(uint8, uint8): size of all metadata entries (key and value)
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
305 #
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
306 # - remaining bytes: the metadata, each (key, value) pair after the other.
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
307 _fm1version = 1
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
308 _fm1fixed = '>IdhHBBB20s'
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
309 _fm1nodesha1 = '20s'
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
310 _fm1nodesha256 = '32s'
23499
b46876c94a93 obsstore: cache size computation for fm1 node
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23498
diff changeset
311 _fm1nodesha1size = _calcsize(_fm1nodesha1)
b46876c94a93 obsstore: cache size computation for fm1 node
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23498
diff changeset
312 _fm1nodesha256size = _calcsize(_fm1nodesha256)
23498
ac910b1f5658 obsstore: prefetch struct.calcsize
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23497
diff changeset
313 _fm1fsize = _calcsize(_fm1fixed)
22851
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
314 _fm1parentnone = 3
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
315 _fm1parentshift = 14
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
316 _fm1parentmask = (_fm1parentnone << _fm1parentshift)
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
317 _fm1metapair = 'BB'
33696
e5f8c7baa729 obsolete: reuse _fm1metapair for computing _fm1metapairsize
Augie Fackler <augie@google.com>
parents: 33504
diff changeset
318 _fm1metapairsize = _calcsize(_fm1metapair)
22851
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
319
33504
5d3ba4395288 obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents: 33499
diff changeset
320 def _fm1purereadmarkers(data, off, stop):
23803
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
321 # make some global constants local for performance
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
322 noneflag = _fm1parentnone
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
323 sha2flag = usingsha256
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
324 sha1size = _fm1nodesha1size
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
325 sha2size = _fm1nodesha256size
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
326 sha1fmt = _fm1nodesha1
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
327 sha2fmt = _fm1nodesha256
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
328 metasize = _fm1metapairsize
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
329 metafmt = _fm1metapair
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
330 fsize = _fm1fsize
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
331 unpack = _unpack
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
332
22851
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
333 # Loop on markers
25211
22f4ce49044d util: drop the 'unpacker' helper
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25149
diff changeset
334 ufixed = struct.Struct(_fm1fixed).unpack
24018
26d6a6a78c1d obsolete: use parsers.fm1readmarker if it exists for a ~38% perf win
Augie Fackler <augie@google.com>
parents: 24014
diff changeset
335
33504
5d3ba4395288 obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents: 33499
diff changeset
336 while off < stop:
22851
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
337 # read fixed part
23803
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
338 o1 = off + fsize
23800
83f361a21d31 readmarkers: drop a temporary
Matt Mackall <mpm@selenic.com>
parents: 23799
diff changeset
339 t, secs, tz, flags, numsuc, numpar, nummeta, prec = ufixed(data[off:o1])
23792
db3bc2bcb689 readmarkers: add some whitespace
Matt Mackall <mpm@selenic.com>
parents: 23791
diff changeset
340
23803
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
341 if flags & sha2flag:
23805
345042e024dc readmarkers: add a SHA256 fixme note
Matt Mackall <mpm@selenic.com>
parents: 23804
diff changeset
342 # FIXME: prec was read as a SHA1, needs to be amended
345042e024dc readmarkers: add a SHA256 fixme note
Matt Mackall <mpm@selenic.com>
parents: 23804
diff changeset
343
23801
20b5701cd90a readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents: 23800
diff changeset
344 # read 0 or more successors
23804
e880036719e3 readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents: 23803
diff changeset
345 if numsuc == 1:
e880036719e3 readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents: 23803
diff changeset
346 o2 = o1 + sha2size
e880036719e3 readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents: 23803
diff changeset
347 sucs = (data[o1:o2],)
e880036719e3 readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents: 23803
diff changeset
348 else:
e880036719e3 readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents: 23803
diff changeset
349 o2 = o1 + sha2size * numsuc
e880036719e3 readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents: 23803
diff changeset
350 sucs = unpack(sha2fmt * numsuc, data[o1:o2])
23792
db3bc2bcb689 readmarkers: add some whitespace
Matt Mackall <mpm@selenic.com>
parents: 23791
diff changeset
351
23801
20b5701cd90a readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents: 23800
diff changeset
352 # read parents
23803
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
353 if numpar == noneflag:
23801
20b5701cd90a readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents: 23800
diff changeset
354 o3 = o2
20b5701cd90a readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents: 23800
diff changeset
355 parents = None
23804
e880036719e3 readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents: 23803
diff changeset
356 elif numpar == 1:
e880036719e3 readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents: 23803
diff changeset
357 o3 = o2 + sha2size
e880036719e3 readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents: 23803
diff changeset
358 parents = (data[o2:o3],)
23801
20b5701cd90a readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents: 23800
diff changeset
359 else:
23803
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
360 o3 = o2 + sha2size * numpar
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
361 parents = unpack(sha2fmt * numpar, data[o2:o3])
23801
20b5701cd90a readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents: 23800
diff changeset
362 else:
20b5701cd90a readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents: 23800
diff changeset
363 # read 0 or more successors
23804
e880036719e3 readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents: 23803
diff changeset
364 if numsuc == 1:
e880036719e3 readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents: 23803
diff changeset
365 o2 = o1 + sha1size
e880036719e3 readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents: 23803
diff changeset
366 sucs = (data[o1:o2],)
e880036719e3 readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents: 23803
diff changeset
367 else:
e880036719e3 readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents: 23803
diff changeset
368 o2 = o1 + sha1size * numsuc
e880036719e3 readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents: 23803
diff changeset
369 sucs = unpack(sha1fmt * numsuc, data[o1:o2])
23792
db3bc2bcb689 readmarkers: add some whitespace
Matt Mackall <mpm@selenic.com>
parents: 23791
diff changeset
370
23801
20b5701cd90a readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents: 23800
diff changeset
371 # read parents
23803
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
372 if numpar == noneflag:
23801
20b5701cd90a readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents: 23800
diff changeset
373 o3 = o2
20b5701cd90a readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents: 23800
diff changeset
374 parents = None
23804
e880036719e3 readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents: 23803
diff changeset
375 elif numpar == 1:
e880036719e3 readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents: 23803
diff changeset
376 o3 = o2 + sha1size
e880036719e3 readmarkers: fast-path single successors and parents
Matt Mackall <mpm@selenic.com>
parents: 23803
diff changeset
377 parents = (data[o2:o3],)
23801
20b5701cd90a readmarkers: read node reading into node length conditional
Matt Mackall <mpm@selenic.com>
parents: 23800
diff changeset
378 else:
23803
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
379 o3 = o2 + sha1size * numpar
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
380 parents = unpack(sha1fmt * numpar, data[o2:o3])
23792
db3bc2bcb689 readmarkers: add some whitespace
Matt Mackall <mpm@selenic.com>
parents: 23791
diff changeset
381
22851
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
382 # read metadata
23803
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
383 off = o3 + metasize * nummeta
2763edaba857 readmarkers: promote global constants to locals for performance
Matt Mackall <mpm@selenic.com>
parents: 23802
diff changeset
384 metapairsize = unpack('>' + (metafmt * nummeta), data[o3:off])
22851
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
385 metadata = []
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
386 for idx in xrange(0, len(metapairsize), 2):
23798
86d2a0c41f44 readmarkers: streamline offset tracking
Matt Mackall <mpm@selenic.com>
parents: 23797
diff changeset
387 o1 = off + metapairsize[idx]
86d2a0c41f44 readmarkers: streamline offset tracking
Matt Mackall <mpm@selenic.com>
parents: 23797
diff changeset
388 o2 = o1 + metapairsize[idx + 1]
86d2a0c41f44 readmarkers: streamline offset tracking
Matt Mackall <mpm@selenic.com>
parents: 23797
diff changeset
389 metadata.append((data[off:o1], data[o1:o2]))
86d2a0c41f44 readmarkers: streamline offset tracking
Matt Mackall <mpm@selenic.com>
parents: 23797
diff changeset
390 off = o2
22851
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
391
23800
83f361a21d31 readmarkers: drop a temporary
Matt Mackall <mpm@selenic.com>
parents: 23799
diff changeset
392 yield (prec, sucs, flags, tuple(metadata), (secs, tz * 60), parents)
22851
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
393
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
394 def _fm1encodeonemarker(marker):
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
395 pre, sucs, flags, metadata, date, parents = marker
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
396 # determine node size
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
397 _fm1node = _fm1nodesha1
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
398 if flags & usingsha256:
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
399 _fm1node = _fm1nodesha256
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
400 numsuc = len(sucs)
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
401 numextranodes = numsuc
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
402 if parents is None:
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
403 numpar = _fm1parentnone
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
404 else:
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
405 numpar = len(parents)
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
406 numextranodes += numpar
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
407 formatnodes = _fm1node * numextranodes
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
408 formatmeta = _fm1metapair * len(metadata)
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
409 format = _fm1fixed + formatnodes + formatmeta
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
410 # tz is stored in minutes so we divide by 60
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
411 tz = date[1]//60
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
412 data = [None, date[0], tz, flags, numsuc, numpar, len(metadata), pre]
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
413 data.extend(sucs)
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
414 if parents is not None:
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
415 data.extend(parents)
23498
ac910b1f5658 obsstore: prefetch struct.calcsize
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23497
diff changeset
416 totalsize = _calcsize(format)
22851
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
417 for key, value in metadata:
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
418 lk = len(key)
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
419 lv = len(value)
34408
b6692ba7d5b0 obsmarker: crash more helpfully when metadata fields are >255bytes (issue5681)
Simon Whitaker <swhitaker@fb.com>
parents: 34389
diff changeset
420 if lk > 255:
b6692ba7d5b0 obsmarker: crash more helpfully when metadata fields are >255bytes (issue5681)
Simon Whitaker <swhitaker@fb.com>
parents: 34389
diff changeset
421 msg = ('obsstore metadata key cannot be longer than 255 bytes'
b6692ba7d5b0 obsmarker: crash more helpfully when metadata fields are >255bytes (issue5681)
Simon Whitaker <swhitaker@fb.com>
parents: 34389
diff changeset
422 ' (key "%s" is %u bytes)') % (key, lk)
b6692ba7d5b0 obsmarker: crash more helpfully when metadata fields are >255bytes (issue5681)
Simon Whitaker <swhitaker@fb.com>
parents: 34389
diff changeset
423 raise error.ProgrammingError(msg)
b6692ba7d5b0 obsmarker: crash more helpfully when metadata fields are >255bytes (issue5681)
Simon Whitaker <swhitaker@fb.com>
parents: 34389
diff changeset
424 if lv > 255:
b6692ba7d5b0 obsmarker: crash more helpfully when metadata fields are >255bytes (issue5681)
Simon Whitaker <swhitaker@fb.com>
parents: 34389
diff changeset
425 msg = ('obsstore metadata value cannot be longer than 255 bytes'
b6692ba7d5b0 obsmarker: crash more helpfully when metadata fields are >255bytes (issue5681)
Simon Whitaker <swhitaker@fb.com>
parents: 34389
diff changeset
426 ' (value "%s" for key "%s" is %u bytes)') % (value, key, lv)
b6692ba7d5b0 obsmarker: crash more helpfully when metadata fields are >255bytes (issue5681)
Simon Whitaker <swhitaker@fb.com>
parents: 34389
diff changeset
427 raise error.ProgrammingError(msg)
22851
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
428 data.append(lk)
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
429 data.append(lv)
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
430 totalsize += lk + lv
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
431 data[0] = totalsize
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
432 data = [_pack(format, *data)]
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
433 for key, value in metadata:
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
434 data.append(key)
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
435 data.append(value)
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
436 return ''.join(data)
22848
72100c1d53d4 obsolete: gather _fm0 meta encoding with other _fm0 code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22847
diff changeset
437
33504
5d3ba4395288 obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents: 33499
diff changeset
438 def _fm1readmarkers(data, off, stop):
24019
26fbf07482b2 _fm1readmarkers: generate list in C
Martin von Zweigbergk <martinvonz@google.com>
parents: 24018
diff changeset
439 native = getattr(parsers, 'fm1readmarkers', None)
26fbf07482b2 _fm1readmarkers: generate list in C
Martin von Zweigbergk <martinvonz@google.com>
parents: 24018
diff changeset
440 if not native:
33504
5d3ba4395288 obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents: 33499
diff changeset
441 return _fm1purereadmarkers(data, off, stop)
24019
26fbf07482b2 _fm1readmarkers: generate list in C
Martin von Zweigbergk <martinvonz@google.com>
parents: 24018
diff changeset
442 return native(data, off, stop)
26fbf07482b2 _fm1readmarkers: generate list in C
Martin von Zweigbergk <martinvonz@google.com>
parents: 24018
diff changeset
443
22331
b130b241718e obsolete: support for any known obsstore format when reading or writing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22330
diff changeset
444 # mapping to read/write various marker formats
b130b241718e obsolete: support for any known obsstore format when reading or writing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22330
diff changeset
445 # <version> -> (decoder, encoder)
22851
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
446 formats = {_fm0version: (_fm0readmarkers, _fm0encodeonemarker),
974389427e5f obsolete: introduce a new binary encoding for obsmarkers (version 1)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22850
diff changeset
447 _fm1version: (_fm1readmarkers, _fm1encodeonemarker)}
22331
b130b241718e obsolete: support for any known obsstore format when reading or writing
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22330
diff changeset
448
32709
9933410bf763 obsstore: move marker version reading to a separate function
Jun Wu <quark@fb.com>
parents: 32708
diff changeset
449 def _readmarkerversion(data):
9933410bf763 obsstore: move marker version reading to a separate function
Jun Wu <quark@fb.com>
parents: 32708
diff changeset
450 return _unpack('>B', data[0:1])[0]
9933410bf763 obsstore: move marker version reading to a separate function
Jun Wu <quark@fb.com>
parents: 32708
diff changeset
451
23497
5817f71c2336 obsstore: disable garbage collection during initialization (issue4456)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23002
diff changeset
452 @util.nogc
33504
5d3ba4395288 obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents: 33499
diff changeset
453 def _readmarkers(data, off=None, stop=None):
22612
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
454 """Read and enumerate markers from raw data"""
32709
9933410bf763 obsstore: move marker version reading to a separate function
Jun Wu <quark@fb.com>
parents: 32708
diff changeset
455 diskversion = _readmarkerversion(data)
33504
5d3ba4395288 obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents: 33499
diff changeset
456 if not off:
5d3ba4395288 obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents: 33499
diff changeset
457 off = 1 # skip 1 byte version number
5d3ba4395288 obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents: 33499
diff changeset
458 if stop is None:
5d3ba4395288 obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents: 33499
diff changeset
459 stop = len(data)
22612
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
460 if diskversion not in formats:
32622
19df975eb555 obsolete: raise richer exception on unknown version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32521
diff changeset
461 msg = _('parsing obsolete marker: unknown version %r') % diskversion
19df975eb555 obsolete: raise richer exception on unknown version
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32521
diff changeset
462 raise error.UnknownVersion(msg, version=diskversion)
33504
5d3ba4395288 obsstore: let read marker API take a range of offsets
Jun Wu <quark@fb.com>
parents: 33499
diff changeset
463 return diskversion, formats[diskversion][0](data, off, stop)
22612
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
464
32712
9576974a3b6d obsstore: move header encoding to a separate function
Jun Wu <quark@fb.com>
parents: 32711
diff changeset
465 def encodeheader(version=_fm0version):
9576974a3b6d obsstore: move header encoding to a separate function
Jun Wu <quark@fb.com>
parents: 32711
diff changeset
466 return _pack('>B', version)
9576974a3b6d obsstore: move header encoding to a separate function
Jun Wu <quark@fb.com>
parents: 32711
diff changeset
467
22612
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
468 def encodemarkers(markers, addheader=False, version=_fm0version):
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
469 # Kept separate from flushmarkers(), it will be reused for
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
470 # markers exchange.
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
471 encodeone = formats[version][1]
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
472 if addheader:
32712
9576974a3b6d obsstore: move header encoding to a separate function
Jun Wu <quark@fb.com>
parents: 32711
diff changeset
473 yield encodeheader(version)
22612
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
474 for marker in markers:
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
475 yield encodeone(marker)
fdfa40ee75cf obsolete: gather all contents related to format version 0 in a single place
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22507
diff changeset
476
24044
e0e6dd806b23 obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24019
diff changeset
477 @util.nogc
e0e6dd806b23 obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24019
diff changeset
478 def _addsuccessors(successors, markers):
e0e6dd806b23 obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24019
diff changeset
479 for mark in markers:
e0e6dd806b23 obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24019
diff changeset
480 successors.setdefault(mark[0], set()).add(mark)
e0e6dd806b23 obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24019
diff changeset
481
e0e6dd806b23 obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24019
diff changeset
482 @util.nogc
33758
32d4f815c4f4 obsolete: rename _addprecursors into _addpredecessors
Boris Feld <boris.feld@octobus.net>
parents: 33745
diff changeset
483 def _addpredecessors(predecessors, markers):
24044
e0e6dd806b23 obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24019
diff changeset
484 for mark in markers:
e0e6dd806b23 obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24019
diff changeset
485 for suc in mark[1]:
33758
32d4f815c4f4 obsolete: rename _addprecursors into _addpredecessors
Boris Feld <boris.feld@octobus.net>
parents: 33745
diff changeset
486 predecessors.setdefault(suc, set()).add(mark)
24044
e0e6dd806b23 obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24019
diff changeset
487
e0e6dd806b23 obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24019
diff changeset
488 @util.nogc
e0e6dd806b23 obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24019
diff changeset
489 def _addchildren(children, markers):
e0e6dd806b23 obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24019
diff changeset
490 for mark in markers:
e0e6dd806b23 obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24019
diff changeset
491 parents = mark[5]
e0e6dd806b23 obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24019
diff changeset
492 if parents is not None:
e0e6dd806b23 obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24019
diff changeset
493 for p in parents:
e0e6dd806b23 obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24019
diff changeset
494 children.setdefault(p, set()).add(mark)
e0e6dd806b23 obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24019
diff changeset
495
24045
43061f96e40e obsolete: pass only new markers to _checkinvalidmarkers()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24044
diff changeset
496 def _checkinvalidmarkers(markers):
23973
18d431147cc1 obsstore: make the invalid markers check wrap-able
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23805
diff changeset
497 """search for marker with invalid data and raise error if needed
18d431147cc1 obsstore: make the invalid markers check wrap-able
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23805
diff changeset
498
18d431147cc1 obsstore: make the invalid markers check wrap-able
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23805
diff changeset
499 Exist as a separated function to allow the evolve extension for a more
18d431147cc1 obsstore: make the invalid markers check wrap-able
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23805
diff changeset
500 subtle handling.
18d431147cc1 obsstore: make the invalid markers check wrap-able
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23805
diff changeset
501 """
24045
43061f96e40e obsolete: pass only new markers to _checkinvalidmarkers()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24044
diff changeset
502 for mark in markers:
43061f96e40e obsolete: pass only new markers to _checkinvalidmarkers()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24044
diff changeset
503 if node.nullid in mark[1]:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26310
diff changeset
504 raise error.Abort(_('bad obsolescence marker detected: '
24045
43061f96e40e obsolete: pass only new markers to _checkinvalidmarkers()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24044
diff changeset
505 'invalid successors nullid'))
23973
18d431147cc1 obsstore: make the invalid markers check wrap-able
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23805
diff changeset
506
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
507 class obsstore(object):
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
508 """Store obsolete markers
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
509
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
510 Markers can be accessed with two mappings:
33760
0c3112f17610 obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents: 33759
diff changeset
511 - predecessors[x] -> set(markers on predecessors edges of x)
17776
072812e9f570 obsolete: flip `obstore.successors` and `obsolete.precursors`
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17775
diff changeset
512 - successors[x] -> set(markers on successors edges of x)
33760
0c3112f17610 obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents: 33759
diff changeset
513 - children[x] -> set(markers on predecessors edges of children(x)
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
514 """
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
515
22254
b8a0e8176693 obsstore: add a `parents` field
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22253
diff changeset
516 fields = ('prec', 'succs', 'flag', 'meta', 'date', 'parents')
33760
0c3112f17610 obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents: 33759
diff changeset
517 # prec: nodeid, predecessors changesets
22254
b8a0e8176693 obsstore: add a `parents` field
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22253
diff changeset
518 # succs: tuple of nodeid, successor changesets (0-N length)
b8a0e8176693 obsstore: add a `parents` field
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22253
diff changeset
519 # flag: integer, flag field carrying modifier for the markers (see doc)
b8a0e8176693 obsstore: add a `parents` field
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22253
diff changeset
520 # meta: binary blob, encoded metadata dictionary
b8a0e8176693 obsstore: add a `parents` field
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22253
diff changeset
521 # date: (float, int) tuple, date of marker creation
33760
0c3112f17610 obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents: 33759
diff changeset
522 # parents: (tuple of nodeid) or None, parents of predecessors
22254
b8a0e8176693 obsstore: add a `parents` field
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22253
diff changeset
523 # None is used when no data has been recorded
22221
e75b1a3c1dbc obsstore: add fields attribute to track each field in a marker
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22219
diff changeset
524
25669
8d94861842e5 obsolete: replace references to 'sopener' with 'svfs'
Siddharth Agarwal <sid0@fb.com>
parents: 25412
diff changeset
525 def __init__(self, svfs, defaultformat=_fm1version, readonly=False):
17469
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
526 # caches for various obsolescence related cache
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
527 self.caches = {}
25669
8d94861842e5 obsolete: replace references to 'sopener' with 'svfs'
Siddharth Agarwal <sid0@fb.com>
parents: 25412
diff changeset
528 self.svfs = svfs
32711
5d8dd6c90af9 obsstore: move _version to a propertycache
Jun Wu <quark@fb.com>
parents: 32710
diff changeset
529 self._defaultformat = defaultformat
22950
bb8278b289ee obsolete: add readonly flag to obstore constructor
Durham Goode <durham@fb.com>
parents: 22949
diff changeset
530 self._readonly = readonly
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
531
17073
3a79a5682af1 obsolete: add easy way to iterate over obsolete marker object
Pierre-Yves.David@ens-lyon.org
parents: 17072
diff changeset
532 def __iter__(self):
3a79a5682af1 obsolete: add easy way to iterate over obsolete marker object
Pierre-Yves.David@ens-lyon.org
parents: 17072
diff changeset
533 return iter(self._all)
3a79a5682af1 obsolete: add easy way to iterate over obsolete marker object
Pierre-Yves.David@ens-lyon.org
parents: 17072
diff changeset
534
20585
f3c8db3d6d66 obsstore: add a __len__ method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20584
diff changeset
535 def __len__(self):
f3c8db3d6d66 obsstore: add a __len__ method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20584
diff changeset
536 return len(self._all)
f3c8db3d6d66 obsstore: add a __len__ method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20584
diff changeset
537
17075
28ed1c4511ce obsolete: exchange obsolete marker over pushkey
Pierre-Yves.David@ens-lyon.org
parents: 17073
diff changeset
538 def __nonzero__(self):
35877
d8f891ec5d7d obsolete: use native string when peeking in __dict__
Augie Fackler <augie@google.com>
parents: 35574
diff changeset
539 if not self._cached(r'_all'):
26310
61efe9ef6ad4 obsstore: fast path to check if obsstore is empty
Yuya Nishihara <yuya@tcha.org>
parents: 26309
diff changeset
540 try:
61efe9ef6ad4 obsstore: fast path to check if obsstore is empty
Yuya Nishihara <yuya@tcha.org>
parents: 26309
diff changeset
541 return self.svfs.stat('obsstore').st_size > 1
61efe9ef6ad4 obsstore: fast path to check if obsstore is empty
Yuya Nishihara <yuya@tcha.org>
parents: 26309
diff changeset
542 except OSError as inst:
61efe9ef6ad4 obsstore: fast path to check if obsstore is empty
Yuya Nishihara <yuya@tcha.org>
parents: 26309
diff changeset
543 if inst.errno != errno.ENOENT:
61efe9ef6ad4 obsstore: fast path to check if obsstore is empty
Yuya Nishihara <yuya@tcha.org>
parents: 26309
diff changeset
544 raise
61efe9ef6ad4 obsstore: fast path to check if obsstore is empty
Yuya Nishihara <yuya@tcha.org>
parents: 26309
diff changeset
545 # just build an empty _all list if no obsstore exists, which
61efe9ef6ad4 obsstore: fast path to check if obsstore is empty
Yuya Nishihara <yuya@tcha.org>
parents: 26309
diff changeset
546 # avoids further stat() syscalls
17075
28ed1c4511ce obsolete: exchange obsolete marker over pushkey
Pierre-Yves.David@ens-lyon.org
parents: 17073
diff changeset
547 return bool(self._all)
28ed1c4511ce obsolete: exchange obsolete marker over pushkey
Pierre-Yves.David@ens-lyon.org
parents: 17073
diff changeset
548
31483
413b44003462 py3: add __bool__ to every class defining __nonzero__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31038
diff changeset
549 __bool__ = __nonzero__
413b44003462 py3: add __bool__ to every class defining __nonzero__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31038
diff changeset
550
26684
74ff350c208c obsstore: make the readonly attribute accessible
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
551 @property
74ff350c208c obsstore: make the readonly attribute accessible
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
552 def readonly(self):
74ff350c208c obsstore: make the readonly attribute accessible
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
553 """True if marker creation is disabled
74ff350c208c obsstore: make the readonly attribute accessible
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
554
74ff350c208c obsstore: make the readonly attribute accessible
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
555 Remove me in the future when obsolete marker is always on."""
74ff350c208c obsstore: make the readonly attribute accessible
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
556 return self._readonly
74ff350c208c obsstore: make the readonly attribute accessible
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26587
diff changeset
557
22255
adb3798dce49 obsstore: add a `parents` argument to obsstore.create
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22254
diff changeset
558 def create(self, transaction, prec, succs=(), flag=0, parents=None,
32449
08d02c1d7e67 devel: use default-date config field when creating obsmarkers
Boris Feld <boris.feld@octobus.net>
parents: 32425
diff changeset
559 date=None, metadata=None, ui=None):
17071
11f26e2669aa command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents: 17070
diff changeset
560 """obsolete: add a new obsolete marker
11f26e2669aa command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents: 17070
diff changeset
561
11f26e2669aa command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents: 17070
diff changeset
562 * ensuring it is hashable
11f26e2669aa command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents: 17070
diff changeset
563 * check mandatory metadata
11f26e2669aa command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents: 17070
diff changeset
564 * encode metadata
20516
3af218cf2007 obsstore: update create docstring to point to the coder friendly function
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 20476
diff changeset
565
3af218cf2007 obsstore: update create docstring to point to the coder friendly function
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 20476
diff changeset
566 If you are a human writing code creating marker you want to use the
3af218cf2007 obsstore: update create docstring to point to the coder friendly function
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 20476
diff changeset
567 `createmarkers` function in this module instead.
20584
224a058f7cd1 obsstore: `create` method return True if a marker is actually added
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20549
diff changeset
568
224a058f7cd1 obsstore: `create` method return True if a marker is actually added
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20549
diff changeset
569 return True if a new marker have been added, False if the markers
224a058f7cd1 obsstore: `create` method return True if a marker is actually added
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20549
diff changeset
570 already existed (no op).
17071
11f26e2669aa command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents: 17070
diff changeset
571 """
11f26e2669aa command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents: 17070
diff changeset
572 if metadata is None:
11f26e2669aa command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents: 17070
diff changeset
573 metadata = {}
22222
883e8b6bd461 obsmarker: add `date` as an explicit field
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22221
diff changeset
574 if date is None:
883e8b6bd461 obsmarker: add `date` as an explicit field
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22221
diff changeset
575 if 'date' in metadata:
883e8b6bd461 obsmarker: add `date` as an explicit field
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22221
diff changeset
576 # as a courtesy for out-of-tree extensions
36636
c6061cadb400 util: extract all date-related utils in utils/dateutil module
Boris Feld <boris.feld@octobus.net>
parents: 36453
diff changeset
577 date = dateutil.parsedate(metadata.pop('date'))
32449
08d02c1d7e67 devel: use default-date config field when creating obsmarkers
Boris Feld <boris.feld@octobus.net>
parents: 32425
diff changeset
578 elif ui is not None:
08d02c1d7e67 devel: use default-date config field when creating obsmarkers
Boris Feld <boris.feld@octobus.net>
parents: 32425
diff changeset
579 date = ui.configdate('devel', 'default-date')
08d02c1d7e67 devel: use default-date config field when creating obsmarkers
Boris Feld <boris.feld@octobus.net>
parents: 32425
diff changeset
580 if date is None:
36636
c6061cadb400 util: extract all date-related utils in utils/dateutil module
Boris Feld <boris.feld@octobus.net>
parents: 36453
diff changeset
581 date = dateutil.makedate()
22222
883e8b6bd461 obsmarker: add `date` as an explicit field
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22221
diff changeset
582 else:
36636
c6061cadb400 util: extract all date-related utils in utils/dateutil module
Boris Feld <boris.feld@octobus.net>
parents: 36453
diff changeset
583 date = dateutil.makedate()
17071
11f26e2669aa command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents: 17070
diff changeset
584 if len(prec) != 20:
11f26e2669aa command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents: 17070
diff changeset
585 raise ValueError(prec)
11f26e2669aa command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents: 17070
diff changeset
586 for succ in succs:
11f26e2669aa command: creation of obsolete marker
Pierre-Yves.David@ens-lyon.org
parents: 17070
diff changeset
587 if len(succ) != 20:
17117
217bfb10e6db obsolete: fix error message at marker creation
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17076
diff changeset
588 raise ValueError(succ)
22177
a56038e6a3c9 obsstore.create: add a simple safeguard against cyclic markers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21166
diff changeset
589 if prec in succs:
a56038e6a3c9 obsstore.create: add a simple safeguard against cyclic markers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21166
diff changeset
590 raise ValueError(_('in-marker cycle with %s') % node.hex(prec))
22845
ef880e28e56a obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22685
diff changeset
591
ef880e28e56a obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22685
diff changeset
592 metadata = tuple(sorted(metadata.iteritems()))
ef880e28e56a obsolete: store metadata as a tuple of (key, value) pairs (API)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22685
diff changeset
593
33745
388901a15bfa obsolete: use bytes() instead of str() so the node is bytes on py3
Augie Fackler <augie@google.com>
parents: 33696
diff changeset
594 marker = (bytes(prec), tuple(succs), int(flag), metadata, date, parents)
20584
224a058f7cd1 obsstore: `create` method return True if a marker is actually added
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20549
diff changeset
595 return bool(self.add(transaction, [marker]))
17220
bdac214a4705 obsolete: obsstore.add now takes a list of markers.
Pierre-Yves.David@ens-lyon.org
parents: 17219
diff changeset
596
bdac214a4705 obsolete: obsstore.add now takes a list of markers.
Pierre-Yves.David@ens-lyon.org
parents: 17219
diff changeset
597 def add(self, transaction, markers):
bdac214a4705 obsolete: obsstore.add now takes a list of markers.
Pierre-Yves.David@ens-lyon.org
parents: 17219
diff changeset
598 """Add new markers to the store
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
599
17220
bdac214a4705 obsolete: obsstore.add now takes a list of markers.
Pierre-Yves.David@ens-lyon.org
parents: 17219
diff changeset
600 Take care of filtering duplicate.
bdac214a4705 obsolete: obsstore.add now takes a list of markers.
Pierre-Yves.David@ens-lyon.org
parents: 17219
diff changeset
601 Return the number of new marker."""
22950
bb8278b289ee obsolete: add readonly flag to obstore constructor
Durham Goode <durham@fb.com>
parents: 22949
diff changeset
602 if self._readonly:
29389
98e8313dcd9e i18n: translate abort messages
liscju <piotr.listkiewicz@gmail.com>
parents: 29281
diff changeset
603 raise error.Abort(_('creating obsolete markers is not enabled on '
98e8313dcd9e i18n: translate abort messages
liscju <piotr.listkiewicz@gmail.com>
parents: 29281
diff changeset
604 'this repo'))
32794
5ffb138d7b1a obsstore: do not load all markers to detect duplication
Jun Wu <quark@fb.com>
parents: 32749
diff changeset
605 known = set()
5ffb138d7b1a obsstore: do not load all markers to detect duplication
Jun Wu <quark@fb.com>
parents: 32749
diff changeset
606 getsuccessors = self.successors.get
20030
5931489b65e0 obsolete: do not accept duplicated marker during exchange
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20028
diff changeset
607 new = []
5931489b65e0 obsolete: do not accept duplicated marker during exchange
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20028
diff changeset
608 for m in markers:
32794
5ffb138d7b1a obsstore: do not load all markers to detect duplication
Jun Wu <quark@fb.com>
parents: 32749
diff changeset
609 if m not in getsuccessors(m[0], ()) and m not in known:
20030
5931489b65e0 obsolete: do not accept duplicated marker during exchange
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20028
diff changeset
610 known.add(m)
5931489b65e0 obsolete: do not accept duplicated marker during exchange
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20028
diff changeset
611 new.append(m)
17220
bdac214a4705 obsolete: obsstore.add now takes a list of markers.
Pierre-Yves.David@ens-lyon.org
parents: 17219
diff changeset
612 if new:
25669
8d94861842e5 obsolete: replace references to 'sopener' with 'svfs'
Siddharth Agarwal <sid0@fb.com>
parents: 25412
diff changeset
613 f = self.svfs('obsstore', 'ab')
17124
f1b7683f3f95 obsolete: move obsolete markers read/write logic to obsstore object
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17117
diff changeset
614 try:
17126
8fa8717b47b6 obsolete: write obsolete marker inside a transaction
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17125
diff changeset
615 offset = f.tell()
8fa8717b47b6 obsolete: write obsolete marker inside a transaction
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17125
diff changeset
616 transaction.add('obsstore', offset)
17219
494a970f68de obsolete: refactor writemarkers to only encode them
Pierre-Yves.David@ens-lyon.org
parents: 17213
diff changeset
617 # offset == 0: new file - add the version header
33479
8b48dad66be4 obsstore: keep self._data updated with _addmarkers
Jun Wu <quark@fb.com>
parents: 33273
diff changeset
618 data = b''.join(encodemarkers(new, offset == 0, self._version))
8b48dad66be4 obsstore: keep self._data updated with _addmarkers
Jun Wu <quark@fb.com>
parents: 33273
diff changeset
619 f.write(data)
17126
8fa8717b47b6 obsolete: write obsolete marker inside a transaction
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17125
diff changeset
620 finally:
8fa8717b47b6 obsolete: write obsolete marker inside a transaction
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17125
diff changeset
621 # XXX: f.close() == filecache invalidation == obsstore rebuilt.
8fa8717b47b6 obsolete: write obsolete marker inside a transaction
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17125
diff changeset
622 # call 'filecacheentry.refresh()' here
17124
f1b7683f3f95 obsolete: move obsolete markers read/write logic to obsstore object
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17117
diff changeset
623 f.close()
33251
a5cb2e4460de transaction: track new obsmarkers in the 'changes' mapping
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33244
diff changeset
624 addedmarkers = transaction.changes.get('obsmarkers')
a5cb2e4460de transaction: track new obsmarkers in the 'changes' mapping
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33244
diff changeset
625 if addedmarkers is not None:
a5cb2e4460de transaction: track new obsmarkers in the 'changes' mapping
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33244
diff changeset
626 addedmarkers.update(new)
33479
8b48dad66be4 obsstore: keep self._data updated with _addmarkers
Jun Wu <quark@fb.com>
parents: 33273
diff changeset
627 self._addmarkers(new, data)
17469
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
628 # new marker *may* have changed several set. invalidate the cache.
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
629 self.caches.clear()
22339
9680da73cfe0 obsmarker: record the number of new markers in the transaction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22335
diff changeset
630 # records the number of new markers for the transaction hooks
9680da73cfe0 obsmarker: record the number of new markers in the transaction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22335
diff changeset
631 previous = int(transaction.hookargs.get('new_obsmarkers', '0'))
36453
2831d918e1b4 py3: convert known-int values to bytes using %d
Augie Fackler <augie@google.com>
parents: 35931
diff changeset
632 transaction.hookargs['new_obsmarkers'] = '%d' % (previous + len(new))
17220
bdac214a4705 obsolete: obsstore.add now takes a list of markers.
Pierre-Yves.David@ens-lyon.org
parents: 17219
diff changeset
633 return len(new)
17126
8fa8717b47b6 obsolete: write obsolete marker inside a transaction
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17125
diff changeset
634
17524
a736e1e15e46 spelling: transaction
timeless@mozdev.org
parents: 17516
diff changeset
635 def mergemarkers(self, transaction, data):
22325
3363f2d36015 obsstore: have the `mergemarkers` method return the number of new markers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22309
diff changeset
636 """merge a binary stream of markers inside the obsstore
3363f2d36015 obsstore: have the `mergemarkers` method return the number of new markers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22309
diff changeset
637
3363f2d36015 obsstore: have the `mergemarkers` method return the number of new markers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22309
diff changeset
638 Returns the number of new markers added."""
22332
13e22358e9d2 obsolete: have _readmarkers return the format version
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22331
diff changeset
639 version, markers = _readmarkers(data)
22325
3363f2d36015 obsstore: have the `mergemarkers` method return the number of new markers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22309
diff changeset
640 return self.add(transaction, markers)
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
641
24046
6e1d9f9932a9 obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents: 24045
diff changeset
642 @propertycache
32710
3fad59faba8a obsstore: separate marker parsing from obsstore reading
Jun Wu <quark@fb.com>
parents: 32709
diff changeset
643 def _data(self):
3fad59faba8a obsstore: separate marker parsing from obsstore reading
Jun Wu <quark@fb.com>
parents: 32709
diff changeset
644 return self.svfs.tryread('obsstore')
3fad59faba8a obsstore: separate marker parsing from obsstore reading
Jun Wu <quark@fb.com>
parents: 32709
diff changeset
645
3fad59faba8a obsstore: separate marker parsing from obsstore reading
Jun Wu <quark@fb.com>
parents: 32709
diff changeset
646 @propertycache
32711
5d8dd6c90af9 obsstore: move _version to a propertycache
Jun Wu <quark@fb.com>
parents: 32710
diff changeset
647 def _version(self):
5d8dd6c90af9 obsstore: move _version to a propertycache
Jun Wu <quark@fb.com>
parents: 32710
diff changeset
648 if len(self._data) >= 1:
5d8dd6c90af9 obsstore: move _version to a propertycache
Jun Wu <quark@fb.com>
parents: 32710
diff changeset
649 return _readmarkerversion(self._data)
5d8dd6c90af9 obsstore: move _version to a propertycache
Jun Wu <quark@fb.com>
parents: 32710
diff changeset
650 else:
5d8dd6c90af9 obsstore: move _version to a propertycache
Jun Wu <quark@fb.com>
parents: 32710
diff changeset
651 return self._defaultformat
5d8dd6c90af9 obsstore: move _version to a propertycache
Jun Wu <quark@fb.com>
parents: 32710
diff changeset
652
5d8dd6c90af9 obsstore: move _version to a propertycache
Jun Wu <quark@fb.com>
parents: 32710
diff changeset
653 @propertycache
26309
44918682093f obsstore: delay loading markers from obsstore file
Yuya Nishihara <yuya@tcha.org>
parents: 26308
diff changeset
654 def _all(self):
32710
3fad59faba8a obsstore: separate marker parsing from obsstore reading
Jun Wu <quark@fb.com>
parents: 32709
diff changeset
655 data = self._data
26309
44918682093f obsstore: delay loading markers from obsstore file
Yuya Nishihara <yuya@tcha.org>
parents: 26308
diff changeset
656 if not data:
44918682093f obsstore: delay loading markers from obsstore file
Yuya Nishihara <yuya@tcha.org>
parents: 26308
diff changeset
657 return []
44918682093f obsstore: delay loading markers from obsstore file
Yuya Nishihara <yuya@tcha.org>
parents: 26308
diff changeset
658 self._version, markers = _readmarkers(data)
44918682093f obsstore: delay loading markers from obsstore file
Yuya Nishihara <yuya@tcha.org>
parents: 26308
diff changeset
659 markers = list(markers)
44918682093f obsstore: delay loading markers from obsstore file
Yuya Nishihara <yuya@tcha.org>
parents: 26308
diff changeset
660 _checkinvalidmarkers(markers)
44918682093f obsstore: delay loading markers from obsstore file
Yuya Nishihara <yuya@tcha.org>
parents: 26308
diff changeset
661 return markers
44918682093f obsstore: delay loading markers from obsstore file
Yuya Nishihara <yuya@tcha.org>
parents: 26308
diff changeset
662
44918682093f obsstore: delay loading markers from obsstore file
Yuya Nishihara <yuya@tcha.org>
parents: 26308
diff changeset
663 @propertycache
24046
6e1d9f9932a9 obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents: 24045
diff changeset
664 def successors(self):
6e1d9f9932a9 obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents: 24045
diff changeset
665 successors = {}
6e1d9f9932a9 obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents: 24045
diff changeset
666 _addsuccessors(successors, self._all)
6e1d9f9932a9 obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents: 24045
diff changeset
667 return successors
6e1d9f9932a9 obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents: 24045
diff changeset
668
6e1d9f9932a9 obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents: 24045
diff changeset
669 @propertycache
33759
d5acd967f95a obsstore: rename precursors into predecessors
Boris Feld <boris.feld@octobus.net>
parents: 33758
diff changeset
670 def predecessors(self):
33758
32d4f815c4f4 obsolete: rename _addprecursors into _addpredecessors
Boris Feld <boris.feld@octobus.net>
parents: 33745
diff changeset
671 predecessors = {}
32d4f815c4f4 obsolete: rename _addprecursors into _addpredecessors
Boris Feld <boris.feld@octobus.net>
parents: 33745
diff changeset
672 _addpredecessors(predecessors, self._all)
32d4f815c4f4 obsolete: rename _addprecursors into _addpredecessors
Boris Feld <boris.feld@octobus.net>
parents: 33745
diff changeset
673 return predecessors
24046
6e1d9f9932a9 obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents: 24045
diff changeset
674
6e1d9f9932a9 obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents: 24045
diff changeset
675 @propertycache
6e1d9f9932a9 obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents: 24045
diff changeset
676 def children(self):
6e1d9f9932a9 obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents: 24045
diff changeset
677 children = {}
6e1d9f9932a9 obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents: 24045
diff changeset
678 _addchildren(children, self._all)
6e1d9f9932a9 obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents: 24045
diff changeset
679 return children
6e1d9f9932a9 obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents: 24045
diff changeset
680
6e1d9f9932a9 obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents: 24045
diff changeset
681 def _cached(self, attr):
6e1d9f9932a9 obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents: 24045
diff changeset
682 return attr in self.__dict__
6e1d9f9932a9 obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents: 24045
diff changeset
683
33479
8b48dad66be4 obsstore: keep self._data updated with _addmarkers
Jun Wu <quark@fb.com>
parents: 33273
diff changeset
684 def _addmarkers(self, markers, rawdata):
24044
e0e6dd806b23 obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24019
diff changeset
685 markers = list(markers) # to allow repeated iteration
33479
8b48dad66be4 obsstore: keep self._data updated with _addmarkers
Jun Wu <quark@fb.com>
parents: 33273
diff changeset
686 self._data = self._data + rawdata
24044
e0e6dd806b23 obsolete: extract helpers from _load()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24019
diff changeset
687 self._all.extend(markers)
35877
d8f891ec5d7d obsolete: use native string when peeking in __dict__
Augie Fackler <augie@google.com>
parents: 35574
diff changeset
688 if self._cached(r'successors'):
24046
6e1d9f9932a9 obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents: 24045
diff changeset
689 _addsuccessors(self.successors, markers)
35877
d8f891ec5d7d obsolete: use native string when peeking in __dict__
Augie Fackler <augie@google.com>
parents: 35574
diff changeset
690 if self._cached(r'predecessors'):
33759
d5acd967f95a obsstore: rename precursors into predecessors
Boris Feld <boris.feld@octobus.net>
parents: 33758
diff changeset
691 _addpredecessors(self.predecessors, markers)
35877
d8f891ec5d7d obsolete: use native string when peeking in __dict__
Augie Fackler <augie@google.com>
parents: 35574
diff changeset
692 if self._cached(r'children'):
24046
6e1d9f9932a9 obsolete: populate successors, precursors, children lazily
Martin von Zweigbergk <martinvonz@google.com>
parents: 24045
diff changeset
693 _addchildren(self.children, markers)
24045
43061f96e40e obsolete: pass only new markers to _checkinvalidmarkers()
Martin von Zweigbergk <martinvonz@google.com>
parents: 24044
diff changeset
694 _checkinvalidmarkers(markers)
23973
18d431147cc1 obsstore: make the invalid markers check wrap-able
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23805
diff changeset
695
22271
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
696 def relevantmarkers(self, nodes):
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
697 """return a set of all obsolescence markers relevant to a set of nodes.
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
698
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
699 "relevant" to a set of nodes mean:
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
700
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
701 - marker that use this changeset as successor
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
702 - prune marker of direct children on this changeset
33760
0c3112f17610 obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents: 33759
diff changeset
703 - recursive application of the two rules on predecessors of these
0c3112f17610 obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents: 33759
diff changeset
704 markers
22271
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
705
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
706 It is a set so you cannot rely on order."""
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
707
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
708 pendingnodes = set(nodes)
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
709 seenmarkers = set()
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
710 seennodes = set(pendingnodes)
33759
d5acd967f95a obsstore: rename precursors into predecessors
Boris Feld <boris.feld@octobus.net>
parents: 33758
diff changeset
711 precursorsmarkers = self.predecessors
32521
176d1a0ce385 obsolete: fix relevant-obsmarkers computation on pruned changeset
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32449
diff changeset
712 succsmarkers = self.successors
22271
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
713 children = self.children
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
714 while pendingnodes:
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
715 direct = set()
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
716 for current in pendingnodes:
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
717 direct.update(precursorsmarkers.get(current, ()))
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
718 pruned = [m for m in children.get(current, ()) if not m[1]]
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
719 direct.update(pruned)
32521
176d1a0ce385 obsolete: fix relevant-obsmarkers computation on pruned changeset
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32449
diff changeset
720 pruned = [m for m in succsmarkers.get(current, ()) if not m[1]]
176d1a0ce385 obsolete: fix relevant-obsmarkers computation on pruned changeset
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32449
diff changeset
721 direct.update(pruned)
22271
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
722 direct -= seenmarkers
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
723 pendingnodes = set([m[0] for m in direct])
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
724 seenmarkers |= direct
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
725 pendingnodes -= seennodes
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
726 seennodes |= pendingnodes
8c69262df82d obsstore: add relevantmarkers method
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22270
diff changeset
727 return seenmarkers
17070
ad0d6c2b3279 obsolete: introduction of obsolete markers
Pierre-Yves.David@ens-lyon.org
parents:
diff changeset
728
32749
c8177792fef6 obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32712
diff changeset
729 def makestore(ui, repo):
c8177792fef6 obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32712
diff changeset
730 """Create an obsstore instance from a repo."""
c8177792fef6 obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32712
diff changeset
731 # read default format for new obsstore.
c8177792fef6 obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32712
diff changeset
732 # developer config: format.obsstore-version
33244
fd50788a2d4f configitems: register the 'format.obsstore-version' config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33155
diff changeset
733 defaultformat = ui.configint('format', 'obsstore-version')
32749
c8177792fef6 obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32712
diff changeset
734 # rely on obsstore class default when possible.
c8177792fef6 obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32712
diff changeset
735 kwargs = {}
c8177792fef6 obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32712
diff changeset
736 if defaultformat is not None:
35374
7354fda50e8b py3: handle keyword arguments correctly in obsolete.py
Pulkit Goyal <7895pulkit@gmail.com>
parents: 35158
diff changeset
737 kwargs[r'defaultformat'] = defaultformat
32749
c8177792fef6 obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32712
diff changeset
738 readonly = not isenabled(repo, createmarkersopt)
c8177792fef6 obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32712
diff changeset
739 store = obsstore(repo.svfs, readonly=readonly, **kwargs)
c8177792fef6 obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32712
diff changeset
740 if store and readonly:
c8177792fef6 obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32712
diff changeset
741 ui.warn(_('obsolete feature not enabled but %i markers found!\n')
c8177792fef6 obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32712
diff changeset
742 % len(list(store)))
c8177792fef6 obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32712
diff changeset
743 return store
c8177792fef6 obsolete: move obsstore creation logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32712
diff changeset
744
22345
cf7014c5f087 obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22339
diff changeset
745 def commonversion(versions):
cf7014c5f087 obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22339
diff changeset
746 """Return the newest version listed in both versions and our local formats.
cf7014c5f087 obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22339
diff changeset
747
cf7014c5f087 obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22339
diff changeset
748 Returns None if no common version exists.
cf7014c5f087 obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22339
diff changeset
749 """
cf7014c5f087 obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22339
diff changeset
750 versions.sort(reverse=True)
cf7014c5f087 obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22339
diff changeset
751 # search for highest version known on both side
cf7014c5f087 obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22339
diff changeset
752 for v in versions:
cf7014c5f087 obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22339
diff changeset
753 if v in formats:
cf7014c5f087 obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22339
diff changeset
754 return v
cf7014c5f087 obsolete: add a `commonversion` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22339
diff changeset
755 return None
17295
1f08ecc7febb pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17253
diff changeset
756
1f08ecc7febb pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17253
diff changeset
757 # arbitrary picked to fit into 8K limit from HTTP server
1f08ecc7febb pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17253
diff changeset
758 # you have to take in account:
1f08ecc7febb pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17253
diff changeset
759 # - the version header
1f08ecc7febb pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17253
diff changeset
760 # - the base85 encoding
1f08ecc7febb pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17253
diff changeset
761 _maxpayload = 5300
17075
28ed1c4511ce obsolete: exchange obsolete marker over pushkey
Pierre-Yves.David@ens-lyon.org
parents: 17073
diff changeset
762
20599
dad29624b056 obsolete: extract encoding of marker for pushkey from the list key function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20585
diff changeset
763 def _pushkeyescape(markers):
dad29624b056 obsolete: extract encoding of marker for pushkey from the list key function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20585
diff changeset
764 """encode markers into a dict suitable for pushkey exchange
dad29624b056 obsolete: extract encoding of marker for pushkey from the list key function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20585
diff changeset
765
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 20599
diff changeset
766 - binary data is base85 encoded
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 20599
diff changeset
767 - split in chunks smaller than 5300 bytes"""
17295
1f08ecc7febb pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17253
diff changeset
768 keys = {}
1f08ecc7febb pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17253
diff changeset
769 parts = []
1f08ecc7febb pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17253
diff changeset
770 currentlen = _maxpayload * 2 # ensure we create a new part
20599
dad29624b056 obsolete: extract encoding of marker for pushkey from the list key function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20585
diff changeset
771 for marker in markers:
22329
ed37aa74d208 obsolete: rename _encodeonemarker to _fm0encodeonemarkers
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22328
diff changeset
772 nextdata = _fm0encodeonemarker(marker)
17295
1f08ecc7febb pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17253
diff changeset
773 if (len(nextdata) + currentlen > _maxpayload):
1f08ecc7febb pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17253
diff changeset
774 currentpart = []
1f08ecc7febb pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17253
diff changeset
775 currentlen = 0
1f08ecc7febb pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17253
diff changeset
776 parts.append(currentpart)
1f08ecc7febb pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17253
diff changeset
777 currentpart.append(nextdata)
17304
0368fc55d572 obsolete: properly increment currentlen when building pushkey payload
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17298
diff changeset
778 currentlen += len(nextdata)
17295
1f08ecc7febb pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17253
diff changeset
779 for idx, part in enumerate(reversed(parts)):
22327
f737631a9f0a obsolete: rename all _fm to _fm0
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22326
diff changeset
780 data = ''.join([_pack('>B', _fm0version)] + part)
32245
4462a981e8df base85: proxy through util module
Yuya Nishihara <yuya@tcha.org>
parents: 31483
diff changeset
781 keys['dump%i' % idx] = util.b85encode(data)
17295
1f08ecc7febb pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17253
diff changeset
782 return keys
17073
3a79a5682af1 obsolete: add easy way to iterate over obsolete marker object
Pierre-Yves.David@ens-lyon.org
parents: 17072
diff changeset
783
20599
dad29624b056 obsolete: extract encoding of marker for pushkey from the list key function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20585
diff changeset
784 def listmarkers(repo):
dad29624b056 obsolete: extract encoding of marker for pushkey from the list key function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20585
diff changeset
785 """List markers over pushkey"""
dad29624b056 obsolete: extract encoding of marker for pushkey from the list key function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20585
diff changeset
786 if not repo.obsstore:
dad29624b056 obsolete: extract encoding of marker for pushkey from the list key function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20585
diff changeset
787 return {}
25118
e632a2429982 obsolete: sort obsmarkers during exchange
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24928
diff changeset
788 return _pushkeyescape(sorted(repo.obsstore))
20599
dad29624b056 obsolete: extract encoding of marker for pushkey from the list key function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20585
diff changeset
789
17075
28ed1c4511ce obsolete: exchange obsolete marker over pushkey
Pierre-Yves.David@ens-lyon.org
parents: 17073
diff changeset
790 def pushmarker(repo, key, old, new):
28ed1c4511ce obsolete: exchange obsolete marker over pushkey
Pierre-Yves.David@ens-lyon.org
parents: 17073
diff changeset
791 """Push markers over pushkey"""
17295
1f08ecc7febb pushkey: splits obsolete marker exchange into multiple keys
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17253
diff changeset
792 if not key.startswith('dump'):
17075
28ed1c4511ce obsolete: exchange obsolete marker over pushkey
Pierre-Yves.David@ens-lyon.org
parents: 17073
diff changeset
793 repo.ui.warn(_('unknown key: %r') % key)
32841
e65ff29dbeb0 pushkey: use False/True for return values from push functions
Martin von Zweigbergk <martinvonz@google.com>
parents: 32794
diff changeset
794 return False
17075
28ed1c4511ce obsolete: exchange obsolete marker over pushkey
Pierre-Yves.David@ens-lyon.org
parents: 17073
diff changeset
795 if old:
21098
399d7770eef2 obsolete: add '%' specifier to the format string to avoid TypeError at runtime
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 21024
diff changeset
796 repo.ui.warn(_('unexpected old value for %r') % key)
32841
e65ff29dbeb0 pushkey: use False/True for return values from push functions
Martin von Zweigbergk <martinvonz@google.com>
parents: 32794
diff changeset
797 return False
32245
4462a981e8df base85: proxy through util module
Yuya Nishihara <yuya@tcha.org>
parents: 31483
diff changeset
798 data = util.b85decode(new)
35574
09285733ad71 obsolete: use context manager for transaction in pushmarker()
Martin von Zweigbergk <martinvonz@google.com>
parents: 35573
diff changeset
799 with repo.lock(), repo.transaction('pushkey: obsolete markers') as tr:
09285733ad71 obsolete: use context manager for transaction in pushmarker()
Martin von Zweigbergk <martinvonz@google.com>
parents: 35573
diff changeset
800 repo.obsstore.mergemarkers(tr, data)
09285733ad71 obsolete: use context manager for transaction in pushmarker()
Martin von Zweigbergk <martinvonz@google.com>
parents: 35573
diff changeset
801 repo.invalidatevolatilesets()
09285733ad71 obsolete: use context manager for transaction in pushmarker()
Martin von Zweigbergk <martinvonz@google.com>
parents: 35573
diff changeset
802 return True
17073
3a79a5682af1 obsolete: add easy way to iterate over obsolete marker object
Pierre-Yves.David@ens-lyon.org
parents: 17072
diff changeset
803
17828
9495be4126ef obsolete: add the detection of bumped changeset.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17827
diff changeset
804 # mapping of 'set-name' -> <function to compute this set>
17469
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
805 cachefuncs = {}
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
806 def cachefor(name):
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
807 """Decorator to register a function as computing the cache for a set"""
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
808 def decorator(func):
32902
9d472b219fb0 obsolete: use ProgrammingError over assert for volatile set registration
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 32841
diff changeset
809 if name in cachefuncs:
9d472b219fb0 obsolete: use ProgrammingError over assert for volatile set registration
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 32841
diff changeset
810 msg = "duplicated registration for volatileset '%s' (existing: %r)"
9d472b219fb0 obsolete: use ProgrammingError over assert for volatile set registration
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 32841
diff changeset
811 raise error.ProgrammingError(msg % (name, cachefuncs[name]))
17469
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
812 cachefuncs[name] = func
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
813 return func
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
814 return decorator
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
815
17825
3cc06457f15e obsolete: rename `getobscache` into `getrevs`
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17776
diff changeset
816 def getrevs(repo, name):
17469
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
817 """Return the set of revision that belong to the <name> set
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
818
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
819 Such access may compute the set and cache it for future use"""
18001
e02feadd15ea clfilter: unfilter computation of obsolescence related computation
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17831
diff changeset
820 repo = repo.unfiltered()
17469
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
821 if not repo.obsstore:
22507
5c00c5298f98 obsolete: ensure that `getrevs` always return a set
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22394
diff changeset
822 return frozenset()
17469
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
823 if name not in repo.obsstore.caches:
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
824 repo.obsstore.caches[name] = cachefuncs[name](repo)
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
825 return repo.obsstore.caches[name]
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
826
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
827 # To be simple we need to invalidate obsolescence cache when:
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
828 #
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
829 # - new changeset is added:
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
830 # - public phase is changed
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
831 # - obsolescence marker are added
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
832 # - strip is used a repo
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
833 def clearobscaches(repo):
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
834 """Remove all obsolescence related cache from a repo
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
835
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
836 This remove all cache in obsstore is the obsstore already exist on the
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
837 repo.
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
838
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
839 (We could be smarter here given the exact event that trigger the cache
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
840 clearing)"""
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
841 # only clear cache is there is obsstore data in this repo
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
842 if 'obsstore' in repo._filecache:
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
843 repo.obsstore.caches.clear()
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
844
33136
765c6ab07a88 obsolete: provide a small function to retrieve all mutable revisions
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32902
diff changeset
845 def _mutablerevs(repo):
765c6ab07a88 obsolete: provide a small function to retrieve all mutable revisions
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32902
diff changeset
846 """the set of mutable revision in the repository"""
765c6ab07a88 obsolete: provide a small function to retrieve all mutable revisions
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32902
diff changeset
847 return repo._phasecache.getrevset(repo, (phases.draft, phases.secret))
765c6ab07a88 obsolete: provide a small function to retrieve all mutable revisions
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32902
diff changeset
848
17469
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
849 @cachefor('obsolete')
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
850 def _computeobsoleteset(repo):
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
851 """the set of obsolete revisions"""
27784
432242f41d9f obsolete: make _computeobsoleteset much faster
Laurent Charignon <lc2817@columbia.edu>
parents: 27332
diff changeset
852 getnode = repo.changelog.node
33136
765c6ab07a88 obsolete: provide a small function to retrieve all mutable revisions
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32902
diff changeset
853 notpublic = _mutablerevs(repo)
32708
2c1400d43fd2 obsstore: minor optimization for the obsolete revset
Jun Wu <quark@fb.com>
parents: 32650
diff changeset
854 isobs = repo.obsstore.successors.__contains__
2c1400d43fd2 obsstore: minor optimization for the obsolete revset
Jun Wu <quark@fb.com>
parents: 32650
diff changeset
855 obs = set(r for r in notpublic if isobs(getnode(r)))
18271
67872e939945 performance: speedup computation of obsolete revisions
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18101
diff changeset
856 return obs
17469
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
857
33798
d4b7496f7d0b obsolete: rename unstable volatile set into orphan volatile set
Boris Feld <boris.feld@octobus.net>
parents: 33793
diff changeset
858 @cachefor('orphan')
d4b7496f7d0b obsolete: rename unstable volatile set into orphan volatile set
Boris Feld <boris.feld@octobus.net>
parents: 33793
diff changeset
859 def _computeorphanset(repo):
17469
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
860 """the set of non obsolete revisions with obsolete parents"""
33137
31ab1912678a obsolete: skip 'changectx' usage in unstable computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33136
diff changeset
861 pfunc = repo.changelog.parentrevs
31ab1912678a obsolete: skip 'changectx' usage in unstable computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33136
diff changeset
862 mutable = _mutablerevs(repo)
31ab1912678a obsolete: skip 'changectx' usage in unstable computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33136
diff changeset
863 obsolete = getrevs(repo, 'obsolete')
31ab1912678a obsolete: skip 'changectx' usage in unstable computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33136
diff changeset
864 others = mutable - obsolete
24928
876a2ebfbf4f obsolete: speed up unstable computation
Laurent Charignon <lcharignon@fb.com>
parents: 24927
diff changeset
865 unstable = set()
33137
31ab1912678a obsolete: skip 'changectx' usage in unstable computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33136
diff changeset
866 for r in sorted(others):
24928
876a2ebfbf4f obsolete: speed up unstable computation
Laurent Charignon <lcharignon@fb.com>
parents: 24927
diff changeset
867 # A rev is unstable if one of its parent is obsolete or unstable
876a2ebfbf4f obsolete: speed up unstable computation
Laurent Charignon <lcharignon@fb.com>
parents: 24927
diff changeset
868 # this works since we traverse following growing rev order
33137
31ab1912678a obsolete: skip 'changectx' usage in unstable computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33136
diff changeset
869 for p in pfunc(r):
31ab1912678a obsolete: skip 'changectx' usage in unstable computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33136
diff changeset
870 if p in obsolete or p in unstable:
31ab1912678a obsolete: skip 'changectx' usage in unstable computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33136
diff changeset
871 unstable.add(r)
31ab1912678a obsolete: skip 'changectx' usage in unstable computation
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 33136
diff changeset
872 break
24928
876a2ebfbf4f obsolete: speed up unstable computation
Laurent Charignon <lcharignon@fb.com>
parents: 24927
diff changeset
873 return unstable
17469
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
874
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
875 @cachefor('suspended')
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
876 def _computesuspendedset(repo):
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
877 """the set of obsolete parents with non obsolete descendants"""
33798
d4b7496f7d0b obsolete: rename unstable volatile set into orphan volatile set
Boris Feld <boris.feld@octobus.net>
parents: 33793
diff changeset
878 suspended = repo.changelog.ancestors(getrevs(repo, 'orphan'))
18276
834ef7e70d0f performance: speedup computation of suspended revisions
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18275
diff changeset
879 return set(r for r in getrevs(repo, 'obsolete') if r in suspended)
17469
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
880
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
881 @cachefor('extinct')
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
882 def _computeextinctset(repo):
fb72eec7efd8 obsolete: introduce caches for all meaningful sets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17429
diff changeset
883 """the set of obsolete parents without non obsolete descendants"""
18277
a58260bc101f performance: speedup computation of extinct revisions
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18276
diff changeset
884 return getrevs(repo, 'obsolete') - getrevs(repo, 'suspended')
17474
f85816af6294 obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17469
diff changeset
885
33800
9fa874fb34e1 obsolete: rename bumped volatile set into phasedivergent volatile set
Boris Feld <boris.feld@octobus.net>
parents: 33799
diff changeset
886 @cachefor('phasedivergent')
9fa874fb34e1 obsolete: rename bumped volatile set into phasedivergent volatile set
Boris Feld <boris.feld@octobus.net>
parents: 33799
diff changeset
887 def _computephasedivergentset(repo):
17828
9495be4126ef obsolete: add the detection of bumped changeset.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17827
diff changeset
888 """the set of revs trying to obsolete public revisions"""
20207
cd62532c62a1 obsolete: order of magnitude speedup in _computebumpedset
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20206
diff changeset
889 bumped = set()
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 20599
diff changeset
890 # util function (avoid attribute lookup in the loop)
20207
cd62532c62a1 obsolete: order of magnitude speedup in _computebumpedset
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20206
diff changeset
891 phase = repo._phasecache.phase # would be faster to grab the full list
cd62532c62a1 obsolete: order of magnitude speedup in _computebumpedset
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20206
diff changeset
892 public = phases.public
cd62532c62a1 obsolete: order of magnitude speedup in _computebumpedset
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20206
diff changeset
893 cl = repo.changelog
cd62532c62a1 obsolete: order of magnitude speedup in _computebumpedset
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20206
diff changeset
894 torev = cl.nodemap.get
35157
82680919d75e obsolete: drop usage of changectx in '_computephasedivergentset'
Boris Feld <boris.feld@octobus.net>
parents: 34902
diff changeset
895 tonode = cl.node
82680919d75e obsolete: drop usage of changectx in '_computephasedivergentset'
Boris Feld <boris.feld@octobus.net>
parents: 34902
diff changeset
896 for rev in repo.revs('(not public()) and (not obsolete())'):
20207
cd62532c62a1 obsolete: order of magnitude speedup in _computebumpedset
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20206
diff changeset
897 # We only evaluate mutable, non-obsolete revision
35157
82680919d75e obsolete: drop usage of changectx in '_computephasedivergentset'
Boris Feld <boris.feld@octobus.net>
parents: 34902
diff changeset
898 node = tonode(rev)
33760
0c3112f17610 obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents: 33759
diff changeset
899 # (future) A cache of predecessors may worth if split is very common
33761
e6d8ee3c9ec3 obsutil: rename allprecursors into allpredecessors
Boris Feld <boris.feld@octobus.net>
parents: 33760
diff changeset
900 for pnode in obsutil.allpredecessors(repo.obsstore, [node],
24927
cd0068232ec0 obsolete: speed up computation of bumped revset
Laurent Charignon <lcharignon@fb.com>
parents: 24393
diff changeset
901 ignoreflags=bumpedfix):
cd0068232ec0 obsolete: speed up computation of bumped revset
Laurent Charignon <lcharignon@fb.com>
parents: 24393
diff changeset
902 prev = torev(pnode) # unfiltered! but so is phasecache
cd0068232ec0 obsolete: speed up computation of bumped revset
Laurent Charignon <lcharignon@fb.com>
parents: 24393
diff changeset
903 if (prev is not None) and (phase(repo, prev) <= public):
33760
0c3112f17610 obsolete: rename precursor into predecessor in obsolete docstrings
Boris Feld <boris.feld@octobus.net>
parents: 33759
diff changeset
904 # we have a public predecessor
24927
cd0068232ec0 obsolete: speed up computation of bumped revset
Laurent Charignon <lcharignon@fb.com>
parents: 24393
diff changeset
905 bumped.add(rev)
cd0068232ec0 obsolete: speed up computation of bumped revset
Laurent Charignon <lcharignon@fb.com>
parents: 24393
diff changeset
906 break # Next draft!
20207
cd62532c62a1 obsolete: order of magnitude speedup in _computebumpedset
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 20206
diff changeset
907 return bumped
17828
9495be4126ef obsolete: add the detection of bumped changeset.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17827
diff changeset
908
33799
f3f06c260e9e obsolete: rename divergent volatile set into contentdivergent volatile set
Boris Feld <boris.feld@octobus.net>
parents: 33798
diff changeset
909 @cachefor('contentdivergent')
f3f06c260e9e obsolete: rename divergent volatile set into contentdivergent volatile set
Boris Feld <boris.feld@octobus.net>
parents: 33798
diff changeset
910 def _computecontentdivergentset(repo):
18070
af632936d3d9 obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18069
diff changeset
911 """the set of rev that compete to be the final successors of some revision.
af632936d3d9 obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18069
diff changeset
912 """
af632936d3d9 obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18069
diff changeset
913 divergent = set()
af632936d3d9 obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18069
diff changeset
914 obsstore = repo.obsstore
af632936d3d9 obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18069
diff changeset
915 newermap = {}
35158
5cd6682b5a90 obsolete: drop usage of changectx in '_computecontentdivergentset'
Boris Feld <boris.feld@octobus.net>
parents: 35157
diff changeset
916 tonode = repo.changelog.node
5cd6682b5a90 obsolete: drop usage of changectx in '_computecontentdivergentset'
Boris Feld <boris.feld@octobus.net>
parents: 35157
diff changeset
917 for rev in repo.revs('(not public()) - obsolete()'):
5cd6682b5a90 obsolete: drop usage of changectx in '_computecontentdivergentset'
Boris Feld <boris.feld@octobus.net>
parents: 35157
diff changeset
918 node = tonode(rev)
5cd6682b5a90 obsolete: drop usage of changectx in '_computecontentdivergentset'
Boris Feld <boris.feld@octobus.net>
parents: 35157
diff changeset
919 mark = obsstore.predecessors.get(node, ())
18070
af632936d3d9 obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18069
diff changeset
920 toprocess = set(mark)
24393
77eace2a63cb obsolete: avoid infinite loop from obs-cycle in divergence (issue4126)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24335
diff changeset
921 seen = set()
18070
af632936d3d9 obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18069
diff changeset
922 while toprocess:
af632936d3d9 obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18069
diff changeset
923 prec = toprocess.pop()[0]
24393
77eace2a63cb obsolete: avoid infinite loop from obs-cycle in divergence (issue4126)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24335
diff changeset
924 if prec in seen:
77eace2a63cb obsolete: avoid infinite loop from obs-cycle in divergence (issue4126)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24335
diff changeset
925 continue # emergency cycle hanging prevention
77eace2a63cb obsolete: avoid infinite loop from obs-cycle in divergence (issue4126)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24335
diff changeset
926 seen.add(prec)
18070
af632936d3d9 obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18069
diff changeset
927 if prec not in newermap:
33273
5724aaa99dd6 obsolete: pass cache argument of successors set explicitly
Boris Feld <boris.feld@octobus.net>
parents: 33251
diff changeset
928 obsutil.successorssets(repo, prec, cache=newermap)
18070
af632936d3d9 obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18069
diff changeset
929 newer = [n for n in newermap[prec] if n]
af632936d3d9 obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18069
diff changeset
930 if len(newer) > 1:
35158
5cd6682b5a90 obsolete: drop usage of changectx in '_computecontentdivergentset'
Boris Feld <boris.feld@octobus.net>
parents: 35157
diff changeset
931 divergent.add(rev)
18070
af632936d3d9 obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18069
diff changeset
932 break
33759
d5acd967f95a obsstore: rename precursors into predecessors
Boris Feld <boris.feld@octobus.net>
parents: 33758
diff changeset
933 toprocess.update(obsstore.predecessors.get(prec, ()))
18070
af632936d3d9 obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18069
diff changeset
934 return divergent
af632936d3d9 obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18069
diff changeset
935
af632936d3d9 obsolete: detect divergent changesets
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 18069
diff changeset
936
32366
3546a771e376 obsolete: add operation metadata to rebase/amend/histedit obsmarkers
Durham Goode <durham@fb.com>
parents: 32318
diff changeset
937 def createmarkers(repo, relations, flag=0, date=None, metadata=None,
3546a771e376 obsolete: add operation metadata to rebase/amend/histedit obsmarkers
Durham Goode <durham@fb.com>
parents: 32318
diff changeset
938 operation=None):
17474
f85816af6294 obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17469
diff changeset
939 """Add obsolete markers between changesets in a repo
f85816af6294 obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17469
diff changeset
940
20517
2158e8f3cbd2 createmarkers: allow to pass metadata for a marker only
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 20516
diff changeset
941 <relations> must be an iterable of (<old>, (<new>, ...)[,{metadata}])
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 20599
diff changeset
942 tuple. `old` and `news` are changectx. metadata is an optional dictionary
20517
2158e8f3cbd2 createmarkers: allow to pass metadata for a marker only
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 20516
diff changeset
943 containing metadata for this marker only. It is merged with the global
2158e8f3cbd2 createmarkers: allow to pass metadata for a marker only
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 20516
diff changeset
944 metadata specified through the `metadata` argument of this function,
17474
f85816af6294 obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17469
diff changeset
945
f85816af6294 obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17469
diff changeset
946 Trying to obsolete a public changeset will raise an exception.
f85816af6294 obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17469
diff changeset
947
f85816af6294 obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17469
diff changeset
948 Current user and date are used except if specified otherwise in the
f85816af6294 obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17469
diff changeset
949 metadata attribute.
f85816af6294 obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17469
diff changeset
950
f85816af6294 obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17469
diff changeset
951 This function operates within a transaction of its own, but does
f85816af6294 obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17469
diff changeset
952 not take any lock on the repo.
f85816af6294 obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17469
diff changeset
953 """
f85816af6294 obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17469
diff changeset
954 # prepare metadata
f85816af6294 obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17469
diff changeset
955 if metadata is None:
f85816af6294 obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17469
diff changeset
956 metadata = {}
f85816af6294 obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17469
diff changeset
957 if 'user' not in metadata:
34575
dc91580a0a88 obsolete: add a devel.user.obsmarker
Boris Feld <boris.feld@octobus.net>
parents: 34414
diff changeset
958 develuser = repo.ui.config('devel', 'user.obsmarker')
dc91580a0a88 obsolete: add a devel.user.obsmarker
Boris Feld <boris.feld@octobus.net>
parents: 34414
diff changeset
959 if develuser:
dc91580a0a88 obsolete: add a devel.user.obsmarker
Boris Feld <boris.feld@octobus.net>
parents: 34414
diff changeset
960 metadata['user'] = develuser
dc91580a0a88 obsolete: add a devel.user.obsmarker
Boris Feld <boris.feld@octobus.net>
parents: 34414
diff changeset
961 else:
dc91580a0a88 obsolete: add a devel.user.obsmarker
Boris Feld <boris.feld@octobus.net>
parents: 34414
diff changeset
962 metadata['user'] = repo.ui.username()
34389
ddcef6d0b0ef obsolete: clean createmarkers part about operation
Boris Feld <boris.feld@octobus.net>
parents: 34380
diff changeset
963
ddcef6d0b0ef obsolete: clean createmarkers part about operation
Boris Feld <boris.feld@octobus.net>
parents: 34380
diff changeset
964 # Operation metadata handling
32393
f432897a9f49 obsmarker: add an experimental flag controlling "operation" recording
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32372
diff changeset
965 useoperation = repo.ui.configbool('experimental',
34863
b1e3f609bf45 config: invert evolution-related configuration aliases
Boris Feld <boris.feld@octobus.net>
parents: 34827
diff changeset
966 'evolution.track-operation')
32393
f432897a9f49 obsmarker: add an experimental flag controlling "operation" recording
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32372
diff changeset
967 if useoperation and operation:
32366
3546a771e376 obsolete: add operation metadata to rebase/amend/histedit obsmarkers
Durham Goode <durham@fb.com>
parents: 32318
diff changeset
968 metadata['operation'] = operation
34389
ddcef6d0b0ef obsolete: clean createmarkers part about operation
Boris Feld <boris.feld@octobus.net>
parents: 34380
diff changeset
969
34414
014d467f9d08 effectflag: store an empty effect flag for the moment
Boris Feld <boris.feld@octobus.net>
parents: 34408
diff changeset
970 # Effect flag metadata handling
014d467f9d08 effectflag: store an empty effect flag for the moment
Boris Feld <boris.feld@octobus.net>
parents: 34408
diff changeset
971 saveeffectflag = repo.ui.configbool('experimental',
34902
cc977ec0b8b9 config: also gather effect-flags on experimental.evolution
Boris Feld <boris.feld@octobus.net>
parents: 34864
diff changeset
972 'evolution.effect-flags')
34414
014d467f9d08 effectflag: store an empty effect flag for the moment
Boris Feld <boris.feld@octobus.net>
parents: 34408
diff changeset
973
35572
8b729856ca03 obsolete: use context manager for transaction in createmarkers()
Martin von Zweigbergk <martinvonz@google.com>
parents: 35374
diff changeset
974 with repo.transaction('add-obsolescence-marker') as tr:
27984
e60e13a86529 obsolete: fix n^2 marker computation behavior
Durham Goode <durham@fb.com>
parents: 27784
diff changeset
975 markerargs = []
20517
2158e8f3cbd2 createmarkers: allow to pass metadata for a marker only
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 20516
diff changeset
976 for rel in relations:
2158e8f3cbd2 createmarkers: allow to pass metadata for a marker only
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 20516
diff changeset
977 prec = rel[0]
2158e8f3cbd2 createmarkers: allow to pass metadata for a marker only
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 20516
diff changeset
978 sucs = rel[1]
2158e8f3cbd2 createmarkers: allow to pass metadata for a marker only
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 20516
diff changeset
979 localmetadata = metadata.copy()
2158e8f3cbd2 createmarkers: allow to pass metadata for a marker only
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 20516
diff changeset
980 if 2 < len(rel):
2158e8f3cbd2 createmarkers: allow to pass metadata for a marker only
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 20516
diff changeset
981 localmetadata.update(rel[2])
2158e8f3cbd2 createmarkers: allow to pass metadata for a marker only
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 20516
diff changeset
982
17474
f85816af6294 obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17469
diff changeset
983 if not prec.mutable():
29389
98e8313dcd9e i18n: translate abort messages
liscju <piotr.listkiewicz@gmail.com>
parents: 29281
diff changeset
984 raise error.Abort(_("cannot obsolete public changeset: %s")
25412
443d4635e630 phases: add `hg help phases` hint to failures to edit public commits
Jordi Guti?rrez Hermoso <jordigh@octave.org>
parents: 25411
diff changeset
985 % prec,
29980
42d031560294 obsolete: use single quotes in use warning
timeless <timeless@mozdev.org>
parents: 29898
diff changeset
986 hint="see 'hg help phases' for details")
17474
f85816af6294 obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17469
diff changeset
987 nprec = prec.node()
f85816af6294 obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17469
diff changeset
988 nsucs = tuple(s.node() for s in sucs)
22256
3ae6cc6173e3 createmarkers: automatically record the parent of pruned changesets
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22255
diff changeset
989 npare = None
3ae6cc6173e3 createmarkers: automatically record the parent of pruned changesets
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22255
diff changeset
990 if not nsucs:
3ae6cc6173e3 createmarkers: automatically record the parent of pruned changesets
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22255
diff changeset
991 npare = tuple(p.node() for p in prec.parents())
17474
f85816af6294 obsolete: add a high level function to create an obsolete marker
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 17469
diff changeset
992 if nprec in nsucs:
29389
98e8313dcd9e i18n: translate abort messages
liscju <piotr.listkiewicz@gmail.com>
parents: 29281
diff changeset
993 raise error.Abort(_("changeset %s cannot obsolete itself")
98e8313dcd9e i18n: translate abort messages
liscju <piotr.listkiewicz@gmail.com>
parents: 29281
diff changeset
994 % prec)
27984
e60e13a86529 obsolete: fix n^2 marker computation behavior
Durham Goode <durham@fb.com>
parents: 27784
diff changeset
995
34414
014d467f9d08 effectflag: store an empty effect flag for the moment
Boris Feld <boris.feld@octobus.net>
parents: 34408
diff changeset
996 # Effect flag can be different by relation
014d467f9d08 effectflag: store an empty effect flag for the moment
Boris Feld <boris.feld@octobus.net>
parents: 34408
diff changeset
997 if saveeffectflag:
014d467f9d08 effectflag: store an empty effect flag for the moment
Boris Feld <boris.feld@octobus.net>
parents: 34408
diff changeset
998 # The effect flag is saved in a versioned field name for future
014d467f9d08 effectflag: store an empty effect flag for the moment
Boris Feld <boris.feld@octobus.net>
parents: 34408
diff changeset
999 # evolution
014d467f9d08 effectflag: store an empty effect flag for the moment
Boris Feld <boris.feld@octobus.net>
parents: 34408
diff changeset
1000 effectflag = obsutil.geteffectflag(rel)
014d467f9d08 effectflag: store an empty effect flag for the moment
Boris Feld <boris.feld@octobus.net>
parents: 34408
diff changeset
1001 localmetadata[obsutil.EFFECTFLAGFIELD] = "%d" % effectflag
014d467f9d08 effectflag: store an empty effect flag for the moment
Boris Feld <boris.feld@octobus.net>
parents: 34408
diff changeset
1002
27984
e60e13a86529 obsolete: fix n^2 marker computation behavior
Durham Goode <durham@fb.com>
parents: 27784
diff changeset
1003 # Creating the marker causes the hidden cache to become invalid,
e60e13a86529 obsolete: fix n^2 marker computation behavior
Durham Goode <durham@fb.com>
parents: 27784
diff changeset
1004 # which causes recomputation when we ask for prec.parents() above.
e60e13a86529 obsolete: fix n^2 marker computation behavior
Durham Goode <durham@fb.com>
parents: 27784
diff changeset
1005 # Resulting in n^2 behavior. So let's prepare all of the args
e60e13a86529 obsolete: fix n^2 marker computation behavior
Durham Goode <durham@fb.com>
parents: 27784
diff changeset
1006 # first, then create the markers.
e60e13a86529 obsolete: fix n^2 marker computation behavior
Durham Goode <durham@fb.com>
parents: 27784
diff changeset
1007 markerargs.append((nprec, nsucs, npare, localmetadata))
e60e13a86529 obsolete: fix n^2 marker computation behavior
Durham Goode <durham@fb.com>
parents: 27784
diff changeset
1008
e60e13a86529 obsolete: fix n^2 marker computation behavior
Durham Goode <durham@fb.com>
parents: 27784
diff changeset
1009 for args in markerargs:
e60e13a86529 obsolete: fix n^2 marker computation behavior
Durham Goode <durham@fb.com>
parents: 27784
diff changeset
1010 nprec, nsucs, npare, localmetadata = args
22256
3ae6cc6173e3 createmarkers: automatically record the parent of pruned changesets
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22255
diff changeset
1011 repo.obsstore.create(tr, nprec, nsucs, flag, parents=npare,
32449
08d02c1d7e67 devel: use default-date config field when creating obsmarkers
Boris Feld <boris.feld@octobus.net>
parents: 32425
diff changeset
1012 date=date, metadata=localmetadata,
08d02c1d7e67 devel: use default-date config field when creating obsmarkers
Boris Feld <boris.feld@octobus.net>
parents: 32425
diff changeset
1013 ui=repo.ui)
18101
a464deecc9dd clfilter: add a cache on repo for set of revision to filter for a given set.
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18070
diff changeset
1014 repo.filteredrevcache.clear()