mercurial-scm/hg: mercurial/metadata.py annotate

annotate mercurial/metadata.py @ 45635:9003e6524f78

changing-files: drop the now useless changelogrevision argument Since all filename are now included in the sidedata block, we no longer need to decode the `files` from the revision. Differential Revision: https://phab.mercurial-scm.org/D9091

author	Pierre-Yves David <pierre-yves.david@octobus.net>
date	Tue, 15 Sep 2020 10:49:50 +0200
parents	9a6b409b8ebc
children	a475db79d84d

rev	line source
44940 4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	1 # metadata.py -- code related to various metadata computation and access.
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	2 #
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	3 # Copyright 2019 Google, Inc <martinvonz@google.com>
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	4 # Copyright 2020 Pierre-Yves David <pierre-yves.david@octobus.net>
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	5 #
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	6 # This software may be used and distributed according to the terms of the
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	7 # GNU General Public License version 2 or any later version.
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	8 from __future__ import absolute_import, print_function
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	9
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	10 import multiprocessing
45634 9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	11 import struct
44940 4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	12
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	13 from . import (
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	14 error,
44941 edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	15 node,
44940 4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	16 pycompat,
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	17 util,
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	18 )
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	19
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	20 from .revlogutils import (
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	21 flagutil as sidedataflag,
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	22 sidedata as sidedatamod,
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	23 )
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	24
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	25
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	26 class ChangingFiles(object):
45621 646a676f5365 changing-files: fix docstring Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45613 diff changeset	27 """A class recording the changes made to files by a changeset
45506 1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	28
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	29 Actions performed on files are gathered into 3 sets:
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	30
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	31 - added: files actively added in the changeset.
45611 e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	32 - merged: files whose history got merged
45506 1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	33 - removed: files removed in the revision
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	34 - touched: files affected by the merge
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	35
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	36 and copies information is held by 2 mappings
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	37
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	38 - copied_from_p1: {"<new-name>": "<source-name-in-p1>"} mapping for copies
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	39 - copied_from_p2: {"<new-name>": "<source-name-in-p2>"} mapping for copies
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	40
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	41 See their inline help for details.
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	42 """
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	43
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	44 def __init__(
45589 3d5b2b8e93fd changing-files: move default constructor value to None Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45571 diff changeset	45 self,
3d5b2b8e93fd changing-files: move default constructor value to None Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45571 diff changeset	46 touched=None,
3d5b2b8e93fd changing-files: move default constructor value to None Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45571 diff changeset	47 added=None,
3d5b2b8e93fd changing-files: move default constructor value to None Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45571 diff changeset	48 removed=None,
45611 e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	49 merged=None,
45589 3d5b2b8e93fd changing-files: move default constructor value to None Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45571 diff changeset	50 p1_copies=None,
3d5b2b8e93fd changing-files: move default constructor value to None Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45571 diff changeset	51 p2_copies=None,
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	52 ):
45589 3d5b2b8e93fd changing-files: move default constructor value to None Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45571 diff changeset	53 self._added = set(() if added is None else added)
45611 e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	54 self._merged = set(() if merged is None else merged)
45589 3d5b2b8e93fd changing-files: move default constructor value to None Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45571 diff changeset	55 self._removed = set(() if removed is None else removed)
3d5b2b8e93fd changing-files: move default constructor value to None Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45571 diff changeset	56 self._touched = set(() if touched is None else touched)
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	57 self._touched.update(self._added)
45611 e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	58 self._touched.update(self._merged)
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	59 self._touched.update(self._removed)
45589 3d5b2b8e93fd changing-files: move default constructor value to None Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45571 diff changeset	60 self._p1_copies = dict(() if p1_copies is None else p1_copies)
3d5b2b8e93fd changing-files: move default constructor value to None Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45571 diff changeset	61 self._p2_copies = dict(() if p2_copies is None else p2_copies)
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	62
45507 df87821081ee changing-files: implement equality checking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45506 diff changeset	63 def __eq__(self, other):
df87821081ee changing-files: implement equality checking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45506 diff changeset	64 return (
df87821081ee changing-files: implement equality checking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45506 diff changeset	65 self.added == other.added
45611 e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	66 and self.merged == other.merged
45507 df87821081ee changing-files: implement equality checking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45506 diff changeset	67 and self.removed == other.removed
df87821081ee changing-files: implement equality checking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45506 diff changeset	68 and self.touched == other.touched
df87821081ee changing-files: implement equality checking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45506 diff changeset	69 and self.copied_from_p1 == other.copied_from_p1
df87821081ee changing-files: implement equality checking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45506 diff changeset	70 and self.copied_from_p2 == other.copied_from_p2
df87821081ee changing-files: implement equality checking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45506 diff changeset	71 )
df87821081ee changing-files: implement equality checking Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45506 diff changeset	72
45623 d31483377673 changing-files: cache the various property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45622 diff changeset	73 @util.propertycache
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	74 def added(self):
45506 1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	75 """files actively added in the changeset
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	76
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	77 Any file present in that revision that was absent in all the changeset's
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	78 parents.
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	79
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	80 In case of merge, this means a file absent in one of the parents but
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	81 existing in the other will not be contained in this set. (They were
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	82 added by an ancestor)
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	83 """
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	84 return frozenset(self._added)
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	85
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	86 def mark_added(self, filename):
45623 d31483377673 changing-files: cache the various property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45622 diff changeset	87 if 'added' in vars(self):
d31483377673 changing-files: cache the various property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45622 diff changeset	88 del self.added
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	89 self._added.add(filename)
45622 42bb6c4f8106 changing-files: always use `mark_touched` to update the touched set Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45621 diff changeset	90 self.mark_touched(filename)
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	91
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	92 def update_added(self, filenames):
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	93 for f in filenames:
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	94 self.mark_added(f)
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	95
45623 d31483377673 changing-files: cache the various property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45622 diff changeset	96 @util.propertycache
45611 e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	97 def merged(self):
e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	98 """files actively merged during a merge
e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	99
e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	100 Any modified files which had modification on both size that needed merging.
e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	101
e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	102 In this case a new filenode was created and it has two parents.
e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	103 """
e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	104 return frozenset(self._merged)
e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	105
e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	106 def mark_merged(self, filename):
45623 d31483377673 changing-files: cache the various property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45622 diff changeset	107 if 'merged' in vars(self):
d31483377673 changing-files: cache the various property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45622 diff changeset	108 del self.merged
45611 e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	109 self._merged.add(filename)
45622 42bb6c4f8106 changing-files: always use `mark_touched` to update the touched set Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45621 diff changeset	110 self.mark_touched(filename)
45611 e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	111
e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	112 def update_merged(self, filenames):
e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	113 for f in filenames:
e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	114 self.mark_merged(f)
e5578dbe36cb changing-files: add the ability to track merged files too Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45589 diff changeset	115
45623 d31483377673 changing-files: cache the various property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45622 diff changeset	116 @util.propertycache
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	117 def removed(self):
45506 1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	118 """files actively removed by the changeset
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	119
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	120 In case of merge this will only contain the set of files removing "new"
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	121 content. For any file absent in the current changeset:
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	122
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	123 a) If the file exists in both parents, it is clearly "actively" removed
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	124 by this changeset.
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	125
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	126 b) If a file exists in only one parent and in none of the common
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	127 ancestors, then the file was newly added in one of the merged branches
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	128 and then got "actively" removed.
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	129
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	130 c) If a file exists in only one parent and at least one of the common
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	131 ancestors using the same filenode, then the file was unchanged on one
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	132 side and deleted on the other side. The merge "passively" propagated
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	133 that deletion, but didn't "actively" remove the file. In this case the
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	134 file is not included in the `removed` set.
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	135
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	136 d) If a file exists in only one parent and at least one of the common
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	137 ancestors using a different filenode, then the file was changed on one
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	138 side and removed on the other side. The merge process "actively"
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	139 decided to drop the new change and delete the file. Unlike in the
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	140 previous case, (c), the file included in the `removed` set.
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	141
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	142 Summary table for merge:
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	143
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	144 case \| exists in parents \| exists in gca \|\| removed
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	145 (a) \| both \| * \|\| yes
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	146 (b) \| one \| none \|\| yes
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	147 (c) \| one \| same filenode \|\| no
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	148 (d) \| one \| new filenode \|\| yes
1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	149 """
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	150 return frozenset(self._removed)
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	151
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	152 def mark_removed(self, filename):
45623 d31483377673 changing-files: cache the various property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45622 diff changeset	153 if 'removed' in vars(self):
d31483377673 changing-files: cache the various property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45622 diff changeset	154 del self.removed
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	155 self._removed.add(filename)
45622 42bb6c4f8106 changing-files: always use `mark_touched` to update the touched set Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45621 diff changeset	156 self.mark_touched(filename)
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	157
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	158 def update_removed(self, filenames):
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	159 for f in filenames:
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	160 self.mark_removed(f)
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	161
45623 d31483377673 changing-files: cache the various property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45622 diff changeset	162 @util.propertycache
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	163 def touched(self):
45506 1f50bcc96595 changing-files: document the various sets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45325 diff changeset	164 """files either actively modified, added or removed"""
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	165 return frozenset(self._touched)
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	166
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	167 def mark_touched(self, filename):
45623 d31483377673 changing-files: cache the various property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45622 diff changeset	168 if 'touched' in vars(self):
d31483377673 changing-files: cache the various property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45622 diff changeset	169 del self.touched
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	170 self._touched.add(filename)
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	171
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	172 def update_touched(self, filenames):
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	173 for f in filenames:
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	174 self.mark_touched(f)
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	175
45623 d31483377673 changing-files: cache the various property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45622 diff changeset	176 @util.propertycache
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	177 def copied_from_p1(self):
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	178 return self._p1_copies.copy()
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	179
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	180 def mark_copied_from_p1(self, source, dest):
45623 d31483377673 changing-files: cache the various property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45622 diff changeset	181 if 'copied_from_p1' in vars(self):
d31483377673 changing-files: cache the various property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45622 diff changeset	182 del self.copied_from_p1
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	183 self._p1_copies[dest] = source
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	184
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	185 def update_copies_from_p1(self, copies):
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	186 for dest, source in copies.items():
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	187 self.mark_copied_from_p1(source, dest)
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	188
45623 d31483377673 changing-files: cache the various property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45622 diff changeset	189 @util.propertycache
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	190 def copied_from_p2(self):
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	191 return self._p2_copies.copy()
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	192
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	193 def mark_copied_from_p2(self, source, dest):
45623 d31483377673 changing-files: cache the various property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45622 diff changeset	194 if 'copied_from_p2' in vars(self):
d31483377673 changing-files: cache the various property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45622 diff changeset	195 del self.copied_from_p2
45323 aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	196 self._p2_copies[dest] = source
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	197
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	198 def update_copies_from_p2(self, copies):
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	199 for dest, source in copies.items():
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	200 self.mark_copied_from_p2(source, dest)
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	201
aea6a812f7cb commitctx: return a richer object from _prepare_files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44942 diff changeset	202
44940 4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	203 def computechangesetfilesadded(ctx):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	204 """return the list of files added in a changeset
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	205 """
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	206 added = []
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	207 for f in ctx.files():
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	208 if not any(f in p for p in ctx.parents()):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	209 added.append(f)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	210 return added
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	211
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	212
44941 edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	213 def get_removal_filter(ctx, x=None):
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	214 """return a function to detect files "wrongly" detected as `removed`
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	215
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	216 When a file is removed relative to p1 in a merge, this
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	217 function determines whether the absence is due to a
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	218 deletion from a parent, or whether the merge commit
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	219 itself deletes the file. We decide this by doing a
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	220 simplified three way merge of the manifest entry for
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	221 the file. There are two ways we decide the merge
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	222 itself didn't delete a file:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	223 - neither parent (nor the merge) contain the file
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	224 - exactly one parent contains the file, and that
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	225 parent has the same filelog entry as the merge
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	226 ancestor (or all of them if there two). In other
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	227 words, that parent left the file unchanged while the
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	228 other one deleted it.
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	229 One way to think about this is that deleting a file is
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	230 similar to emptying it, so the list of changed files
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	231 should be similar either way. The computation
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	232 described above is not done directly in _filecommit
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	233 when creating the list of changed files, however
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	234 it does something very similar by comparing filelog
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	235 nodes.
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	236 """
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	237
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	238 if x is not None:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	239 p1, p2, m1, m2 = x
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	240 else:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	241 p1 = ctx.p1()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	242 p2 = ctx.p2()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	243 m1 = p1.manifest()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	244 m2 = p2.manifest()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	245
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	246 @util.cachefunc
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	247 def mas():
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	248 p1n = p1.node()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	249 p2n = p2.node()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	250 cahs = ctx.repo().changelog.commonancestorsheads(p1n, p2n)
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	251 if not cahs:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	252 cahs = [node.nullrev]
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	253 return [ctx.repo()[r].manifest() for r in cahs]
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	254
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	255 def deletionfromparent(f):
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	256 if f in m1:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	257 return f not in m2 and all(
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	258 f in ma and ma.find(f) == m1.find(f) for ma in mas()
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	259 )
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	260 elif f in m2:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	261 return all(f in ma and ma.find(f) == m2.find(f) for ma in mas())
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	262 else:
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	263 return True
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	264
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	265 return deletionfromparent
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	266
edd08aa193fb files: extract code for extra filtering of the `removed` entry into copies Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44940 diff changeset	267
44940 4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	268 def computechangesetfilesremoved(ctx):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	269 """return the list of files removed in a changeset
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	270 """
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	271 removed = []
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	272 for f in ctx.files():
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	273 if f not in ctx:
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	274 removed.append(f)
44942 25512a65cefd metadata: filter the `removed` set to only contains relevant data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44941 diff changeset	275 if removed:
25512a65cefd metadata: filter the `removed` set to only contains relevant data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44941 diff changeset	276 rf = get_removal_filter(ctx)
25512a65cefd metadata: filter the `removed` set to only contains relevant data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44941 diff changeset	277 removed = [r for r in removed if not rf(r)]
44940 4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	278 return removed
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	279
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	280
45613 ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45611 diff changeset	281 def computechangesetfilesmerged(ctx):
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45611 diff changeset	282 """return the list of files merged in a changeset
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45611 diff changeset	283 """
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45611 diff changeset	284 merged = []
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45611 diff changeset	285 if len(ctx.parents()) < 2:
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45611 diff changeset	286 return merged
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45611 diff changeset	287 for f in ctx.files():
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45611 diff changeset	288 if f in ctx:
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45611 diff changeset	289 fctx = ctx[f]
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45611 diff changeset	290 parents = fctx._filelog.parents(fctx._filenode)
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45611 diff changeset	291 if parents[1] != node.nullid:
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45611 diff changeset	292 merged.append(f)
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45611 diff changeset	293 return merged
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45611 diff changeset	294
ddcee0b0fd67 changing-files: add a utility to compute the merged files post-commit Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45611 diff changeset	295
44940 4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	296 def computechangesetcopies(ctx):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	297 """return the copies data for a changeset
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	298
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	299 The copies data are returned as a pair of dictionnary (p1copies, p2copies).
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	300
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	301 Each dictionnary are in the form: `{newname: oldname}`
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	302 """
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	303 p1copies = {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	304 p2copies = {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	305 p1 = ctx.p1()
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	306 p2 = ctx.p2()
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	307 narrowmatch = ctx._repo.narrowmatch()
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	308 for dst in ctx.files():
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	309 if not narrowmatch(dst) or dst not in ctx:
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	310 continue
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	311 copied = ctx[dst].renamed()
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	312 if not copied:
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	313 continue
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	314 src, srcnode = copied
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	315 if src in p1 and p1[src].filenode() == srcnode:
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	316 p1copies[dst] = src
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	317 elif src in p2 and p2[src].filenode() == srcnode:
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	318 p2copies[dst] = src
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	319 return p1copies, p2copies
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	320
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	321
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	322 def encodecopies(files, copies):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	323 items = []
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	324 for i, dst in enumerate(files):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	325 if dst in copies:
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	326 items.append(b'%d\0%s' % (i, copies[dst]))
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	327 if len(items) != len(copies):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	328 raise error.ProgrammingError(
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	329 b'some copy targets missing from file list'
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	330 )
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	331 return b"\n".join(items)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	332
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	333
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	334 def decodecopies(files, data):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	335 try:
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	336 copies = {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	337 if not data:
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	338 return copies
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	339 for l in data.split(b'\n'):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	340 strindex, src = l.split(b'\0')
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	341 i = int(strindex)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	342 dst = files[i]
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	343 copies[dst] = src
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	344 return copies
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	345 except (ValueError, IndexError):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	346 # Perhaps someone had chosen the same key name (e.g. "p1copies") and
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	347 # used different syntax for the value.
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	348 return None
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	349
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	350
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	351 def encodefileindices(files, subset):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	352 subset = set(subset)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	353 indices = []
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	354 for i, f in enumerate(files):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	355 if f in subset:
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	356 indices.append(b'%d' % i)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	357 return b'\n'.join(indices)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	358
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	359
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	360 def decodefileindices(files, data):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	361 try:
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	362 subset = []
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	363 if not data:
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	364 return subset
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	365 for strindex in data.split(b'\n'):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	366 i = int(strindex)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	367 if i < 0 or i >= len(files):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	368 return None
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	369 subset.append(files[i])
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	370 return subset
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	371 except (ValueError, IndexError):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	372 # Perhaps someone had chosen the same key name (e.g. "added") and
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	373 # used different syntax for the value.
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	374 return None
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	375
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	376
45634 9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	377 # see mercurial/helptext/internals/revlogs.txt for details about the format
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	378
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	379 ACTION_MASK = int("111" "00", 2)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	380 # note: untouched file used as copy source will as `000` for this mask.
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	381 ADDED_FLAG = int("001" "00", 2)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	382 MERGED_FLAG = int("010" "00", 2)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	383 REMOVED_FLAG = int("011" "00", 2)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	384 # `100` is reserved for future use
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	385 TOUCHED_FLAG = int("101" "00", 2)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	386
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	387 COPIED_MASK = int("11", 2)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	388 COPIED_FROM_P1_FLAG = int("10", 2)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	389 COPIED_FROM_P2_FLAG = int("11", 2)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	390
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	391 # structure is <flag><filename-end><copy-source>
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	392 INDEX_HEADER = struct.Struct(">L")
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	393 INDEX_ENTRY = struct.Struct(">bLL")
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	394
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	395
45569 64d18e9e8508 sidedata: rename `encode_copies_sidedata` to `encode_files_sidedata` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45507 diff changeset	396 def encode_files_sidedata(files):
45634 9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	397 all_files = set(files.touched)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	398 all_files.update(files.copied_from_p1.values())
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	399 all_files.update(files.copied_from_p2.values())
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	400 all_files = sorted(all_files)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	401 file_idx = {f: i for (i, f) in enumerate(all_files)}
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	402 file_idx[None] = 0
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	403
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	404 chunks = [INDEX_HEADER.pack(len(all_files))]
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	405
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	406 filename_length = 0
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	407 for f in all_files:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	408 filename_size = len(f)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	409 filename_length += filename_size
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	410 flag = 0
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	411 if f in files.added:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	412 flag \|= ADDED_FLAG
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	413 elif f in files.merged:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	414 flag \|= MERGED_FLAG
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	415 elif f in files.removed:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	416 flag \|= REMOVED_FLAG
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	417 elif f in files.touched:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	418 flag \|= TOUCHED_FLAG
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	419
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	420 copy = None
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	421 if f in files.copied_from_p1:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	422 flag \|= COPIED_FROM_P1_FLAG
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	423 copy = files.copied_from_p1.get(f)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	424 elif f in files.copied_from_p2:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	425 copy = files.copied_from_p2.get(f)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	426 flag \|= COPIED_FROM_P2_FLAG
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	427 copy_idx = file_idx[copy]
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	428 chunks.append(INDEX_ENTRY.pack(flag, filename_length, copy_idx))
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	429 chunks.extend(all_files)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	430 return {sidedatamod.SD_FILES: b''.join(chunks)}
45325 c6eea5804551 commitctx: extract sidedata encoding inside its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45323 diff changeset	431
c6eea5804551 commitctx: extract sidedata encoding inside its own function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45323 diff changeset	432
45635 9003e6524f78 changing-files: drop the now useless changelogrevision argument Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45634 diff changeset	433 def decode_files_sidedata(sidedata):
45634 9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	434 md = ChangingFiles()
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	435 raw = sidedata.get(sidedatamod.SD_FILES)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	436
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	437 if raw is None:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	438 return md
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	439
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	440 copies = []
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	441 all_files = []
45571 7543b5072e84 sidedata: add a `decode_files_sidedata` function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45569 diff changeset	442
45634 9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	443 assert len(raw) >= INDEX_HEADER.size
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	444 total_files = INDEX_HEADER.unpack_from(raw, 0)[0]
45571 7543b5072e84 sidedata: add a `decode_files_sidedata` function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45569 diff changeset	445
45634 9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	446 offset = INDEX_HEADER.size
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	447 file_offset_base = offset + (INDEX_ENTRY.size * total_files)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	448 file_offset_last = file_offset_base
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	449
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	450 assert len(raw) >= file_offset_base
45571 7543b5072e84 sidedata: add a `decode_files_sidedata` function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45569 diff changeset	451
45634 9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	452 for idx in range(total_files):
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	453 flag, file_end, copy_idx = INDEX_ENTRY.unpack_from(raw, offset)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	454 file_end += file_offset_base
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	455 filename = raw[file_offset_last:file_end]
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	456 filesize = file_end - file_offset_last
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	457 assert len(filename) == filesize
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	458 offset += INDEX_ENTRY.size
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	459 file_offset_last = file_end
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	460 all_files.append(filename)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	461 if flag & ACTION_MASK == ADDED_FLAG:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	462 md.mark_added(filename)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	463 elif flag & ACTION_MASK == MERGED_FLAG:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	464 md.mark_merged(filename)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	465 elif flag & ACTION_MASK == REMOVED_FLAG:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	466 md.mark_removed(filename)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	467 elif flag & ACTION_MASK == TOUCHED_FLAG:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	468 md.mark_touched(filename)
45571 7543b5072e84 sidedata: add a `decode_files_sidedata` function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45569 diff changeset	469
45634 9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	470 copied = None
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	471 if flag & COPIED_MASK == COPIED_FROM_P1_FLAG:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	472 copied = md.mark_copied_from_p1
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	473 elif flag & COPIED_MASK == COPIED_FROM_P2_FLAG:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	474 copied = md.mark_copied_from_p2
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	475
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	476 if copied is not None:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	477 copies.append((copied, filename, copy_idx))
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	478
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	479 for copied, filename, copy_idx in copies:
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	480 copied(all_files[copy_idx], filename)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	481
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	482 return md
45571 7543b5072e84 sidedata: add a `decode_files_sidedata` function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45569 diff changeset	483
7543b5072e84 sidedata: add a `decode_files_sidedata` function Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45569 diff changeset	484
44940 4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	485 def _getsidedata(srcrepo, rev):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	486 ctx = srcrepo[rev]
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	487 filescopies = computechangesetcopies(ctx)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	488 filesadded = computechangesetfilesadded(ctx)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	489 filesremoved = computechangesetfilesremoved(ctx)
45634 9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	490 filesmerged = computechangesetfilesmerged(ctx)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	491 files = ChangingFiles()
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	492 files.update_touched(ctx.files())
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	493 files.update_added(filesadded)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	494 files.update_removed(filesremoved)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	495 files.update_merged(filesmerged)
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	496 files.update_copies_from_p1(filescopies[0])
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	497 files.update_copies_from_p2(filescopies[1])
9a6b409b8ebc changing-files: rework the way we store changed files in side-data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45623 diff changeset	498 return encode_files_sidedata(files)
44940 4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	499
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	500
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	501 def getsidedataadder(srcrepo, destrepo):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	502 use_w = srcrepo.ui.configbool(b'experimental', b'worker.repository-upgrade')
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	503 if pycompat.iswindows or not use_w:
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	504 return _get_simple_sidedata_adder(srcrepo, destrepo)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	505 else:
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	506 return _get_worker_sidedata_adder(srcrepo, destrepo)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	507
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	508
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	509 def _sidedata_worker(srcrepo, revs_queue, sidedata_queue, tokens):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	510 """The function used by worker precomputing sidedata
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	511
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	512 It read an input queue containing revision numbers
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	513 It write in an output queue containing (rev, <sidedata-map>)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	514
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	515 The `None` input value is used as a stop signal.
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	516
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	517 The `tokens` semaphore is user to avoid having too many unprocessed
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	518 entries. The workers needs to acquire one token before fetching a task.
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	519 They will be released by the consumer of the produced data.
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	520 """
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	521 tokens.acquire()
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	522 rev = revs_queue.get()
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	523 while rev is not None:
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	524 data = _getsidedata(srcrepo, rev)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	525 sidedata_queue.put((rev, data))
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	526 tokens.acquire()
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	527 rev = revs_queue.get()
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	528 # processing of `None` is completed, release the token.
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	529 tokens.release()
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	530
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	531
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	532 BUFF_PER_WORKER = 50
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	533
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	534
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	535 def _get_worker_sidedata_adder(srcrepo, destrepo):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	536 """The parallel version of the sidedata computation
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	537
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	538 This code spawn a pool of worker that precompute a buffer of sidedata
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	539 before we actually need them"""
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	540 # avoid circular import copies -> scmutil -> worker -> copies
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	541 from . import worker
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	542
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	543 nbworkers = worker._numworkers(srcrepo.ui)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	544
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	545 tokens = multiprocessing.BoundedSemaphore(nbworkers * BUFF_PER_WORKER)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	546 revsq = multiprocessing.Queue()
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	547 sidedataq = multiprocessing.Queue()
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	548
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	549 assert srcrepo.filtername is None
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	550 # queue all tasks beforehand, revision numbers are small and it make
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	551 # synchronisation simpler
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	552 #
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	553 # Since the computation for each node can be quite expensive, the overhead
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	554 # of using a single queue is not revelant. In practice, most computation
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	555 # are fast but some are very expensive and dominate all the other smaller
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	556 # cost.
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	557 for r in srcrepo.changelog.revs():
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	558 revsq.put(r)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	559 # queue the "no more tasks" markers
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	560 for i in range(nbworkers):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	561 revsq.put(None)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	562
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	563 allworkers = []
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	564 for i in range(nbworkers):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	565 args = (srcrepo, revsq, sidedataq, tokens)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	566 w = multiprocessing.Process(target=_sidedata_worker, args=args)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	567 allworkers.append(w)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	568 w.start()
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	569
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	570 # dictionnary to store results for revision higher than we one we are
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	571 # looking for. For example, if we need the sidedatamap for 42, and 43 is
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	572 # received, when shelve 43 for later use.
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	573 staging = {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	574
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	575 def sidedata_companion(revlog, rev):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	576 sidedata = {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	577 if util.safehasattr(revlog, b'filteredrevs'): # this is a changelog
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	578 # Is the data previously shelved ?
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	579 sidedata = staging.pop(rev, None)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	580 if sidedata is None:
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	581 # look at the queued result until we find the one we are lookig
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	582 # for (shelve the other ones)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	583 r, sidedata = sidedataq.get()
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	584 while r != rev:
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	585 staging[r] = sidedata
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	586 r, sidedata = sidedataq.get()
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	587 tokens.release()
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	588 return False, (), sidedata
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	589
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	590 return sidedata_companion
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	591
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	592
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	593 def _get_simple_sidedata_adder(srcrepo, destrepo):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	594 """The simple version of the sidedata computation
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	595
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	596 It just compute it in the same thread on request"""
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	597
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	598 def sidedatacompanion(revlog, rev):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	599 sidedata = {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	600 if util.safehasattr(revlog, 'filteredrevs'): # this is a changelog
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	601 sidedata = _getsidedata(srcrepo, rev)
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	602 return False, (), sidedata
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	603
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	604 return sidedatacompanion
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	605
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	606
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	607 def getsidedataremover(srcrepo, destrepo):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	608 def sidedatacompanion(revlog, rev):
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	609 f = ()
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	610 if util.safehasattr(revlog, 'filteredrevs'): # this is a changelog
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	611 if revlog.flags(rev) & sidedataflag.REVIDX_SIDEDATA:
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	612 f = (
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	613 sidedatamod.SD_P1COPIES,
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	614 sidedatamod.SD_P2COPIES,
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	615 sidedatamod.SD_FILESADDED,
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	616 sidedatamod.SD_FILESREMOVED,
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	617 )
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	618 return False, f, {}
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	619
4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: diff changeset	620 return sidedatacompanion

Mercurial > public > mercurial-scm > hg

annotate mercurial/metadata.py @ 45635:9003e6524f78