mercurial-scm/hg-stable: mercurial/upgrade

annotate mercurial/upgrade_utils/engine.py @ 46218:3f92a9bb80f0

engine: prevent multiple checking of re-delta-multibase The _perform_clone function is called for each revlog cloned, hence we should prevent this function call overhead. Differential Revision: https://phab.mercurial-scm.org/D9669

author	Pulkit Goyal <7895pulkit@gmail.com>
date	Wed, 30 Dec 2020 16:39:35 +0530
parents	02df91e895bd
children	481d9aed669c

rev	line source
31894 9379689b6c10 upgrade: update the header comment Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31893 diff changeset	1 # upgrade.py - functions for in place upgrade of Mercurial repository
4702 18e91c9def0c strip: move strip code to a new repair module Matt Mackall <mpm@selenic.com> parents: diff changeset	2 #
31895 783b4c9bd5f5 upgrade: update the copyright statement Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31894 diff changeset	3 # Copyright (c) 2016-present, Gregory Szorc
4702 18e91c9def0c strip: move strip code to a new repair module Matt Mackall <mpm@selenic.com> parents: diff changeset	4 #
8225 46293a0c7e9f updated license to be explicit about GPL version 2 Martin Geisler <mg@lazybytes.net> parents: 8073 diff changeset	5 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 9150 diff changeset	6 # GNU General Public License version 2 or any later version.
4702 18e91c9def0c strip: move strip code to a new repair module Matt Mackall <mpm@selenic.com> parents: diff changeset	7
25970 d1419cfbd4f4 repair: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25874 diff changeset	8 from __future__ import absolute_import
d1419cfbd4f4 repair: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25874 diff changeset	9
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	10 import stat
4702 18e91c9def0c strip: move strip code to a new repair module Matt Mackall <mpm@selenic.com> parents: diff changeset	11
46050 f105c49e89cd upgrade: split actual upgrade code away from the main module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46039 diff changeset	12 from ..i18n import _
f105c49e89cd upgrade: split actual upgrade code away from the main module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46039 diff changeset	13 from ..pycompat import getattr
f105c49e89cd upgrade: split actual upgrade code away from the main module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46039 diff changeset	14 from .. import (
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	15 changelog,
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26425 diff changeset	16 error,
35353 8f3f8b8dbab7 upgrade: use actual filelog to convert filelog Boris Feld <boris.feld@octobus.net> parents: 35352 diff changeset	17 filelog,
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	18 manifest,
44981 4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44900 diff changeset	19 metadata,
38198 2ce60954b1b7 py3: wrap tempfile.mkdtemp() to use bytes path Yuya Nishihara <yuya@tcha.org> parents: 37444 diff changeset	20 pycompat,
45392 77b8588dd84e requirements: introduce new requirements related module Pulkit Goyal <7895pulkit@gmail.com> parents: 45391 diff changeset	21 requirements,
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	22 revlog,
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	23 scmutil,
25970 d1419cfbd4f4 repair: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25874 diff changeset	24 util,
31242 254f98326eef vfs: use 'vfs' module directly in 'mercurial.repair' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30995 diff changeset	25 vfs as vfsmod,
25970 d1419cfbd4f4 repair: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25874 diff changeset	26 )
d1419cfbd4f4 repair: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25874 diff changeset	27
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	28
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	29 def _revlogfrompath(repo, path):
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	30 """Obtain a revlog from a repo path.
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	31
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	32 An instance of the appropriate class is returned.
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	33 """
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	34 if path == b'00changelog.i':
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	35 return changelog.changelog(repo.svfs)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	36 elif path.endswith(b'00manifest.i'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	37 mandir = path[: -len(b'00manifest.i')]
39271 0d97530eb535 manifest: rename dir argument and attribute to tree Gregory Szorc <gregory.szorc@gmail.com> parents: 38736 diff changeset	38 return manifest.manifestrevlog(repo.svfs, tree=mandir)
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	39 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	40 # reverse of "/".join(("data", path + ".i"))
35353 8f3f8b8dbab7 upgrade: use actual filelog to convert filelog Boris Feld <boris.feld@octobus.net> parents: 35352 diff changeset	41 return filelog.filelog(repo.svfs, path[5:-2])
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	42
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	43
42714 5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	44 def _copyrevlog(tr, destrepo, oldrl, unencodedname):
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	45 """copy all relevant files for `oldrl` into `destrepo` store
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	46
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	47 Files are copied "as is" without any transformation. The copy is performed
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	48 without extra checks. Callers are responsible for making sure the copied
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	49 content is compatible with format of the destination repository.
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	50 """
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	51 oldrl = getattr(oldrl, '_revlog', oldrl)
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	52 newrl = _revlogfrompath(destrepo, unencodedname)
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	53 newrl = getattr(newrl, '_revlog', newrl)
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	54
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	55 oldvfs = oldrl.opener
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	56 newvfs = newrl.opener
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	57 oldindex = oldvfs.join(oldrl.indexfile)
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	58 newindex = newvfs.join(newrl.indexfile)
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	59 olddata = oldvfs.join(oldrl.datafile)
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	60 newdata = newvfs.join(newrl.datafile)
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	61
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	62 with newvfs(newrl.indexfile, b'w'):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	63 pass # create all the directories
42714 5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	64
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	65 util.copyfile(oldindex, newindex)
43004 373749982924 upgrade: also register copied `.d` files to fncache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43002 diff changeset	66 copydata = oldrl.opener.exists(oldrl.datafile)
373749982924 upgrade: also register copied `.d` files to fncache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43002 diff changeset	67 if copydata:
42714 5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	68 util.copyfile(olddata, newdata)
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	69
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	70 if not (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	71 unencodedname.endswith(b'00changelog.i')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	72 or unencodedname.endswith(b'00manifest.i')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	73 ):
42714 5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	74 destrepo.svfs.fncache.add(unencodedname)
43004 373749982924 upgrade: also register copied `.d` files to fncache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43002 diff changeset	75 if copydata:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	76 destrepo.svfs.fncache.add(unencodedname[:-2] + b'.d')
42714 5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	77
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	78
46039 6c960b708ac4 upgrade: display the list of processed revlog before proceeding Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46018 diff changeset	79 UPGRADE_CHANGELOG = b"changelog"
6c960b708ac4 upgrade: display the list of processed revlog before proceeding Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46018 diff changeset	80 UPGRADE_MANIFEST = b"manifest"
6c960b708ac4 upgrade: display the list of processed revlog before proceeding Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46018 diff changeset	81 UPGRADE_FILELOGS = b"all-filelogs"
42715 0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	82
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	83 UPGRADE_ALL_REVLOGS = frozenset(
45995 fe7d7917ceb5 upgrade: rename UPGRADE_FILELOG to UPGRADE_FILELOGS Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45735 diff changeset	84 [UPGRADE_CHANGELOG, UPGRADE_MANIFEST, UPGRADE_FILELOGS]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	85 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	86
42715 0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	87
43135 6e3dc1eff5c7 upgrade: allow for `sidedata` removal Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43134 diff changeset	88 def getsidedatacompanion(srcrepo, dstrepo):
6e3dc1eff5c7 upgrade: allow for `sidedata` removal Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43134 diff changeset	89 sidedatacompanion = None
6e3dc1eff5c7 upgrade: allow for `sidedata` removal Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43134 diff changeset	90 removedreqs = srcrepo.requirements - dstrepo.requirements
43148 843da18386d5 sidedatacopies: deal with upgrading and downgrading to that format Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43138 diff changeset	91 addedreqs = dstrepo.requirements - srcrepo.requirements
45393 d7dcc75a3eae localrepo: move requirements constant to requirements module Pulkit Goyal <7895pulkit@gmail.com> parents: 45392 diff changeset	92 if requirements.SIDEDATA_REQUIREMENT in removedreqs:
43135 6e3dc1eff5c7 upgrade: allow for `sidedata` removal Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43134 diff changeset	93
6e3dc1eff5c7 upgrade: allow for `sidedata` removal Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43134 diff changeset	94 def sidedatacompanion(rl, rev):
6e3dc1eff5c7 upgrade: allow for `sidedata` removal Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43134 diff changeset	95 rl = getattr(rl, '_revlog', rl)
6e3dc1eff5c7 upgrade: allow for `sidedata` removal Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43134 diff changeset	96 if rl.flags(rev) & revlog.REVIDX_SIDEDATA:
45735 edf4fa06df94 upgrade: allow sidedata upgrade to modify revision flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45694 diff changeset	97 return True, (), {}, 0, 0
edf4fa06df94 upgrade: allow sidedata upgrade to modify revision flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45694 diff changeset	98 return False, (), {}, 0, 0
43135 6e3dc1eff5c7 upgrade: allow for `sidedata` removal Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43134 diff changeset	99
45393 d7dcc75a3eae localrepo: move requirements constant to requirements module Pulkit Goyal <7895pulkit@gmail.com> parents: 45392 diff changeset	100 elif requirements.COPIESSDC_REQUIREMENT in addedreqs:
44981 4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44900 diff changeset	101 sidedatacompanion = metadata.getsidedataadder(srcrepo, dstrepo)
45393 d7dcc75a3eae localrepo: move requirements constant to requirements module Pulkit Goyal <7895pulkit@gmail.com> parents: 45392 diff changeset	102 elif requirements.COPIESSDC_REQUIREMENT in removedreqs:
44981 4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44900 diff changeset	103 sidedatacompanion = metadata.getsidedataremover(srcrepo, dstrepo)
43135 6e3dc1eff5c7 upgrade: allow for `sidedata` removal Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43134 diff changeset	104 return sidedatacompanion
43134 75ad8af9c95e upgrade: allow upgrade to repository using sidedata Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43117 diff changeset	105
75ad8af9c95e upgrade: allow upgrade to repository using sidedata Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43117 diff changeset	106
42715 0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	107 def matchrevlog(revlogfilter, entry):
45694 d1c10d33a85c upgrade: improve documentation of matchrevlog() Pulkit Goyal <7895pulkit@gmail.com> parents: 45498 diff changeset	108 """check if a revlog is selected for cloning.
d1c10d33a85c upgrade: improve documentation of matchrevlog() Pulkit Goyal <7895pulkit@gmail.com> parents: 45498 diff changeset	109
d1c10d33a85c upgrade: improve documentation of matchrevlog() Pulkit Goyal <7895pulkit@gmail.com> parents: 45498 diff changeset	110 In other words, are there any updates which need to be done on revlog
d1c10d33a85c upgrade: improve documentation of matchrevlog() Pulkit Goyal <7895pulkit@gmail.com> parents: 45498 diff changeset	111 or it can be blindly copied.
42715 0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	112
0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	113 The store entry is checked against the passed filter"""
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	114 if entry.endswith(b'00changelog.i'):
42715 0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	115 return UPGRADE_CHANGELOG in revlogfilter
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	116 elif entry.endswith(b'00manifest.i'):
42715 0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	117 return UPGRADE_MANIFEST in revlogfilter
45995 fe7d7917ceb5 upgrade: rename UPGRADE_FILELOG to UPGRADE_FILELOGS Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45735 diff changeset	118 return UPGRADE_FILELOGS in revlogfilter
42715 0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	119
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	120
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	121 def _perform_clone(
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	122 ui,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	123 dstrepo,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	124 tr,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	125 old_revlog,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	126 unencoded,
46217 02df91e895bd engine: pass upgrade operation inside `_perform_clone()` Pulkit Goyal <7895pulkit@gmail.com> parents: 46216 diff changeset	127 upgrade_op,
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	128 sidedatacompanion,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	129 oncopiedrevision,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	130 ):
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	131 """ returns the new revlog object created"""
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	132 newrl = None
46217 02df91e895bd engine: pass upgrade operation inside `_perform_clone()` Pulkit Goyal <7895pulkit@gmail.com> parents: 46216 diff changeset	133 if matchrevlog(upgrade_op.revlogs_to_process, unencoded):
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	134 ui.note(
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	135 _(b'cloning %d revisions from %s\n') % (len(old_revlog), unencoded)
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	136 )
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	137 newrl = _revlogfrompath(dstrepo, unencoded)
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	138 old_revlog.clone(
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	139 tr,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	140 newrl,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	141 addrevisioncb=oncopiedrevision,
46217 02df91e895bd engine: pass upgrade operation inside `_perform_clone()` Pulkit Goyal <7895pulkit@gmail.com> parents: 46216 diff changeset	142 deltareuse=upgrade_op.delta_reuse_mode,
46218 3f92a9bb80f0 engine: prevent multiple checking of re-delta-multibase Pulkit Goyal <7895pulkit@gmail.com> parents: 46217 diff changeset	143 forcedeltabothparents=upgrade_op.force_re_delta_both_parents,
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	144 sidedatacompanion=sidedatacompanion,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	145 )
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	146 else:
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	147 msg = _(b'blindly copying %s containing %i revisions\n')
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	148 ui.note(msg % (unencoded, len(old_revlog)))
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	149 _copyrevlog(tr, dstrepo, old_revlog, unencoded)
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	150
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	151 newrl = _revlogfrompath(dstrepo, unencoded)
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	152 return newrl
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	153
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	154
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	155 def _clonerevlogs(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	156 ui,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	157 srcrepo,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	158 dstrepo,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	159 tr,
46216 34efa84a43a1 engine: pass upgrade operation inside _clonerevlogs() Pulkit Goyal <7895pulkit@gmail.com> parents: 46215 diff changeset	160 upgrade_op,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	161 ):
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	162 """Copy revlogs between 2 repos."""
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	163 revcount = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	164 srcsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	165 srcrawsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	166 dstsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	167 fcount = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	168 frevcount = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	169 fsrcsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	170 frawsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	171 fdstsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	172 mcount = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	173 mrevcount = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	174 msrcsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	175 mrawsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	176 mdstsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	177 crevcount = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	178 csrcsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	179 crawsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	180 cdstsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	181
42712 896fb9deeaf8 upgrade: walk the source store file only once Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42137 diff changeset	182 alldatafiles = list(srcrepo.store.walk())
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	183 # mapping of data files which needs to be cloned
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	184 # key is unencoded filename
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	185 # value is revlog_object_from_srcrepo
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	186 manifests = {}
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	187 changelogs = {}
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	188 filelogs = {}
42712 896fb9deeaf8 upgrade: walk the source store file only once Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42137 diff changeset	189
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	190 # Perform a pass to collect metadata. This validates we can open all
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	191 # source files and allows a unified progress bar to be displayed.
42712 896fb9deeaf8 upgrade: walk the source store file only once Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42137 diff changeset	192 for unencoded, encoded, size in alldatafiles:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	193 if unencoded.endswith(b'.d'):
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	194 continue
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	195
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	196 rl = _revlogfrompath(srcrepo, unencoded)
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	197
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	198 info = rl.storageinfo(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	199 exclusivefiles=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	200 revisionscount=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	201 trackedsize=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	202 storedsize=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	203 )
39862 8dab7c8a93eb upgrade: report size of backing files, not internal storage size Gregory Szorc <gregory.szorc@gmail.com> parents: 39563 diff changeset	204
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	205 revcount += info[b'revisionscount'] or 0
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	206 datasize = info[b'storedsize'] or 0
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	207 rawsize = info[b'trackedsize'] or 0
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	208
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	209 srcsize += datasize
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	210 srcrawsize += rawsize
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	211
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	212 # This is for the separate progress bars.
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	213 if isinstance(rl, changelog.changelog):
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	214 changelogs[unencoded] = rl
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	215 crevcount += len(rl)
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	216 csrcsize += datasize
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	217 crawsize += rawsize
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	218 elif isinstance(rl, manifest.manifestrevlog):
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	219 manifests[unencoded] = rl
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	220 mcount += 1
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	221 mrevcount += len(rl)
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	222 msrcsize += datasize
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	223 mrawsize += rawsize
37444 c8666a9e9e11 upgrade: sniff for filelog type Gregory Szorc <gregory.szorc@gmail.com> parents: 36404 diff changeset	224 elif isinstance(rl, filelog.filelog):
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	225 filelogs[unencoded] = rl
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	226 fcount += 1
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	227 frevcount += len(rl)
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	228 fsrcsize += datasize
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	229 frawsize += rawsize
37444 c8666a9e9e11 upgrade: sniff for filelog type Gregory Szorc <gregory.szorc@gmail.com> parents: 36404 diff changeset	230 else:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	231 error.ProgrammingError(b'unknown revlog type')
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	232
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	233 if not revcount:
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	234 return
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	235
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	236 ui.status(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	237 _(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	238 b'migrating %d total revisions (%d in filelogs, %d in manifests, '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	239 b'%d in changelog)\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	240 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	241 % (revcount, frevcount, mrevcount, crevcount)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	242 )
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	243 ui.status(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	244 _(b'migrating %s in store; %s tracked data\n')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	245 % ((util.bytecount(srcsize), util.bytecount(srcrawsize)))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	246 )
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	247
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	248 # Used to keep track of progress.
38405 185588cb0c4b upgrade: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38404 diff changeset	249 progress = None
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	250
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	251 def oncopiedrevision(rl, rev, node):
38405 185588cb0c4b upgrade: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38404 diff changeset	252 progress.increment()
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	253
43134 75ad8af9c95e upgrade: allow upgrade to repository using sidedata Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43117 diff changeset	254 sidedatacompanion = getsidedatacompanion(srcrepo, dstrepo)
75ad8af9c95e upgrade: allow upgrade to repository using sidedata Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43117 diff changeset	255
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	256 # Migrating filelogs
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	257 ui.status(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	258 _(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	259 b'migrating %d filelogs containing %d revisions '
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	260 b'(%s in store; %s tracked data)\n'
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	261 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	262 % (
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	263 fcount,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	264 frevcount,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	265 util.bytecount(fsrcsize),
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	266 util.bytecount(frawsize),
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	267 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	268 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	269 progress = srcrepo.ui.makeprogress(_(b'file revisions'), total=frevcount)
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	270 for unencoded, oldrl in sorted(filelogs.items()):
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	271 newrl = _perform_clone(
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	272 ui,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	273 dstrepo,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	274 tr,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	275 oldrl,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	276 unencoded,
46217 02df91e895bd engine: pass upgrade operation inside `_perform_clone()` Pulkit Goyal <7895pulkit@gmail.com> parents: 46216 diff changeset	277 upgrade_op,
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	278 sidedatacompanion,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	279 oncopiedrevision,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	280 )
39875 b399ff55ee6d upgrade: use storageinfo() for obtaining storage metadata Gregory Szorc <gregory.szorc@gmail.com> parents: 39864 diff changeset	281 info = newrl.storageinfo(storedsize=True)
46214 5dfa837d933e engine: refactor how total dstsize is calculated Pulkit Goyal <7895pulkit@gmail.com> parents: 46210 diff changeset	282 fdstsize += info[b'storedsize'] or 0
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	283 ui.status(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	284 _(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	285 b'finished migrating %d filelog revisions across %d '
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	286 b'filelogs; change in size: %s\n'
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	287 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	288 % (frevcount, fcount, util.bytecount(fdstsize - fsrcsize))
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	289 )
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	290
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	291 # Migrating manifests
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	292 ui.status(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	293 _(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	294 b'migrating %d manifests containing %d revisions '
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	295 b'(%s in store; %s tracked data)\n'
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	296 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	297 % (
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	298 mcount,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	299 mrevcount,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	300 util.bytecount(msrcsize),
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	301 util.bytecount(mrawsize),
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	302 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	303 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	304 if progress:
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	305 progress.complete()
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	306 progress = srcrepo.ui.makeprogress(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	307 _(b'manifest revisions'), total=mrevcount
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	308 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	309 for unencoded, oldrl in sorted(manifests.items()):
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	310 newrl = _perform_clone(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	311 ui,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	312 dstrepo,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	313 tr,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	314 oldrl,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	315 unencoded,
46217 02df91e895bd engine: pass upgrade operation inside `_perform_clone()` Pulkit Goyal <7895pulkit@gmail.com> parents: 46216 diff changeset	316 upgrade_op,
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	317 sidedatacompanion,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	318 oncopiedrevision,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	319 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	320 info = newrl.storageinfo(storedsize=True)
46214 5dfa837d933e engine: refactor how total dstsize is calculated Pulkit Goyal <7895pulkit@gmail.com> parents: 46210 diff changeset	321 mdstsize += info[b'storedsize'] or 0
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	322 ui.status(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	323 _(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	324 b'finished migrating %d manifest revisions across %d '
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	325 b'manifests; change in size: %s\n'
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	326 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	327 % (mrevcount, mcount, util.bytecount(mdstsize - msrcsize))
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	328 )
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	329
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	330 # Migrating changelog
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	331 ui.status(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	332 _(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	333 b'migrating changelog containing %d revisions '
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	334 b'(%s in store; %s tracked data)\n'
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	335 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	336 % (
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	337 crevcount,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	338 util.bytecount(csrcsize),
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	339 util.bytecount(crawsize),
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	340 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	341 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	342 if progress:
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	343 progress.complete()
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	344 progress = srcrepo.ui.makeprogress(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	345 _(b'changelog revisions'), total=crevcount
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	346 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	347 for unencoded, oldrl in sorted(changelogs.items()):
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	348 newrl = _perform_clone(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	349 ui,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	350 dstrepo,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	351 tr,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	352 oldrl,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	353 unencoded,
46217 02df91e895bd engine: pass upgrade operation inside `_perform_clone()` Pulkit Goyal <7895pulkit@gmail.com> parents: 46216 diff changeset	354 upgrade_op,
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	355 sidedatacompanion,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	356 oncopiedrevision,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	357 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	358 info = newrl.storageinfo(storedsize=True)
46214 5dfa837d933e engine: refactor how total dstsize is calculated Pulkit Goyal <7895pulkit@gmail.com> parents: 46210 diff changeset	359 cdstsize += info[b'storedsize'] or 0
38405 185588cb0c4b upgrade: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38404 diff changeset	360 progress.complete()
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	361 ui.status(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	362 _(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	363 b'finished migrating %d changelog revisions; change in size: '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	364 b'%s\n'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	365 )
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	366 % (crevcount, util.bytecount(cdstsize - csrcsize))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	367 )
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	368
46214 5dfa837d933e engine: refactor how total dstsize is calculated Pulkit Goyal <7895pulkit@gmail.com> parents: 46210 diff changeset	369 dstsize = fdstsize + mdstsize + cdstsize
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	370 ui.status(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	371 _(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	372 b'finished migrating %d total revisions; total change in store '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	373 b'size: %s\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	374 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	375 % (revcount, util.bytecount(dstsize - srcsize))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	376 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	377
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	378
31873 d6c2005484b6 upgrade: drop the prefix to the '_filterstorefile' function Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31872 diff changeset	379 def _filterstorefile(srcrepo, dstrepo, requirements, path, mode, st):
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	380 """Determine whether to copy a store file during upgrade.
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	381
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	382 This function is called when migrating store files from ``srcrepo`` to
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	383 ``dstrepo`` as part of upgrading a repository.
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	384
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	385 Args:
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	386 srcrepo: repo we are copying from
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	387 dstrepo: repo we are copying to
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	388 requirements: set of requirements for ``dstrepo``
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	389 path: store file being examined
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	390 mode: the ``ST_MODE`` file type of ``path``
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	391 st: ``stat`` data structure for ``path``
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	392
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	393 Function should return ``True`` if the file is to be copied.
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	394 """
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	395 # Skip revlogs.
44900 526d69eeea31 upgrade: support upgrade and downgrade from persistent nodemap Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44876 diff changeset	396 if path.endswith((b'.i', b'.d', b'.n', b'.nd')):
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	397 return False
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	398 # Skip transaction related files.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	399 if path.startswith(b'undo'):
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	400 return False
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	401 # Only copy regular files.
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	402 if mode != stat.S_IFREG:
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	403 return False
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	404 # Skip other skipped files.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	405 if path in (b'lock', b'fncache'):
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	406 return False
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	407
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	408 return True
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	409
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	410
31874 27ec6517e40e upgrade: drop the prefix to the '_finishdatamigration' function Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31873 diff changeset	411 def _finishdatamigration(ui, srcrepo, dstrepo, requirements):
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	412 """Hook point for extensions to perform additional actions during upgrade.
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	413
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	414 This function is called after revlogs and store files have been copied but
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	415 before the new store is swapped into the original location.
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	416 """
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	417
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	418
46060 c407513a44a3 upgrade: start moving the "to be happening" data in a dedicated object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46050 diff changeset	419 def upgrade(ui, srcrepo, dstrepo, upgrade_op):
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	420 """Do the low-level work of upgrading a repository.
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	421
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	422 The upgrade is effectively performed as a copy between a source
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	423 repository and a temporary destination repository.
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	424
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	425 The source repository is unmodified for as long as possible so the
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	426 upgrade can abort at any time without causing loss of service for
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	427 readers and without corrupting the source repository.
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	428 """
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	429 assert srcrepo.currentwlock()
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	430 assert dstrepo.currentwlock()
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	431
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	432 ui.status(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	433 _(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	434 b'(it is safe to interrupt this process any time before '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	435 b'data migration completes)\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	436 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	437 )
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	438
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	439 with dstrepo.transaction(b'upgrade') as tr:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	440 _clonerevlogs(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	441 ui,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	442 srcrepo,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	443 dstrepo,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	444 tr,
46216 34efa84a43a1 engine: pass upgrade operation inside _clonerevlogs() Pulkit Goyal <7895pulkit@gmail.com> parents: 46215 diff changeset	445 upgrade_op,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	446 )
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	447
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	448 # Now copy other files in the store directory.
30895 0b913e1e725b repair: iterate store files deterministically Gregory Szorc <gregory.szorc@gmail.com> parents: 30781 diff changeset	449 # The sorted() makes execution deterministic.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	450 for p, kind, st in sorted(srcrepo.store.vfs.readdir(b'', stat=True)):
46060 c407513a44a3 upgrade: start moving the "to be happening" data in a dedicated object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46050 diff changeset	451 if not _filterstorefile(
46191 aba979b1b90b upgrade: move `printrequirements()` to UpgradeOperation class Pulkit Goyal <7895pulkit@gmail.com> parents: 46189 diff changeset	452 srcrepo, dstrepo, upgrade_op.new_requirements, p, kind, st
46060 c407513a44a3 upgrade: start moving the "to be happening" data in a dedicated object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46050 diff changeset	453 ):
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	454 continue
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	455
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	456 srcrepo.ui.status(_(b'copying %s\n') % p)
30897 8110d49e0558 repair: use rawvfs when copying extra store files Gregory Szorc <gregory.szorc@gmail.com> parents: 30895 diff changeset	457 src = srcrepo.store.rawvfs.join(p)
8110d49e0558 repair: use rawvfs when copying extra store files Gregory Szorc <gregory.szorc@gmail.com> parents: 30895 diff changeset	458 dst = dstrepo.store.rawvfs.join(p)
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	459 util.copyfile(src, dst, copystat=True)
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	460
31874 27ec6517e40e upgrade: drop the prefix to the '_finishdatamigration' function Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31873 diff changeset	461 _finishdatamigration(ui, srcrepo, dstrepo, requirements)
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	462
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	463 ui.status(_(b'data fully migrated to temporary repository\n'))
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	464
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	465 backuppath = pycompat.mkdtemp(prefix=b'upgradebackup.', dir=srcrepo.path)
31242 254f98326eef vfs: use 'vfs' module directly in 'mercurial.repair' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30995 diff changeset	466 backupvfs = vfsmod.vfs(backuppath)
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	467
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	468 # Make a backup of requires file first, as it is the first to be modified.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	469 util.copyfile(srcrepo.vfs.join(b'requires'), backupvfs.join(b'requires'))
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	470
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	471 # We install an arbitrary requirement that clients must not support
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	472 # as a mechanism to lock out new clients during the data swap. This is
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	473 # better than allowing a client to continue while the repository is in
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	474 # an inconsistent state.
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	475 ui.status(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	476 _(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	477 b'marking source repository as being upgraded; clients will be '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	478 b'unable to read from repository\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	479 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	480 )
45106 a03c177a4679 scmutil: add writereporequirements() and route requires writing through it Pulkit Goyal <7895pulkit@gmail.com> parents: 44981 diff changeset	481 scmutil.writereporequirements(
a03c177a4679 scmutil: add writereporequirements() and route requires writing through it Pulkit Goyal <7895pulkit@gmail.com> parents: 44981 diff changeset	482 srcrepo, srcrepo.requirements \| {b'upgradeinprogress'}
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	483 )
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	484
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	485 ui.status(_(b'starting in-place swap of repository data\n'))
e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	486 ui.status(_(b'replaced files will be backed up at %s\n') % backuppath)
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	487
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	488 # Now swap in the new store directory. Doing it as a rename should make
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	489 # the operation nearly instantaneous and atomic (at least in well-behaved
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	490 # environments).
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	491 ui.status(_(b'replacing store...\n'))
30995 22fbca1d11ed mercurial: switch to util.timer for all interval timings Simon Farnsworth <simonfar@fb.com> parents: 30781 diff changeset	492 tstart = util.timer()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	493 util.rename(srcrepo.spath, backupvfs.join(b'store'))
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	494 util.rename(dstrepo.spath, srcrepo.spath)
30995 22fbca1d11ed mercurial: switch to util.timer for all interval timings Simon Farnsworth <simonfar@fb.com> parents: 30781 diff changeset	495 elapsed = util.timer() - tstart
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	496 ui.status(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	497 _(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	498 b'store replacement complete; repository was inconsistent for '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	499 b'%0.1fs\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	500 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	501 % elapsed
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	502 )
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	503
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	504 # We first write the requirements file. Any new requirements will lock
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	505 # out legacy clients.
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	506 ui.status(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	507 _(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	508 b'finalizing requirements file and making repository readable '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	509 b'again\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	510 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	511 )
46191 aba979b1b90b upgrade: move `printrequirements()` to UpgradeOperation class Pulkit Goyal <7895pulkit@gmail.com> parents: 46189 diff changeset	512 scmutil.writereporequirements(srcrepo, upgrade_op.new_requirements)
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	513
30781 f2c069bf78ee repair: clean up stale lock file from store backup Gregory Szorc <gregory.szorc@gmail.com> parents: 30780 diff changeset	514 # The lock file from the old store won't be removed because nothing has a
f2c069bf78ee repair: clean up stale lock file from store backup Gregory Szorc <gregory.szorc@gmail.com> parents: 30780 diff changeset	515 # reference to its new location. So clean it up manually. Alternatively, we
f2c069bf78ee repair: clean up stale lock file from store backup Gregory Szorc <gregory.szorc@gmail.com> parents: 30780 diff changeset	516 # could update srcrepo.svfs and other variables to point to the new
f2c069bf78ee repair: clean up stale lock file from store backup Gregory Szorc <gregory.szorc@gmail.com> parents: 30780 diff changeset	517 # location. This is simpler.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	518 backupvfs.unlink(b'store/lock')
30781 f2c069bf78ee repair: clean up stale lock file from store backup Gregory Szorc <gregory.szorc@gmail.com> parents: 30780 diff changeset	519
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	520 return backuppath

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/upgrade_utils/engine.py @ 46218:3f92a9bb80f0