mercurial-scm/hg-stable: mercurial/upgrade

annotate mercurial/upgrade_utils/engine.py @ 46220:1ca7865c245d

engine: refactor code to replace stores in separate function In not all upgrades, we need to change the whole store. For example, when upgrading repository to share-safe mode, we don't touch revlogs at all hence store cloning and copying is not required. The store replacing code needs to be made aware about what all has changed and hence only copy/rename those things. To kickstart that, this patch moves existing logic into a separate function. Differential Revision: https://phab.mercurial-scm.org/D9674

author	Pulkit Goyal <7895pulkit@gmail.com>
date	Thu, 31 Dec 2020 14:45:16 +0530
parents	481d9aed669c
children	e22aed089567

rev	line source
31894 9379689b6c10 upgrade: update the header comment Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31893 diff changeset	1 # upgrade.py - functions for in place upgrade of Mercurial repository
4702 18e91c9def0c strip: move strip code to a new repair module Matt Mackall <mpm@selenic.com> parents: diff changeset	2 #
31895 783b4c9bd5f5 upgrade: update the copyright statement Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31894 diff changeset	3 # Copyright (c) 2016-present, Gregory Szorc
4702 18e91c9def0c strip: move strip code to a new repair module Matt Mackall <mpm@selenic.com> parents: diff changeset	4 #
8225 46293a0c7e9f updated license to be explicit about GPL version 2 Martin Geisler <mg@lazybytes.net> parents: 8073 diff changeset	5 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 9150 diff changeset	6 # GNU General Public License version 2 or any later version.
4702 18e91c9def0c strip: move strip code to a new repair module Matt Mackall <mpm@selenic.com> parents: diff changeset	7
25970 d1419cfbd4f4 repair: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25874 diff changeset	8 from __future__ import absolute_import
d1419cfbd4f4 repair: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25874 diff changeset	9
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	10 import stat
4702 18e91c9def0c strip: move strip code to a new repair module Matt Mackall <mpm@selenic.com> parents: diff changeset	11
46050 f105c49e89cd upgrade: split actual upgrade code away from the main module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46039 diff changeset	12 from ..i18n import _
f105c49e89cd upgrade: split actual upgrade code away from the main module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46039 diff changeset	13 from ..pycompat import getattr
f105c49e89cd upgrade: split actual upgrade code away from the main module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46039 diff changeset	14 from .. import (
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	15 changelog,
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26425 diff changeset	16 error,
35353 8f3f8b8dbab7 upgrade: use actual filelog to convert filelog Boris Feld <boris.feld@octobus.net> parents: 35352 diff changeset	17 filelog,
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	18 manifest,
44981 4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44900 diff changeset	19 metadata,
38198 2ce60954b1b7 py3: wrap tempfile.mkdtemp() to use bytes path Yuya Nishihara <yuya@tcha.org> parents: 37444 diff changeset	20 pycompat,
45392 77b8588dd84e requirements: introduce new requirements related module Pulkit Goyal <7895pulkit@gmail.com> parents: 45391 diff changeset	21 requirements,
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	22 revlog,
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	23 scmutil,
25970 d1419cfbd4f4 repair: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25874 diff changeset	24 util,
31242 254f98326eef vfs: use 'vfs' module directly in 'mercurial.repair' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30995 diff changeset	25 vfs as vfsmod,
25970 d1419cfbd4f4 repair: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25874 diff changeset	26 )
d1419cfbd4f4 repair: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25874 diff changeset	27
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	28
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	29 def _revlogfrompath(repo, path):
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	30 """Obtain a revlog from a repo path.
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	31
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	32 An instance of the appropriate class is returned.
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	33 """
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	34 if path == b'00changelog.i':
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	35 return changelog.changelog(repo.svfs)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	36 elif path.endswith(b'00manifest.i'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	37 mandir = path[: -len(b'00manifest.i')]
39271 0d97530eb535 manifest: rename dir argument and attribute to tree Gregory Szorc <gregory.szorc@gmail.com> parents: 38736 diff changeset	38 return manifest.manifestrevlog(repo.svfs, tree=mandir)
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	39 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	40 # reverse of "/".join(("data", path + ".i"))
35353 8f3f8b8dbab7 upgrade: use actual filelog to convert filelog Boris Feld <boris.feld@octobus.net> parents: 35352 diff changeset	41 return filelog.filelog(repo.svfs, path[5:-2])
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	42
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	43
42714 5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	44 def _copyrevlog(tr, destrepo, oldrl, unencodedname):
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	45 """copy all relevant files for `oldrl` into `destrepo` store
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	46
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	47 Files are copied "as is" without any transformation. The copy is performed
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	48 without extra checks. Callers are responsible for making sure the copied
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	49 content is compatible with format of the destination repository.
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	50 """
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	51 oldrl = getattr(oldrl, '_revlog', oldrl)
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	52 newrl = _revlogfrompath(destrepo, unencodedname)
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	53 newrl = getattr(newrl, '_revlog', newrl)
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	54
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	55 oldvfs = oldrl.opener
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	56 newvfs = newrl.opener
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	57 oldindex = oldvfs.join(oldrl.indexfile)
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	58 newindex = newvfs.join(newrl.indexfile)
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	59 olddata = oldvfs.join(oldrl.datafile)
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	60 newdata = newvfs.join(newrl.datafile)
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	61
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	62 with newvfs(newrl.indexfile, b'w'):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	63 pass # create all the directories
42714 5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	64
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	65 util.copyfile(oldindex, newindex)
43004 373749982924 upgrade: also register copied `.d` files to fncache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43002 diff changeset	66 copydata = oldrl.opener.exists(oldrl.datafile)
373749982924 upgrade: also register copied `.d` files to fncache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43002 diff changeset	67 if copydata:
42714 5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	68 util.copyfile(olddata, newdata)
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	69
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	70 if not (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	71 unencodedname.endswith(b'00changelog.i')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	72 or unencodedname.endswith(b'00manifest.i')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	73 ):
42714 5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	74 destrepo.svfs.fncache.add(unencodedname)
43004 373749982924 upgrade: also register copied `.d` files to fncache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43002 diff changeset	75 if copydata:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	76 destrepo.svfs.fncache.add(unencodedname[:-2] + b'.d')
42714 5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	77
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	78
46039 6c960b708ac4 upgrade: display the list of processed revlog before proceeding Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46018 diff changeset	79 UPGRADE_CHANGELOG = b"changelog"
6c960b708ac4 upgrade: display the list of processed revlog before proceeding Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46018 diff changeset	80 UPGRADE_MANIFEST = b"manifest"
6c960b708ac4 upgrade: display the list of processed revlog before proceeding Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46018 diff changeset	81 UPGRADE_FILELOGS = b"all-filelogs"
42715 0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	82
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	83 UPGRADE_ALL_REVLOGS = frozenset(
45995 fe7d7917ceb5 upgrade: rename UPGRADE_FILELOG to UPGRADE_FILELOGS Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45735 diff changeset	84 [UPGRADE_CHANGELOG, UPGRADE_MANIFEST, UPGRADE_FILELOGS]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	85 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	86
42715 0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	87
43135 6e3dc1eff5c7 upgrade: allow for `sidedata` removal Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43134 diff changeset	88 def getsidedatacompanion(srcrepo, dstrepo):
6e3dc1eff5c7 upgrade: allow for `sidedata` removal Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43134 diff changeset	89 sidedatacompanion = None
6e3dc1eff5c7 upgrade: allow for `sidedata` removal Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43134 diff changeset	90 removedreqs = srcrepo.requirements - dstrepo.requirements
43148 843da18386d5 sidedatacopies: deal with upgrading and downgrading to that format Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43138 diff changeset	91 addedreqs = dstrepo.requirements - srcrepo.requirements
45393 d7dcc75a3eae localrepo: move requirements constant to requirements module Pulkit Goyal <7895pulkit@gmail.com> parents: 45392 diff changeset	92 if requirements.SIDEDATA_REQUIREMENT in removedreqs:
43135 6e3dc1eff5c7 upgrade: allow for `sidedata` removal Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43134 diff changeset	93
6e3dc1eff5c7 upgrade: allow for `sidedata` removal Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43134 diff changeset	94 def sidedatacompanion(rl, rev):
6e3dc1eff5c7 upgrade: allow for `sidedata` removal Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43134 diff changeset	95 rl = getattr(rl, '_revlog', rl)
6e3dc1eff5c7 upgrade: allow for `sidedata` removal Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43134 diff changeset	96 if rl.flags(rev) & revlog.REVIDX_SIDEDATA:
45735 edf4fa06df94 upgrade: allow sidedata upgrade to modify revision flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45694 diff changeset	97 return True, (), {}, 0, 0
edf4fa06df94 upgrade: allow sidedata upgrade to modify revision flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45694 diff changeset	98 return False, (), {}, 0, 0
43135 6e3dc1eff5c7 upgrade: allow for `sidedata` removal Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43134 diff changeset	99
45393 d7dcc75a3eae localrepo: move requirements constant to requirements module Pulkit Goyal <7895pulkit@gmail.com> parents: 45392 diff changeset	100 elif requirements.COPIESSDC_REQUIREMENT in addedreqs:
44981 4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44900 diff changeset	101 sidedatacompanion = metadata.getsidedataadder(srcrepo, dstrepo)
45393 d7dcc75a3eae localrepo: move requirements constant to requirements module Pulkit Goyal <7895pulkit@gmail.com> parents: 45392 diff changeset	102 elif requirements.COPIESSDC_REQUIREMENT in removedreqs:
44981 4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44900 diff changeset	103 sidedatacompanion = metadata.getsidedataremover(srcrepo, dstrepo)
43135 6e3dc1eff5c7 upgrade: allow for `sidedata` removal Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43134 diff changeset	104 return sidedatacompanion
43134 75ad8af9c95e upgrade: allow upgrade to repository using sidedata Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43117 diff changeset	105
75ad8af9c95e upgrade: allow upgrade to repository using sidedata Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43117 diff changeset	106
42715 0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	107 def matchrevlog(revlogfilter, entry):
45694 d1c10d33a85c upgrade: improve documentation of matchrevlog() Pulkit Goyal <7895pulkit@gmail.com> parents: 45498 diff changeset	108 """check if a revlog is selected for cloning.
d1c10d33a85c upgrade: improve documentation of matchrevlog() Pulkit Goyal <7895pulkit@gmail.com> parents: 45498 diff changeset	109
d1c10d33a85c upgrade: improve documentation of matchrevlog() Pulkit Goyal <7895pulkit@gmail.com> parents: 45498 diff changeset	110 In other words, are there any updates which need to be done on revlog
d1c10d33a85c upgrade: improve documentation of matchrevlog() Pulkit Goyal <7895pulkit@gmail.com> parents: 45498 diff changeset	111 or it can be blindly copied.
42715 0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	112
0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	113 The store entry is checked against the passed filter"""
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	114 if entry.endswith(b'00changelog.i'):
42715 0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	115 return UPGRADE_CHANGELOG in revlogfilter
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	116 elif entry.endswith(b'00manifest.i'):
42715 0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	117 return UPGRADE_MANIFEST in revlogfilter
45995 fe7d7917ceb5 upgrade: rename UPGRADE_FILELOG to UPGRADE_FILELOGS Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45735 diff changeset	118 return UPGRADE_FILELOGS in revlogfilter
42715 0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	119
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	120
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	121 def _perform_clone(
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	122 ui,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	123 dstrepo,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	124 tr,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	125 old_revlog,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	126 unencoded,
46217 02df91e895bd engine: pass upgrade operation inside `_perform_clone()` Pulkit Goyal <7895pulkit@gmail.com> parents: 46216 diff changeset	127 upgrade_op,
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	128 sidedatacompanion,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	129 oncopiedrevision,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	130 ):
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	131 """ returns the new revlog object created"""
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	132 newrl = None
46217 02df91e895bd engine: pass upgrade operation inside `_perform_clone()` Pulkit Goyal <7895pulkit@gmail.com> parents: 46216 diff changeset	133 if matchrevlog(upgrade_op.revlogs_to_process, unencoded):
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	134 ui.note(
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	135 _(b'cloning %d revisions from %s\n') % (len(old_revlog), unencoded)
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	136 )
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	137 newrl = _revlogfrompath(dstrepo, unencoded)
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	138 old_revlog.clone(
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	139 tr,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	140 newrl,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	141 addrevisioncb=oncopiedrevision,
46217 02df91e895bd engine: pass upgrade operation inside `_perform_clone()` Pulkit Goyal <7895pulkit@gmail.com> parents: 46216 diff changeset	142 deltareuse=upgrade_op.delta_reuse_mode,
46218 3f92a9bb80f0 engine: prevent multiple checking of re-delta-multibase Pulkit Goyal <7895pulkit@gmail.com> parents: 46217 diff changeset	143 forcedeltabothparents=upgrade_op.force_re_delta_both_parents,
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	144 sidedatacompanion=sidedatacompanion,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	145 )
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	146 else:
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	147 msg = _(b'blindly copying %s containing %i revisions\n')
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	148 ui.note(msg % (unencoded, len(old_revlog)))
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	149 _copyrevlog(tr, dstrepo, old_revlog, unencoded)
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	150
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	151 newrl = _revlogfrompath(dstrepo, unencoded)
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	152 return newrl
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	153
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	154
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	155 def _clonerevlogs(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	156 ui,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	157 srcrepo,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	158 dstrepo,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	159 tr,
46216 34efa84a43a1 engine: pass upgrade operation inside _clonerevlogs() Pulkit Goyal <7895pulkit@gmail.com> parents: 46215 diff changeset	160 upgrade_op,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	161 ):
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	162 """Copy revlogs between 2 repos."""
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	163 revcount = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	164 srcsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	165 srcrawsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	166 dstsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	167 fcount = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	168 frevcount = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	169 fsrcsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	170 frawsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	171 fdstsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	172 mcount = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	173 mrevcount = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	174 msrcsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	175 mrawsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	176 mdstsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	177 crevcount = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	178 csrcsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	179 crawsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	180 cdstsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	181
42712 896fb9deeaf8 upgrade: walk the source store file only once Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42137 diff changeset	182 alldatafiles = list(srcrepo.store.walk())
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	183 # mapping of data files which needs to be cloned
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	184 # key is unencoded filename
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	185 # value is revlog_object_from_srcrepo
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	186 manifests = {}
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	187 changelogs = {}
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	188 filelogs = {}
42712 896fb9deeaf8 upgrade: walk the source store file only once Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42137 diff changeset	189
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	190 # Perform a pass to collect metadata. This validates we can open all
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	191 # source files and allows a unified progress bar to be displayed.
42712 896fb9deeaf8 upgrade: walk the source store file only once Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42137 diff changeset	192 for unencoded, encoded, size in alldatafiles:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	193 if unencoded.endswith(b'.d'):
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	194 continue
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	195
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	196 rl = _revlogfrompath(srcrepo, unencoded)
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	197
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	198 info = rl.storageinfo(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	199 exclusivefiles=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	200 revisionscount=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	201 trackedsize=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	202 storedsize=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	203 )
39862 8dab7c8a93eb upgrade: report size of backing files, not internal storage size Gregory Szorc <gregory.szorc@gmail.com> parents: 39563 diff changeset	204
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	205 revcount += info[b'revisionscount'] or 0
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	206 datasize = info[b'storedsize'] or 0
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	207 rawsize = info[b'trackedsize'] or 0
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	208
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	209 srcsize += datasize
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	210 srcrawsize += rawsize
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	211
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	212 # This is for the separate progress bars.
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	213 if isinstance(rl, changelog.changelog):
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	214 changelogs[unencoded] = rl
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	215 crevcount += len(rl)
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	216 csrcsize += datasize
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	217 crawsize += rawsize
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	218 elif isinstance(rl, manifest.manifestrevlog):
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	219 manifests[unencoded] = rl
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	220 mcount += 1
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	221 mrevcount += len(rl)
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	222 msrcsize += datasize
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	223 mrawsize += rawsize
37444 c8666a9e9e11 upgrade: sniff for filelog type Gregory Szorc <gregory.szorc@gmail.com> parents: 36404 diff changeset	224 elif isinstance(rl, filelog.filelog):
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	225 filelogs[unencoded] = rl
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	226 fcount += 1
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	227 frevcount += len(rl)
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	228 fsrcsize += datasize
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	229 frawsize += rawsize
37444 c8666a9e9e11 upgrade: sniff for filelog type Gregory Szorc <gregory.szorc@gmail.com> parents: 36404 diff changeset	230 else:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	231 error.ProgrammingError(b'unknown revlog type')
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	232
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	233 if not revcount:
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	234 return
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	235
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	236 ui.status(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	237 _(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	238 b'migrating %d total revisions (%d in filelogs, %d in manifests, '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	239 b'%d in changelog)\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	240 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	241 % (revcount, frevcount, mrevcount, crevcount)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	242 )
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	243 ui.status(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	244 _(b'migrating %s in store; %s tracked data\n')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	245 % ((util.bytecount(srcsize), util.bytecount(srcrawsize)))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	246 )
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	247
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	248 # Used to keep track of progress.
38405 185588cb0c4b upgrade: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38404 diff changeset	249 progress = None
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	250
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	251 def oncopiedrevision(rl, rev, node):
38405 185588cb0c4b upgrade: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38404 diff changeset	252 progress.increment()
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	253
43134 75ad8af9c95e upgrade: allow upgrade to repository using sidedata Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43117 diff changeset	254 sidedatacompanion = getsidedatacompanion(srcrepo, dstrepo)
75ad8af9c95e upgrade: allow upgrade to repository using sidedata Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43117 diff changeset	255
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	256 # Migrating filelogs
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	257 ui.status(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	258 _(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	259 b'migrating %d filelogs containing %d revisions '
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	260 b'(%s in store; %s tracked data)\n'
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	261 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	262 % (
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	263 fcount,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	264 frevcount,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	265 util.bytecount(fsrcsize),
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	266 util.bytecount(frawsize),
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	267 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	268 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	269 progress = srcrepo.ui.makeprogress(_(b'file revisions'), total=frevcount)
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	270 for unencoded, oldrl in sorted(filelogs.items()):
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	271 newrl = _perform_clone(
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	272 ui,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	273 dstrepo,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	274 tr,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	275 oldrl,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	276 unencoded,
46217 02df91e895bd engine: pass upgrade operation inside `_perform_clone()` Pulkit Goyal <7895pulkit@gmail.com> parents: 46216 diff changeset	277 upgrade_op,
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	278 sidedatacompanion,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	279 oncopiedrevision,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	280 )
39875 b399ff55ee6d upgrade: use storageinfo() for obtaining storage metadata Gregory Szorc <gregory.szorc@gmail.com> parents: 39864 diff changeset	281 info = newrl.storageinfo(storedsize=True)
46214 5dfa837d933e engine: refactor how total dstsize is calculated Pulkit Goyal <7895pulkit@gmail.com> parents: 46210 diff changeset	282 fdstsize += info[b'storedsize'] or 0
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	283 ui.status(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	284 _(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	285 b'finished migrating %d filelog revisions across %d '
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	286 b'filelogs; change in size: %s\n'
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	287 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	288 % (frevcount, fcount, util.bytecount(fdstsize - fsrcsize))
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	289 )
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	290
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	291 # Migrating manifests
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	292 ui.status(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	293 _(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	294 b'migrating %d manifests containing %d revisions '
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	295 b'(%s in store; %s tracked data)\n'
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	296 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	297 % (
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	298 mcount,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	299 mrevcount,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	300 util.bytecount(msrcsize),
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	301 util.bytecount(mrawsize),
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	302 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	303 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	304 if progress:
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	305 progress.complete()
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	306 progress = srcrepo.ui.makeprogress(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	307 _(b'manifest revisions'), total=mrevcount
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	308 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	309 for unencoded, oldrl in sorted(manifests.items()):
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	310 newrl = _perform_clone(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	311 ui,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	312 dstrepo,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	313 tr,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	314 oldrl,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	315 unencoded,
46217 02df91e895bd engine: pass upgrade operation inside `_perform_clone()` Pulkit Goyal <7895pulkit@gmail.com> parents: 46216 diff changeset	316 upgrade_op,
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	317 sidedatacompanion,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	318 oncopiedrevision,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	319 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	320 info = newrl.storageinfo(storedsize=True)
46214 5dfa837d933e engine: refactor how total dstsize is calculated Pulkit Goyal <7895pulkit@gmail.com> parents: 46210 diff changeset	321 mdstsize += info[b'storedsize'] or 0
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	322 ui.status(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	323 _(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	324 b'finished migrating %d manifest revisions across %d '
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	325 b'manifests; change in size: %s\n'
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	326 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	327 % (mrevcount, mcount, util.bytecount(mdstsize - msrcsize))
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	328 )
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	329
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	330 # Migrating changelog
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	331 ui.status(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	332 _(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	333 b'migrating changelog containing %d revisions '
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	334 b'(%s in store; %s tracked data)\n'
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	335 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	336 % (
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	337 crevcount,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	338 util.bytecount(csrcsize),
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	339 util.bytecount(crawsize),
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	340 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	341 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	342 if progress:
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	343 progress.complete()
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	344 progress = srcrepo.ui.makeprogress(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	345 _(b'changelog revisions'), total=crevcount
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	346 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	347 for unencoded, oldrl in sorted(changelogs.items()):
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	348 newrl = _perform_clone(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	349 ui,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	350 dstrepo,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	351 tr,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	352 oldrl,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	353 unencoded,
46217 02df91e895bd engine: pass upgrade operation inside `_perform_clone()` Pulkit Goyal <7895pulkit@gmail.com> parents: 46216 diff changeset	354 upgrade_op,
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	355 sidedatacompanion,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	356 oncopiedrevision,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	357 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	358 info = newrl.storageinfo(storedsize=True)
46214 5dfa837d933e engine: refactor how total dstsize is calculated Pulkit Goyal <7895pulkit@gmail.com> parents: 46210 diff changeset	359 cdstsize += info[b'storedsize'] or 0
38405 185588cb0c4b upgrade: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38404 diff changeset	360 progress.complete()
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	361 ui.status(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	362 _(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	363 b'finished migrating %d changelog revisions; change in size: '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	364 b'%s\n'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	365 )
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	366 % (crevcount, util.bytecount(cdstsize - csrcsize))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	367 )
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	368
46214 5dfa837d933e engine: refactor how total dstsize is calculated Pulkit Goyal <7895pulkit@gmail.com> parents: 46210 diff changeset	369 dstsize = fdstsize + mdstsize + cdstsize
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	370 ui.status(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	371 _(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	372 b'finished migrating %d total revisions; total change in store '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	373 b'size: %s\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	374 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	375 % (revcount, util.bytecount(dstsize - srcsize))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	376 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	377
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	378
31873 d6c2005484b6 upgrade: drop the prefix to the '_filterstorefile' function Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31872 diff changeset	379 def _filterstorefile(srcrepo, dstrepo, requirements, path, mode, st):
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	380 """Determine whether to copy a store file during upgrade.
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	381
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	382 This function is called when migrating store files from ``srcrepo`` to
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	383 ``dstrepo`` as part of upgrading a repository.
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	384
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	385 Args:
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	386 srcrepo: repo we are copying from
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	387 dstrepo: repo we are copying to
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	388 requirements: set of requirements for ``dstrepo``
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	389 path: store file being examined
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	390 mode: the ``ST_MODE`` file type of ``path``
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	391 st: ``stat`` data structure for ``path``
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	392
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	393 Function should return ``True`` if the file is to be copied.
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	394 """
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	395 # Skip revlogs.
44900 526d69eeea31 upgrade: support upgrade and downgrade from persistent nodemap Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44876 diff changeset	396 if path.endswith((b'.i', b'.d', b'.n', b'.nd')):
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	397 return False
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	398 # Skip transaction related files.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	399 if path.startswith(b'undo'):
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	400 return False
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	401 # Only copy regular files.
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	402 if mode != stat.S_IFREG:
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	403 return False
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	404 # Skip other skipped files.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	405 if path in (b'lock', b'fncache'):
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	406 return False
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	407
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	408 return True
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	409
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	410
46220 1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	411 def _replacestores(currentrepo, upgradedrepo, backupvfs, upgrade_op):
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	412 """Replace the stores after current repository is upgraded
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	413
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	414 Creates a backup of current repository store at backup path
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	415 Replaces upgraded store files in current repo from upgraded one
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	416
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	417 Arguments:
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	418 currentrepo: repo object of current repository
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	419 upgradedrepo: repo object of the upgraded data
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	420 backupvfs: vfs object for the backup path
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	421 upgrade_op: upgrade operation object
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	422 to be used to decide what all is upgraded
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	423 """
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	424 # TODO: don't blindly rename everything in store
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	425 # There can be upgrades where store is not touched at all
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	426 util.rename(currentrepo.spath, backupvfs.join(b'store'))
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	427 util.rename(upgradedrepo.spath, currentrepo.spath)
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	428
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	429
46219 481d9aed669c engine: make hook point for extension a public function Pulkit Goyal <7895pulkit@gmail.com> parents: 46218 diff changeset	430 def finishdatamigration(ui, srcrepo, dstrepo, requirements):
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	431 """Hook point for extensions to perform additional actions during upgrade.
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	432
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	433 This function is called after revlogs and store files have been copied but
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	434 before the new store is swapped into the original location.
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	435 """
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	436
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	437
46060 c407513a44a3 upgrade: start moving the "to be happening" data in a dedicated object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46050 diff changeset	438 def upgrade(ui, srcrepo, dstrepo, upgrade_op):
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	439 """Do the low-level work of upgrading a repository.
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	440
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	441 The upgrade is effectively performed as a copy between a source
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	442 repository and a temporary destination repository.
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	443
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	444 The source repository is unmodified for as long as possible so the
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	445 upgrade can abort at any time without causing loss of service for
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	446 readers and without corrupting the source repository.
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	447 """
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	448 assert srcrepo.currentwlock()
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	449 assert dstrepo.currentwlock()
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	450
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	451 ui.status(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	452 _(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	453 b'(it is safe to interrupt this process any time before '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	454 b'data migration completes)\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	455 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	456 )
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	457
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	458 with dstrepo.transaction(b'upgrade') as tr:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	459 _clonerevlogs(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	460 ui,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	461 srcrepo,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	462 dstrepo,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	463 tr,
46216 34efa84a43a1 engine: pass upgrade operation inside _clonerevlogs() Pulkit Goyal <7895pulkit@gmail.com> parents: 46215 diff changeset	464 upgrade_op,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	465 )
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	466
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	467 # Now copy other files in the store directory.
30895 0b913e1e725b repair: iterate store files deterministically Gregory Szorc <gregory.szorc@gmail.com> parents: 30781 diff changeset	468 # The sorted() makes execution deterministic.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	469 for p, kind, st in sorted(srcrepo.store.vfs.readdir(b'', stat=True)):
46060 c407513a44a3 upgrade: start moving the "to be happening" data in a dedicated object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46050 diff changeset	470 if not _filterstorefile(
46191 aba979b1b90b upgrade: move `printrequirements()` to UpgradeOperation class Pulkit Goyal <7895pulkit@gmail.com> parents: 46189 diff changeset	471 srcrepo, dstrepo, upgrade_op.new_requirements, p, kind, st
46060 c407513a44a3 upgrade: start moving the "to be happening" data in a dedicated object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46050 diff changeset	472 ):
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	473 continue
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	474
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	475 srcrepo.ui.status(_(b'copying %s\n') % p)
30897 8110d49e0558 repair: use rawvfs when copying extra store files Gregory Szorc <gregory.szorc@gmail.com> parents: 30895 diff changeset	476 src = srcrepo.store.rawvfs.join(p)
8110d49e0558 repair: use rawvfs when copying extra store files Gregory Szorc <gregory.szorc@gmail.com> parents: 30895 diff changeset	477 dst = dstrepo.store.rawvfs.join(p)
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	478 util.copyfile(src, dst, copystat=True)
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	479
46219 481d9aed669c engine: make hook point for extension a public function Pulkit Goyal <7895pulkit@gmail.com> parents: 46218 diff changeset	480 finishdatamigration(ui, srcrepo, dstrepo, requirements)
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	481
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	482 ui.status(_(b'data fully migrated to temporary repository\n'))
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	483
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	484 backuppath = pycompat.mkdtemp(prefix=b'upgradebackup.', dir=srcrepo.path)
31242 254f98326eef vfs: use 'vfs' module directly in 'mercurial.repair' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30995 diff changeset	485 backupvfs = vfsmod.vfs(backuppath)
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	486
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	487 # Make a backup of requires file first, as it is the first to be modified.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	488 util.copyfile(srcrepo.vfs.join(b'requires'), backupvfs.join(b'requires'))
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	489
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	490 # We install an arbitrary requirement that clients must not support
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	491 # as a mechanism to lock out new clients during the data swap. This is
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	492 # better than allowing a client to continue while the repository is in
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	493 # an inconsistent state.
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	494 ui.status(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	495 _(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	496 b'marking source repository as being upgraded; clients will be '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	497 b'unable to read from repository\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	498 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	499 )
45106 a03c177a4679 scmutil: add writereporequirements() and route requires writing through it Pulkit Goyal <7895pulkit@gmail.com> parents: 44981 diff changeset	500 scmutil.writereporequirements(
a03c177a4679 scmutil: add writereporequirements() and route requires writing through it Pulkit Goyal <7895pulkit@gmail.com> parents: 44981 diff changeset	501 srcrepo, srcrepo.requirements \| {b'upgradeinprogress'}
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	502 )
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	503
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	504 ui.status(_(b'starting in-place swap of repository data\n'))
e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	505 ui.status(_(b'replaced files will be backed up at %s\n') % backuppath)
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	506
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	507 # Now swap in the new store directory. Doing it as a rename should make
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	508 # the operation nearly instantaneous and atomic (at least in well-behaved
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	509 # environments).
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	510 ui.status(_(b'replacing store...\n'))
30995 22fbca1d11ed mercurial: switch to util.timer for all interval timings Simon Farnsworth <simonfar@fb.com> parents: 30781 diff changeset	511 tstart = util.timer()
46220 1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	512 _replacestores(srcrepo, dstrepo, backupvfs, upgrade_op)
30995 22fbca1d11ed mercurial: switch to util.timer for all interval timings Simon Farnsworth <simonfar@fb.com> parents: 30781 diff changeset	513 elapsed = util.timer() - tstart
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	514 ui.status(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	515 _(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	516 b'store replacement complete; repository was inconsistent for '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	517 b'%0.1fs\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	518 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	519 % elapsed
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	520 )
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	521
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	522 # We first write the requirements file. Any new requirements will lock
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	523 # out legacy clients.
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	524 ui.status(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	525 _(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	526 b'finalizing requirements file and making repository readable '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	527 b'again\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	528 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	529 )
46191 aba979b1b90b upgrade: move `printrequirements()` to UpgradeOperation class Pulkit Goyal <7895pulkit@gmail.com> parents: 46189 diff changeset	530 scmutil.writereporequirements(srcrepo, upgrade_op.new_requirements)
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	531
30781 f2c069bf78ee repair: clean up stale lock file from store backup Gregory Szorc <gregory.szorc@gmail.com> parents: 30780 diff changeset	532 # The lock file from the old store won't be removed because nothing has a
f2c069bf78ee repair: clean up stale lock file from store backup Gregory Szorc <gregory.szorc@gmail.com> parents: 30780 diff changeset	533 # reference to its new location. So clean it up manually. Alternatively, we
f2c069bf78ee repair: clean up stale lock file from store backup Gregory Szorc <gregory.szorc@gmail.com> parents: 30780 diff changeset	534 # could update srcrepo.svfs and other variables to point to the new
f2c069bf78ee repair: clean up stale lock file from store backup Gregory Szorc <gregory.szorc@gmail.com> parents: 30780 diff changeset	535 # location. This is simpler.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	536 backupvfs.unlink(b'store/lock')
30781 f2c069bf78ee repair: clean up stale lock file from store backup Gregory Szorc <gregory.szorc@gmail.com> parents: 30780 diff changeset	537
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	538 return backuppath

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/upgrade_utils/engine.py @ 46220:1ca7865c245d