mercurial-scm/hg-stable: mercurial/upgrade

annotate mercurial/upgrade_utils/engine.py @ 47659:f030c7d22032

walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Changeset 0b569c75d180 introduced new code in store.walk to filter out undo files left behind by the transaction. However doing so is also filtering out legitimate revlog file starting with `undo.` So this changeset is mostly rolling back that change and adding tests tests to catch this kind of error in the future. As a result we the transaction undo files a considered again by various code (in practice mostly persistent nodemap related). We either live with it (low inconvenient) or explicitly work around it for now. This should be good enough to no longer block the 5.9rc release with this issue. We shall build something cleaner within the 6.0 cycle. Differential Revision: https://phab.mercurial-scm.org/D11201

author	Pierre-Yves David <pierre-yves.david@octobus.net>
date	Mon, 19 Jul 2021 22:39:08 +0200
parents	a43d256c041a
children	aa2296893168

rev	line source
31894 9379689b6c10 upgrade: update the header comment Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31893 diff changeset	1 # upgrade.py - functions for in place upgrade of Mercurial repository
4702 18e91c9def0c strip: move strip code to a new repair module Matt Mackall <mpm@selenic.com> parents: diff changeset	2 #
31895 783b4c9bd5f5 upgrade: update the copyright statement Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31894 diff changeset	3 # Copyright (c) 2016-present, Gregory Szorc
4702 18e91c9def0c strip: move strip code to a new repair module Matt Mackall <mpm@selenic.com> parents: diff changeset	4 #
8225 46293a0c7e9f updated license to be explicit about GPL version 2 Martin Geisler <mg@lazybytes.net> parents: 8073 diff changeset	5 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 9150 diff changeset	6 # GNU General Public License version 2 or any later version.
4702 18e91c9def0c strip: move strip code to a new repair module Matt Mackall <mpm@selenic.com> parents: diff changeset	7
25970 d1419cfbd4f4 repair: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25874 diff changeset	8 from __future__ import absolute_import
d1419cfbd4f4 repair: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25874 diff changeset	9
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	10 import stat
4702 18e91c9def0c strip: move strip code to a new repair module Matt Mackall <mpm@selenic.com> parents: diff changeset	11
46050 f105c49e89cd upgrade: split actual upgrade code away from the main module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46039 diff changeset	12 from ..i18n import _
f105c49e89cd upgrade: split actual upgrade code away from the main module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46039 diff changeset	13 from ..pycompat import getattr
f105c49e89cd upgrade: split actual upgrade code away from the main module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46039 diff changeset	14 from .. import (
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	15 changelog,
26587 56b2bcea2529 error: get Abort from 'error' instead of 'util' Pierre-Yves David <pierre-yves.david@fb.com> parents: 26425 diff changeset	16 error,
35353 8f3f8b8dbab7 upgrade: use actual filelog to convert filelog Boris Feld <boris.feld@octobus.net> parents: 35352 diff changeset	17 filelog,
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	18 manifest,
44981 4c1d39215034 metadata: move computation related to files touched in a dedicated module Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44900 diff changeset	19 metadata,
38198 2ce60954b1b7 py3: wrap tempfile.mkdtemp() to use bytes path Yuya Nishihara <yuya@tcha.org> parents: 37444 diff changeset	20 pycompat,
45392 77b8588dd84e requirements: introduce new requirements related module Pulkit Goyal <7895pulkit@gmail.com> parents: 45391 diff changeset	21 requirements,
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	22 scmutil,
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	23 store,
25970 d1419cfbd4f4 repair: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25874 diff changeset	24 util,
31242 254f98326eef vfs: use 'vfs' module directly in 'mercurial.repair' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 30995 diff changeset	25 vfs as vfsmod,
25970 d1419cfbd4f4 repair: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25874 diff changeset	26 )
47098 27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	27 from ..revlogutils import (
27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	28 constants as revlogconst,
27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	29 flagutil,
27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	30 nodemap,
27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	31 sidedata as sidedatamod,
27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	32 )
47328 a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	33 from . import actions as upgrade_actions
47098 27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	34
27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	35
27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	36 def get_sidedata_helpers(srcrepo, dstrepo):
27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	37 use_w = srcrepo.ui.configbool(b'experimental', b'worker.repository-upgrade')
27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	38 sequential = pycompat.iswindows or not use_w
27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	39 if not sequential:
27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	40 srcrepo.register_sidedata_computer(
27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	41 revlogconst.KIND_CHANGELOG,
27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	42 sidedatamod.SD_FILES,
27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	43 (sidedatamod.SD_FILES,),
27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	44 metadata._get_worker_sidedata_adder(srcrepo, dstrepo),
27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	45 flagutil.REVIDX_HASCOPIESINFO,
27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	46 replace=True,
27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	47 )
47099 3aab2330b7d3 sidedata: move sidedata-related utils to the dedicated module Rapha?l Gom?s <rgomes@octobus.net> parents: 47098 diff changeset	48 return sidedatamod.get_sidedata_helpers(srcrepo, dstrepo._wanted_sidedata)
25970 d1419cfbd4f4 repair: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 25874 diff changeset	49
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	50
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	51 def _revlogfrompath(repo, rl_type, path):
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	52 """Obtain a revlog from a repo path.
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	53
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	54 An instance of the appropriate class is returned.
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	55 """
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	56 if rl_type & store.FILEFLAGS_CHANGELOG:
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	57 return changelog.changelog(repo.svfs)
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	58 elif rl_type & store.FILEFLAGS_MANIFESTLOG:
cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	59 mandir = b''
cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	60 if b'/' in path:
cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	61 mandir = path.rsplit(b'/', 1)[0]
46793 6266d19556ad node: introduce nodeconstants class Joerg Sonnenberger <joerg@bec.de> parents: 46576 diff changeset	62 return manifest.manifestrevlog(
6266d19556ad node: introduce nodeconstants class Joerg Sonnenberger <joerg@bec.de> parents: 46576 diff changeset	63 repo.nodeconstants, repo.svfs, tree=mandir
6266d19556ad node: introduce nodeconstants class Joerg Sonnenberger <joerg@bec.de> parents: 46576 diff changeset	64 )
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	65 else:
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	66 # drop the extension and the `data/` prefix
cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	67 path = path.rsplit(b'.', 1)[0].split(b'/', 1)[1]
cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	68 return filelog.filelog(repo.svfs, path)
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	69
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	70
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	71 def _copyrevlog(tr, destrepo, oldrl, rl_type, unencodedname):
42714 5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	72 """copy all relevant files for `oldrl` into `destrepo` store
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	73
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	74 Files are copied "as is" without any transformation. The copy is performed
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	75 without extra checks. Callers are responsible for making sure the copied
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	76 content is compatible with format of the destination repository.
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	77 """
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	78 oldrl = getattr(oldrl, '_revlog', oldrl)
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	79 newrl = _revlogfrompath(destrepo, rl_type, unencodedname)
42714 5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	80 newrl = getattr(newrl, '_revlog', newrl)
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	81
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	82 oldvfs = oldrl.opener
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	83 newvfs = newrl.opener
47162 a07d5cb03a85 revlog: rename `indexfile` to `_indexfile` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47142 diff changeset	84 oldindex = oldvfs.join(oldrl._indexfile)
a07d5cb03a85 revlog: rename `indexfile` to `_indexfile` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47142 diff changeset	85 newindex = newvfs.join(newrl._indexfile)
47163 396442cd7e6a revlog: rename `datafile` to `datafile` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47162 diff changeset	86 olddata = oldvfs.join(oldrl._datafile)
396442cd7e6a revlog: rename `datafile` to `datafile` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47162 diff changeset	87 newdata = newvfs.join(newrl._datafile)
42714 5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	88
47162 a07d5cb03a85 revlog: rename `indexfile` to `_indexfile` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47142 diff changeset	89 with newvfs(newrl._indexfile, b'w'):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	90 pass # create all the directories
42714 5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	91
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	92 util.copyfile(oldindex, newindex)
47163 396442cd7e6a revlog: rename `datafile` to `datafile` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47162 diff changeset	93 copydata = oldrl.opener.exists(oldrl._datafile)
43004 373749982924 upgrade: also register copied `.d` files to fncache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43002 diff changeset	94 if copydata:
42714 5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	95 util.copyfile(olddata, newdata)
5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	96
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	97 if rl_type & store.FILEFLAGS_FILELOG:
42714 5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	98 destrepo.svfs.fncache.add(unencodedname)
43004 373749982924 upgrade: also register copied `.d` files to fncache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43002 diff changeset	99 if copydata:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	100 destrepo.svfs.fncache.add(unencodedname[:-2] + b'.d')
42714 5535a2201ff1 upgrade: introduce a _copyrevlog method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42713 diff changeset	101
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	102
46039 6c960b708ac4 upgrade: display the list of processed revlog before proceeding Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46018 diff changeset	103 UPGRADE_CHANGELOG = b"changelog"
6c960b708ac4 upgrade: display the list of processed revlog before proceeding Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46018 diff changeset	104 UPGRADE_MANIFEST = b"manifest"
6c960b708ac4 upgrade: display the list of processed revlog before proceeding Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46018 diff changeset	105 UPGRADE_FILELOGS = b"all-filelogs"
42715 0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	106
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	107 UPGRADE_ALL_REVLOGS = frozenset(
45995 fe7d7917ceb5 upgrade: rename UPGRADE_FILELOG to UPGRADE_FILELOGS Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45735 diff changeset	108 [UPGRADE_CHANGELOG, UPGRADE_MANIFEST, UPGRADE_FILELOGS]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	109 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	110
42715 0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	111
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	112 def matchrevlog(revlogfilter, rl_type):
45694 d1c10d33a85c upgrade: improve documentation of matchrevlog() Pulkit Goyal <7895pulkit@gmail.com> parents: 45498 diff changeset	113 """check if a revlog is selected for cloning.
d1c10d33a85c upgrade: improve documentation of matchrevlog() Pulkit Goyal <7895pulkit@gmail.com> parents: 45498 diff changeset	114
d1c10d33a85c upgrade: improve documentation of matchrevlog() Pulkit Goyal <7895pulkit@gmail.com> parents: 45498 diff changeset	115 In other words, are there any updates which need to be done on revlog
d1c10d33a85c upgrade: improve documentation of matchrevlog() Pulkit Goyal <7895pulkit@gmail.com> parents: 45498 diff changeset	116 or it can be blindly copied.
42715 0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	117
0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	118 The store entry is checked against the passed filter"""
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	119 if rl_type & store.FILEFLAGS_CHANGELOG:
42715 0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	120 return UPGRADE_CHANGELOG in revlogfilter
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	121 elif rl_type & store.FILEFLAGS_MANIFESTLOG:
42715 0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	122 return UPGRADE_MANIFEST in revlogfilter
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	123 assert rl_type & store.FILEFLAGS_FILELOG
45995 fe7d7917ceb5 upgrade: rename UPGRADE_FILELOG to UPGRADE_FILELOGS Pierre-Yves David <pierre-yves.david@octobus.net> parents: 45735 diff changeset	124 return UPGRADE_FILELOGS in revlogfilter
42715 0812d9fb63fe upgrade: introduce the internal code for revlog cloning selection Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42714 diff changeset	125
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	126
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	127 def _perform_clone(
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	128 ui,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	129 dstrepo,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	130 tr,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	131 old_revlog,
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	132 rl_type,
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	133 unencoded,
46217 02df91e895bd engine: pass upgrade operation inside `_perform_clone()` Pulkit Goyal <7895pulkit@gmail.com> parents: 46216 diff changeset	134 upgrade_op,
47098 27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	135 sidedata_helpers,
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	136 oncopiedrevision,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	137 ):
47031 f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1 Kyle Lippincott <spectral@google.com> parents: 46897 diff changeset	138 """returns the new revlog object created"""
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	139 newrl = None
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	140 if matchrevlog(upgrade_op.revlogs_to_process, rl_type):
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	141 ui.note(
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	142 _(b'cloning %d revisions from %s\n') % (len(old_revlog), unencoded)
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	143 )
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	144 newrl = _revlogfrompath(dstrepo, rl_type, unencoded)
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	145 old_revlog.clone(
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	146 tr,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	147 newrl,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	148 addrevisioncb=oncopiedrevision,
46217 02df91e895bd engine: pass upgrade operation inside `_perform_clone()` Pulkit Goyal <7895pulkit@gmail.com> parents: 46216 diff changeset	149 deltareuse=upgrade_op.delta_reuse_mode,
46218 3f92a9bb80f0 engine: prevent multiple checking of re-delta-multibase Pulkit Goyal <7895pulkit@gmail.com> parents: 46217 diff changeset	150 forcedeltabothparents=upgrade_op.force_re_delta_both_parents,
47098 27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	151 sidedata_helpers=sidedata_helpers,
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	152 )
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	153 else:
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	154 msg = _(b'blindly copying %s containing %i revisions\n')
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	155 ui.note(msg % (unencoded, len(old_revlog)))
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	156 _copyrevlog(tr, dstrepo, old_revlog, rl_type, unencoded)
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	157
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	158 newrl = _revlogfrompath(dstrepo, rl_type, unencoded)
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	159 return newrl
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	160
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	161
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	162 def _clonerevlogs(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	163 ui,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	164 srcrepo,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	165 dstrepo,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	166 tr,
46216 34efa84a43a1 engine: pass upgrade operation inside _clonerevlogs() Pulkit Goyal <7895pulkit@gmail.com> parents: 46215 diff changeset	167 upgrade_op,
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	168 ):
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	169 """Copy revlogs between 2 repos."""
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	170 revcount = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	171 srcsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	172 srcrawsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	173 dstsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	174 fcount = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	175 frevcount = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	176 fsrcsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	177 frawsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	178 fdstsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	179 mcount = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	180 mrevcount = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	181 msrcsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	182 mrawsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	183 mdstsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	184 crevcount = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	185 csrcsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	186 crawsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	187 cdstsize = 0
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	188
42712 896fb9deeaf8 upgrade: walk the source store file only once Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42137 diff changeset	189 alldatafiles = list(srcrepo.store.walk())
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	190 # mapping of data files which needs to be cloned
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	191 # key is unencoded filename
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	192 # value is revlog_object_from_srcrepo
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	193 manifests = {}
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	194 changelogs = {}
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	195 filelogs = {}
42712 896fb9deeaf8 upgrade: walk the source store file only once Pierre-Yves David <pierre-yves.david@octobus.net> parents: 42137 diff changeset	196
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	197 # Perform a pass to collect metadata. This validates we can open all
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	198 # source files and allows a unified progress bar to be displayed.
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	199 for rl_type, unencoded, encoded, size in alldatafiles:
cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	200 if not rl_type & store.FILEFLAGS_REVLOG_MAIN:
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	201 continue
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	202
47659 f030c7d22032 walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47328 diff changeset	203 # the store.walk function will wrongly pickup transaction backup and
f030c7d22032 walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47328 diff changeset	204 # get confused. As a quick fix for 5.9 release, we ignore those.
f030c7d22032 walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47328 diff changeset	205 # (this is not a module constants because it seems better to keep the
f030c7d22032 walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47328 diff changeset	206 # hack together)
f030c7d22032 walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47328 diff changeset	207 skip_undo = (
f030c7d22032 walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47328 diff changeset	208 b'undo.backup.00changelog.i',
f030c7d22032 walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47328 diff changeset	209 b'undo.backup.00manifest.i',
f030c7d22032 walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47328 diff changeset	210 )
f030c7d22032 walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47328 diff changeset	211 if unencoded in skip_undo:
f030c7d22032 walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47328 diff changeset	212 continue
f030c7d22032 walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47328 diff changeset	213
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	214 rl = _revlogfrompath(srcrepo, rl_type, unencoded)
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	215
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	216 info = rl.storageinfo(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	217 exclusivefiles=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	218 revisionscount=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	219 trackedsize=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	220 storedsize=True,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	221 )
39862 8dab7c8a93eb upgrade: report size of backing files, not internal storage size Gregory Szorc <gregory.szorc@gmail.com> parents: 39563 diff changeset	222
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	223 revcount += info[b'revisionscount'] or 0
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	224 datasize = info[b'storedsize'] or 0
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	225 rawsize = info[b'trackedsize'] or 0
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	226
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	227 srcsize += datasize
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	228 srcrawsize += rawsize
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	229
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	230 # This is for the separate progress bars.
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	231 if rl_type & store.FILEFLAGS_CHANGELOG:
cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	232 changelogs[unencoded] = (rl_type, rl)
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	233 crevcount += len(rl)
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	234 csrcsize += datasize
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	235 crawsize += rawsize
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	236 elif rl_type & store.FILEFLAGS_MANIFESTLOG:
cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	237 manifests[unencoded] = (rl_type, rl)
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	238 mcount += 1
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	239 mrevcount += len(rl)
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	240 msrcsize += datasize
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	241 mrawsize += rawsize
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	242 elif rl_type & store.FILEFLAGS_FILELOG:
cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	243 filelogs[unencoded] = (rl_type, rl)
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	244 fcount += 1
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	245 frevcount += len(rl)
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	246 fsrcsize += datasize
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	247 frawsize += rawsize
37444 c8666a9e9e11 upgrade: sniff for filelog type Gregory Szorc <gregory.szorc@gmail.com> parents: 36404 diff changeset	248 else:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	249 error.ProgrammingError(b'unknown revlog type')
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	250
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	251 if not revcount:
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	252 return
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	253
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	254 ui.status(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	255 _(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	256 b'migrating %d total revisions (%d in filelogs, %d in manifests, '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	257 b'%d in changelog)\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	258 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	259 % (revcount, frevcount, mrevcount, crevcount)
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	260 )
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	261 ui.status(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	262 _(b'migrating %s in store; %s tracked data\n')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	263 % ((util.bytecount(srcsize), util.bytecount(srcrawsize)))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	264 )
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	265
38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	266 # Used to keep track of progress.
38405 185588cb0c4b upgrade: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38404 diff changeset	267 progress = None
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	268
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	269 def oncopiedrevision(rl, rev, node):
38405 185588cb0c4b upgrade: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38404 diff changeset	270 progress.increment()
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	271
47098 27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	272 sidedata_helpers = get_sidedata_helpers(srcrepo, dstrepo)
43134 75ad8af9c95e upgrade: allow upgrade to repository using sidedata Pierre-Yves David <pierre-yves.david@octobus.net> parents: 43117 diff changeset	273
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	274 # Migrating filelogs
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	275 ui.status(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	276 _(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	277 b'migrating %d filelogs containing %d revisions '
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	278 b'(%s in store; %s tracked data)\n'
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	279 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	280 % (
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	281 fcount,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	282 frevcount,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	283 util.bytecount(fsrcsize),
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	284 util.bytecount(frawsize),
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	285 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	286 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	287 progress = srcrepo.ui.makeprogress(_(b'file revisions'), total=frevcount)
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	288 for unencoded, (rl_type, oldrl) in sorted(filelogs.items()):
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	289 newrl = _perform_clone(
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	290 ui,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	291 dstrepo,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	292 tr,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	293 oldrl,
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	294 rl_type,
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	295 unencoded,
46217 02df91e895bd engine: pass upgrade operation inside `_perform_clone()` Pulkit Goyal <7895pulkit@gmail.com> parents: 46216 diff changeset	296 upgrade_op,
47098 27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	297 sidedata_helpers,
46193 85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	298 oncopiedrevision,
85f7cf314b39 engine: refactor actual cloning code into separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46191 diff changeset	299 )
39875 b399ff55ee6d upgrade: use storageinfo() for obtaining storage metadata Gregory Szorc <gregory.szorc@gmail.com> parents: 39864 diff changeset	300 info = newrl.storageinfo(storedsize=True)
46214 5dfa837d933e engine: refactor how total dstsize is calculated Pulkit Goyal <7895pulkit@gmail.com> parents: 46210 diff changeset	301 fdstsize += info[b'storedsize'] or 0
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	302 ui.status(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	303 _(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	304 b'finished migrating %d filelog revisions across %d '
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	305 b'filelogs; change in size: %s\n'
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	306 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	307 % (frevcount, fcount, util.bytecount(fdstsize - fsrcsize))
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	308 )
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	309
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	310 # Migrating manifests
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	311 ui.status(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	312 _(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	313 b'migrating %d manifests containing %d revisions '
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	314 b'(%s in store; %s tracked data)\n'
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	315 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	316 % (
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	317 mcount,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	318 mrevcount,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	319 util.bytecount(msrcsize),
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	320 util.bytecount(mrawsize),
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	321 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	322 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	323 if progress:
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	324 progress.complete()
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	325 progress = srcrepo.ui.makeprogress(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	326 _(b'manifest revisions'), total=mrevcount
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	327 )
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	328 for unencoded, (rl_type, oldrl) in sorted(manifests.items()):
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	329 newrl = _perform_clone(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	330 ui,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	331 dstrepo,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	332 tr,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	333 oldrl,
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	334 rl_type,
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	335 unencoded,
46217 02df91e895bd engine: pass upgrade operation inside `_perform_clone()` Pulkit Goyal <7895pulkit@gmail.com> parents: 46216 diff changeset	336 upgrade_op,
47098 27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	337 sidedata_helpers,
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	338 oncopiedrevision,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	339 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	340 info = newrl.storageinfo(storedsize=True)
46214 5dfa837d933e engine: refactor how total dstsize is calculated Pulkit Goyal <7895pulkit@gmail.com> parents: 46210 diff changeset	341 mdstsize += info[b'storedsize'] or 0
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	342 ui.status(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	343 _(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	344 b'finished migrating %d manifest revisions across %d '
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	345 b'manifests; change in size: %s\n'
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	346 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	347 % (mrevcount, mcount, util.bytecount(mdstsize - msrcsize))
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	348 )
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	349
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	350 # Migrating changelog
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	351 ui.status(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	352 _(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	353 b'migrating changelog containing %d revisions '
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	354 b'(%s in store; %s tracked data)\n'
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	355 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	356 % (
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	357 crevcount,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	358 util.bytecount(csrcsize),
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	359 util.bytecount(crawsize),
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	360 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	361 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	362 if progress:
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	363 progress.complete()
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	364 progress = srcrepo.ui.makeprogress(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	365 _(b'changelog revisions'), total=crevcount
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	366 )
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	367 for unencoded, (rl_type, oldrl) in sorted(changelogs.items()):
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	368 newrl = _perform_clone(
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	369 ui,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	370 dstrepo,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	371 tr,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	372 oldrl,
46896 cf49e54ef965 upgrade: take advantage of the new information returned by `store.walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	373 rl_type,
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	374 unencoded,
46217 02df91e895bd engine: pass upgrade operation inside `_perform_clone()` Pulkit Goyal <7895pulkit@gmail.com> parents: 46216 diff changeset	375 upgrade_op,
47098 27f1191b1305 sidedata: replace sidedata upgrade mechanism with the new one Rapha?l Gom?s <rgomes@octobus.net> parents: 46897 diff changeset	376 sidedata_helpers,
46194 4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	377 oncopiedrevision,
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	378 )
4d1cec4e5e1f engine: unwrap a hard to understand for loop Pulkit Goyal <7895pulkit@gmail.com> parents: 46193 diff changeset	379 info = newrl.storageinfo(storedsize=True)
46214 5dfa837d933e engine: refactor how total dstsize is calculated Pulkit Goyal <7895pulkit@gmail.com> parents: 46210 diff changeset	380 cdstsize += info[b'storedsize'] or 0
38405 185588cb0c4b upgrade: use progress helper Martin von Zweigbergk <martinvonz@google.com> parents: 38404 diff changeset	381 progress.complete()
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	382 ui.status(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	383 _(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	384 b'finished migrating %d changelog revisions; change in size: '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	385 b'%s\n'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	386 )
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	387 % (crevcount, util.bytecount(cdstsize - csrcsize))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	388 )
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	389
46214 5dfa837d933e engine: refactor how total dstsize is calculated Pulkit Goyal <7895pulkit@gmail.com> parents: 46210 diff changeset	390 dstsize = fdstsize + mdstsize + cdstsize
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	391 ui.status(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	392 _(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	393 b'finished migrating %d total revisions; total change in store '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	394 b'size: %s\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	395 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	396 % (revcount, util.bytecount(dstsize - srcsize))
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	397 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	398
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	399
46229 52abb1af2995 engine: prevent a function call for each store file Pulkit Goyal <7895pulkit@gmail.com> parents: 46222 diff changeset	400 def _files_to_copy_post_revlog_clone(srcrepo):
52abb1af2995 engine: prevent a function call for each store file Pulkit Goyal <7895pulkit@gmail.com> parents: 46222 diff changeset	401 """yields files which should be copied to destination after revlogs
52abb1af2995 engine: prevent a function call for each store file Pulkit Goyal <7895pulkit@gmail.com> parents: 46222 diff changeset	402 are cloned"""
52abb1af2995 engine: prevent a function call for each store file Pulkit Goyal <7895pulkit@gmail.com> parents: 46222 diff changeset	403 for path, kind, st in sorted(srcrepo.store.vfs.readdir(b'', stat=True)):
52abb1af2995 engine: prevent a function call for each store file Pulkit Goyal <7895pulkit@gmail.com> parents: 46222 diff changeset	404 # don't copy revlogs as they are already cloned
46897 1c52d77d7861 upgrade: do not hardcore file extension of revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46896 diff changeset	405 if store.revlog_type(path) is not None:
46229 52abb1af2995 engine: prevent a function call for each store file Pulkit Goyal <7895pulkit@gmail.com> parents: 46222 diff changeset	406 continue
52abb1af2995 engine: prevent a function call for each store file Pulkit Goyal <7895pulkit@gmail.com> parents: 46222 diff changeset	407 # Skip transaction related files.
52abb1af2995 engine: prevent a function call for each store file Pulkit Goyal <7895pulkit@gmail.com> parents: 46222 diff changeset	408 if path.startswith(b'undo'):
52abb1af2995 engine: prevent a function call for each store file Pulkit Goyal <7895pulkit@gmail.com> parents: 46222 diff changeset	409 continue
52abb1af2995 engine: prevent a function call for each store file Pulkit Goyal <7895pulkit@gmail.com> parents: 46222 diff changeset	410 # Only copy regular files.
52abb1af2995 engine: prevent a function call for each store file Pulkit Goyal <7895pulkit@gmail.com> parents: 46222 diff changeset	411 if kind != stat.S_IFREG:
52abb1af2995 engine: prevent a function call for each store file Pulkit Goyal <7895pulkit@gmail.com> parents: 46222 diff changeset	412 continue
52abb1af2995 engine: prevent a function call for each store file Pulkit Goyal <7895pulkit@gmail.com> parents: 46222 diff changeset	413 # Skip other skipped files.
52abb1af2995 engine: prevent a function call for each store file Pulkit Goyal <7895pulkit@gmail.com> parents: 46222 diff changeset	414 if path in (b'lock', b'fncache'):
52abb1af2995 engine: prevent a function call for each store file Pulkit Goyal <7895pulkit@gmail.com> parents: 46222 diff changeset	415 continue
52abb1af2995 engine: prevent a function call for each store file Pulkit Goyal <7895pulkit@gmail.com> parents: 46222 diff changeset	416 # TODO: should we skip cache too?
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	417
46229 52abb1af2995 engine: prevent a function call for each store file Pulkit Goyal <7895pulkit@gmail.com> parents: 46222 diff changeset	418 yield path
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	419
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	420
46220 1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	421 def _replacestores(currentrepo, upgradedrepo, backupvfs, upgrade_op):
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	422 """Replace the stores after current repository is upgraded
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	423
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	424 Creates a backup of current repository store at backup path
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	425 Replaces upgraded store files in current repo from upgraded one
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	426
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	427 Arguments:
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	428 currentrepo: repo object of current repository
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	429 upgradedrepo: repo object of the upgraded data
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	430 backupvfs: vfs object for the backup path
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	431 upgrade_op: upgrade operation object
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	432 to be used to decide what all is upgraded
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	433 """
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	434 # TODO: don't blindly rename everything in store
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	435 # There can be upgrades where store is not touched at all
46447 2e8a844d0ae0 upgrade: don't create store backup if `--no-backup` is passed Pulkit Goyal <7895pulkit@gmail.com> parents: 46252 diff changeset	436 if upgrade_op.backup_store:
2e8a844d0ae0 upgrade: don't create store backup if `--no-backup` is passed Pulkit Goyal <7895pulkit@gmail.com> parents: 46252 diff changeset	437 util.rename(currentrepo.spath, backupvfs.join(b'store'))
2e8a844d0ae0 upgrade: don't create store backup if `--no-backup` is passed Pulkit Goyal <7895pulkit@gmail.com> parents: 46252 diff changeset	438 else:
2e8a844d0ae0 upgrade: don't create store backup if `--no-backup` is passed Pulkit Goyal <7895pulkit@gmail.com> parents: 46252 diff changeset	439 currentrepo.vfs.rmtree(b'store', forcibly=True)
46220 1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	440 util.rename(upgradedrepo.spath, currentrepo.spath)
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	441
1ca7865c245d engine: refactor code to replace stores in separate function Pulkit Goyal <7895pulkit@gmail.com> parents: 46219 diff changeset	442
46219 481d9aed669c engine: make hook point for extension a public function Pulkit Goyal <7895pulkit@gmail.com> parents: 46218 diff changeset	443 def finishdatamigration(ui, srcrepo, dstrepo, requirements):
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	444 """Hook point for extensions to perform additional actions during upgrade.
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	445
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	446 This function is called after revlogs and store files have been copied but
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	447 before the new store is swapped into the original location.
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	448 """
2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	449
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	450
46060 c407513a44a3 upgrade: start moving the "to be happening" data in a dedicated object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46050 diff changeset	451 def upgrade(ui, srcrepo, dstrepo, upgrade_op):
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	452 """Do the low-level work of upgrading a repository.
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	453
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	454 The upgrade is effectively performed as a copy between a source
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	455 repository and a temporary destination repository.
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	456
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	457 The source repository is unmodified for as long as possible so the
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	458 upgrade can abort at any time without causing loss of service for
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	459 readers and without corrupting the source repository.
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	460 """
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	461 assert srcrepo.currentwlock()
7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	462 assert dstrepo.currentwlock()
46447 2e8a844d0ae0 upgrade: don't create store backup if `--no-backup` is passed Pulkit Goyal <7895pulkit@gmail.com> parents: 46252 diff changeset	463 backuppath = None
2e8a844d0ae0 upgrade: don't create store backup if `--no-backup` is passed Pulkit Goyal <7895pulkit@gmail.com> parents: 46252 diff changeset	464 backupvfs = None
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	465
44875 e295ba238bd8 upgrade: support the --quiet flag Pierre-Yves David <pierre-yves.david@octobus.net> parents: 44874 diff changeset	466 ui.status(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	467 _(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	468 b'(it is safe to interrupt this process any time before '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	469 b'data migration completes)\n'
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	470 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	471 )
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	472
47328 a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	473 if upgrade_actions.dirstatev2 in upgrade_op.upgrade_actions:
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	474 ui.status(_(b'upgrading to dirstate-v2 from v1\n'))
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	475 upgrade_dirstate(ui, srcrepo, upgrade_op, b'v1', b'v2')
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	476 upgrade_op.upgrade_actions.remove(upgrade_actions.dirstatev2)
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	477
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	478 if upgrade_actions.dirstatev2 in upgrade_op.removed_actions:
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	479 ui.status(_(b'downgrading from dirstate-v2 to v1\n'))
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	480 upgrade_dirstate(ui, srcrepo, upgrade_op, b'v2', b'v1')
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	481 upgrade_op.removed_actions.remove(upgrade_actions.dirstatev2)
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	482
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	483 if not (upgrade_op.upgrade_actions or upgrade_op.removed_actions):
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	484 return
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	485
46528 45c3a263d5d1 engine: 'if not, else' -> 'if, else' Pulkit Goyal <7895pulkit@gmail.com> parents: 46452 diff changeset	486 if upgrade_op.requirements_only:
45c3a263d5d1 engine: 'if not, else' -> 'if, else' Pulkit Goyal <7895pulkit@gmail.com> parents: 46452 diff changeset	487 ui.status(_(b'upgrading repository requirements\n'))
45c3a263d5d1 engine: 'if not, else' -> 'if, else' Pulkit Goyal <7895pulkit@gmail.com> parents: 46452 diff changeset	488 scmutil.writereporequirements(srcrepo, upgrade_op.new_requirements)
46533 98e39f04d60e upgrade: implement partial upgrade for upgrading persistent-nodemap Pulkit Goyal <7895pulkit@gmail.com> parents: 46528 diff changeset	489 # if there is only one action and that is persistent nodemap upgrade
98e39f04d60e upgrade: implement partial upgrade for upgrading persistent-nodemap Pulkit Goyal <7895pulkit@gmail.com> parents: 46528 diff changeset	490 # directly write the nodemap file and update requirements instead of going
98e39f04d60e upgrade: implement partial upgrade for upgrading persistent-nodemap Pulkit Goyal <7895pulkit@gmail.com> parents: 46528 diff changeset	491 # through the whole cloning process
98e39f04d60e upgrade: implement partial upgrade for upgrading persistent-nodemap Pulkit Goyal <7895pulkit@gmail.com> parents: 46528 diff changeset	492 elif (
98e39f04d60e upgrade: implement partial upgrade for upgrading persistent-nodemap Pulkit Goyal <7895pulkit@gmail.com> parents: 46528 diff changeset	493 len(upgrade_op.upgrade_actions) == 1
47328 a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	494 and b'persistent-nodemap' in upgrade_op.upgrade_actions_names
46533 98e39f04d60e upgrade: implement partial upgrade for upgrading persistent-nodemap Pulkit Goyal <7895pulkit@gmail.com> parents: 46528 diff changeset	495 and not upgrade_op.removed_actions
98e39f04d60e upgrade: implement partial upgrade for upgrading persistent-nodemap Pulkit Goyal <7895pulkit@gmail.com> parents: 46528 diff changeset	496 ):
98e39f04d60e upgrade: implement partial upgrade for upgrading persistent-nodemap Pulkit Goyal <7895pulkit@gmail.com> parents: 46528 diff changeset	497 ui.status(
98e39f04d60e upgrade: implement partial upgrade for upgrading persistent-nodemap Pulkit Goyal <7895pulkit@gmail.com> parents: 46528 diff changeset	498 _(b'upgrading repository to use persistent nodemap feature\n')
98e39f04d60e upgrade: implement partial upgrade for upgrading persistent-nodemap Pulkit Goyal <7895pulkit@gmail.com> parents: 46528 diff changeset	499 )
98e39f04d60e upgrade: implement partial upgrade for upgrading persistent-nodemap Pulkit Goyal <7895pulkit@gmail.com> parents: 46528 diff changeset	500 with srcrepo.transaction(b'upgrade') as tr:
98e39f04d60e upgrade: implement partial upgrade for upgrading persistent-nodemap Pulkit Goyal <7895pulkit@gmail.com> parents: 46528 diff changeset	501 unfi = srcrepo.unfiltered()
98e39f04d60e upgrade: implement partial upgrade for upgrading persistent-nodemap Pulkit Goyal <7895pulkit@gmail.com> parents: 46528 diff changeset	502 cl = unfi.changelog
98e39f04d60e upgrade: implement partial upgrade for upgrading persistent-nodemap Pulkit Goyal <7895pulkit@gmail.com> parents: 46528 diff changeset	503 nodemap.persist_nodemap(tr, cl, force=True)
46575 636853347e14 upgrade: write nodemap for manifests too Pulkit Goyal <7895pulkit@gmail.com> parents: 46533 diff changeset	504 # we want to directly operate on the underlying revlog to force
636853347e14 upgrade: write nodemap for manifests too Pulkit Goyal <7895pulkit@gmail.com> parents: 46533 diff changeset	505 # create a nodemap file. This is fine since this is upgrade code
636853347e14 upgrade: write nodemap for manifests too Pulkit Goyal <7895pulkit@gmail.com> parents: 46533 diff changeset	506 # and it heavily relies on repository being revlog based
636853347e14 upgrade: write nodemap for manifests too Pulkit Goyal <7895pulkit@gmail.com> parents: 46533 diff changeset	507 # hence accessing private attributes can be justified
636853347e14 upgrade: write nodemap for manifests too Pulkit Goyal <7895pulkit@gmail.com> parents: 46533 diff changeset	508 nodemap.persist_nodemap(
636853347e14 upgrade: write nodemap for manifests too Pulkit Goyal <7895pulkit@gmail.com> parents: 46533 diff changeset	509 tr, unfi.manifestlog._rootstore._revlog, force=True
636853347e14 upgrade: write nodemap for manifests too Pulkit Goyal <7895pulkit@gmail.com> parents: 46533 diff changeset	510 )
46533 98e39f04d60e upgrade: implement partial upgrade for upgrading persistent-nodemap Pulkit Goyal <7895pulkit@gmail.com> parents: 46528 diff changeset	511 scmutil.writereporequirements(srcrepo, upgrade_op.new_requirements)
46576 67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	512 elif (
67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	513 len(upgrade_op.removed_actions) == 1
67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	514 and [
67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	515 x
67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	516 for x in upgrade_op.removed_actions
67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	517 if x.name == b'persistent-nodemap'
67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	518 ]
67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	519 and not upgrade_op.upgrade_actions
67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	520 ):
67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	521 ui.status(
67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	522 _(b'downgrading repository to not use persistent nodemap feature\n')
67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	523 )
67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	524 with srcrepo.transaction(b'upgrade') as tr:
67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	525 unfi = srcrepo.unfiltered()
67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	526 cl = unfi.changelog
67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	527 nodemap.delete_nodemap(tr, srcrepo, cl)
67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	528 # check comment 20 lines above for accessing private attributes
67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	529 nodemap.delete_nodemap(
67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	530 tr, srcrepo, unfi.manifestlog._rootstore._revlog
67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	531 )
67b5fafd3a46 upgrade: speed up when we have only nodemap to downgrade Pulkit Goyal <7895pulkit@gmail.com> parents: 46575 diff changeset	532 scmutil.writereporequirements(srcrepo, upgrade_op.new_requirements)
46528 45c3a263d5d1 engine: 'if not, else' -> 'if, else' Pulkit Goyal <7895pulkit@gmail.com> parents: 46452 diff changeset	533 else:
46451 ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	534 with dstrepo.transaction(b'upgrade') as tr:
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	535 _clonerevlogs(
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	536 ui,
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	537 srcrepo,
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	538 dstrepo,
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	539 tr,
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	540 upgrade_op,
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	541 )
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	542
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	543 # Now copy other files in the store directory.
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	544 for p in _files_to_copy_post_revlog_clone(srcrepo):
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	545 srcrepo.ui.status(_(b'copying %s\n') % p)
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	546 src = srcrepo.store.rawvfs.join(p)
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	547 dst = dstrepo.store.rawvfs.join(p)
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	548 util.copyfile(src, dst, copystat=True)
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	549
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	550 finishdatamigration(ui, srcrepo, dstrepo, requirements)
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	551
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	552 ui.status(_(b'data fully upgraded in a temporary repository\n'))
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	553
46451 ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	554 if upgrade_op.backup_store:
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	555 backuppath = pycompat.mkdtemp(
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	556 prefix=b'upgradebackup.', dir=srcrepo.path
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	557 )
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	558 backupvfs = vfsmod.vfs(backuppath)
30780 2603d04889e1 repair: copy non-revlog store files during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30779 diff changeset	559
46451 ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	560 # Make a backup of requires file first, as it is the first to be modified.
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	561 util.copyfile(
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	562 srcrepo.vfs.join(b'requires'), backupvfs.join(b'requires')
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	563 )
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	564
46451 ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	565 # We install an arbitrary requirement that clients must not support
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	566 # as a mechanism to lock out new clients during the data swap. This is
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	567 # better than allowing a client to continue while the repository is in
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	568 # an inconsistent state.
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	569 ui.status(
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	570 _(
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	571 b'marking source repository as being upgraded; clients will be '
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	572 b'unable to read from repository\n'
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	573 )
46447 2e8a844d0ae0 upgrade: don't create store backup if `--no-backup` is passed Pulkit Goyal <7895pulkit@gmail.com> parents: 46252 diff changeset	574 )
46451 ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	575 scmutil.writereporequirements(
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	576 srcrepo, srcrepo.requirements \| {b'upgradeinprogress'}
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	577 )
30779 38aa1ca97b6a repair: migrate revlogs during upgrade Gregory Szorc <gregory.szorc@gmail.com> parents: 30777 diff changeset	578
46451 ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	579 ui.status(_(b'starting in-place swap of repository data\n'))
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	580 if upgrade_op.backup_store:
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	581 ui.status(
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	582 _(b'replaced files will be backed up at %s\n') % backuppath
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	583 )
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	584
46451 ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	585 # Now swap in the new store directory. Doing it as a rename should make
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	586 # the operation nearly instantaneous and atomic (at least in well-behaved
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	587 # environments).
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	588 ui.status(_(b'replacing store...\n'))
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	589 tstart = util.timer()
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	590 _replacestores(srcrepo, dstrepo, backupvfs, upgrade_op)
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	591 elapsed = util.timer() - tstart
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	592 ui.status(
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	593 _(
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	594 b'store replacement complete; repository was inconsistent for '
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	595 b'%0.1fs\n'
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	596 )
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	597 % elapsed
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	598 )
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	599
46451 ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	600 # We first write the requirements file. Any new requirements will lock
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	601 # out legacy clients.
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	602 ui.status(
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	603 _(
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	604 b'finalizing requirements file and making repository readable '
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	605 b'again\n'
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	606 )
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 43031 diff changeset	607 )
46451 ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	608 scmutil.writereporequirements(srcrepo, upgrade_op.new_requirements)
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	609
46451 ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	610 if upgrade_op.backup_store:
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	611 # The lock file from the old store won't be removed because nothing has a
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	612 # reference to its new location. So clean it up manually. Alternatively, we
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	613 # could update srcrepo.svfs and other variables to point to the new
ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	614 # location. This is simpler.
46796 e2f7b2695ba1 merge with stable Matt Harbison <matt_harbison@yahoo.com> parents: 46793 46402 diff changeset	615 assert backupvfs is not None # help pytype
46451 ee9002b99595 engine: add `if True` to prepare for next patch Pulkit Goyal <7895pulkit@gmail.com> parents: 46447 diff changeset	616 backupvfs.unlink(b'store/lock')
30781 f2c069bf78ee repair: clean up stale lock file from store backup Gregory Szorc <gregory.szorc@gmail.com> parents: 30780 diff changeset	617
30777 7de7afd8bdd9 repair: begin implementation of in-place upgrading Gregory Szorc <gregory.szorc@gmail.com> parents: 30776 diff changeset	618 return backuppath
47328 a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	619
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	620
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	621 def upgrade_dirstate(ui, srcrepo, upgrade_op, old, new):
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	622 if upgrade_op.backup_store:
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	623 backuppath = pycompat.mkdtemp(
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	624 prefix=b'upgradebackup.', dir=srcrepo.path
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	625 )
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	626 ui.status(_(b'replaced files will be backed up at %s\n') % backuppath)
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	627 backupvfs = vfsmod.vfs(backuppath)
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	628 util.copyfile(
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	629 srcrepo.vfs.join(b'requires'), backupvfs.join(b'requires')
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	630 )
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	631 util.copyfile(
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	632 srcrepo.vfs.join(b'dirstate'), backupvfs.join(b'dirstate')
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	633 )
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	634
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	635 assert srcrepo.dirstate._use_dirstate_v2 == (old == b'v2')
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	636 srcrepo.dirstate._map._use_dirstate_tree = True
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	637 srcrepo.dirstate._map.preload()
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	638 srcrepo.dirstate._use_dirstate_v2 = new == b'v2'
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	639 srcrepo.dirstate._map._use_dirstate_v2 = srcrepo.dirstate._use_dirstate_v2
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	640 srcrepo.dirstate._dirty = True
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	641 srcrepo.dirstate.write(None)
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	642
a43d256c041a dirstate-v2: Add `hg debugupgraderepo` command support Simon Sapin <simon.sapin@octobus.net> parents: 47163 diff changeset	643 scmutil.writereporequirements(srcrepo, upgrade_op.new_requirements)

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/upgrade_utils/engine.py @ 47659:f030c7d22032