mercurial-scm/hg-stable: mercurial/store.py annotate

annotate mercurial/store.py @ 52391:3f0cf7bb3086

stream: preserve volatile cache early Since cache file are not protected by the lock, their state might change between their initial detection, the computation of their size and they preservation by the VolatileManager. So we gather all theses step in a single one to avoid such race. This also handle disappearing cache file in the "copy/hardlink" clone cases.

author	Pierre-Yves David <pierre-yves.david@octobus.net>
date	Wed, 04 Dec 2024 23:31:46 +0100
parents	a260d326458f
children	9db77d46de79

rev	line source
50688 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	1 # store.py - repository store handling for Mercurial)
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	2 #
46819 d4ba4d51f85f contributor: change mentions of mpm to olivia Rapha?l Gom?s <rgomes@octobus.net> parents: 46793 diff changeset	3 # Copyright 2008 Olivia Mackall <olivia@selenic.com>
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	4 #
8225 46293a0c7e9f updated license to be explicit about GPL version 2 Martin Geisler <mg@lazybytes.net> parents: 8210 diff changeset	5 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 9133 diff changeset	6 # GNU General Public License version 2 or any later version.
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	7
51901 f4733654f144 typing: add `from __future__ import annotations` to most files Matt Harbison <matt_harbison@yahoo.com> parents: 51868 diff changeset	8 from __future__ import annotations
f4733654f144 typing: add `from __future__ import annotations` to most files Matt Harbison <matt_harbison@yahoo.com> parents: 51868 diff changeset	9
50513 5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	10 import collections
41980 a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	11 import functools
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	12 import os
46990 0b569c75d180 store: exclude `undo.` nodemap's file from `walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46989 diff changeset	13 import re
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	14 import stat
51788 278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51787 diff changeset	15 import typing
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51787 diff changeset	16
51919 2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	17 from typing import (
2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	18 Generator,
2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	19 List,
2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	20 Optional,
2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	21 )
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	22
509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	23 from .i18n import _
50505 521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50503 diff changeset	24 from .thirdparty import attr
51788 278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51787 diff changeset	25
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51787 diff changeset	26 # Force pytype to use the non-vendored package
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51787 diff changeset	27 if typing.TYPE_CHECKING:
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51787 diff changeset	28 # noinspection PyPackageRequirements
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51787 diff changeset	29 import attr
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51787 diff changeset	30
46114 59fa3890d40a node: import symbols explicitly Joerg Sonnenberger <joerg@bec.de> parents: 45957 diff changeset	31 from .node import hex
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	32 from .revlogutils.constants import (
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	33 INDEX_HEADER,
50688 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	34 KIND_CHANGELOG,
e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	35 KIND_FILELOG,
e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	36 KIND_MANIFESTLOG,
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	37 )
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	38 from . import (
42911 3df3b139a43d localrepo: push manifestlog and changelog construction code into store Augie Fackler <augie@google.com> parents: 42664 diff changeset	39 changelog,
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	40 error,
50657 e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	41 filelog,
42911 3df3b139a43d localrepo: push manifestlog and changelog construction code into store Augie Fackler <augie@google.com> parents: 42664 diff changeset	42 manifest,
32411 df448de7cf3b parsers: switch to policy importer Yuya Nishihara <yuya@tcha.org> parents: 31371 diff changeset	43 policy,
30077 8f42d8c412c8 py3: make encodefun in store.py compatible with py3k Mateusz Kwapich <mitrandir@fb.com> parents: 30076 diff changeset	44 pycompat,
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	45 revlog as revlogmod,
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	46 util,
31244 9b7a2ef4f27c vfs: use 'vfs' module directly in 'mercurial.store' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31229 diff changeset	47 vfs as vfsmod,
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	48 )
44060 a61287a95dc3 core: migrate uses of hashlib.sha1 to hashutil.sha1 Augie Fackler <augie@google.com> parents: 43554 diff changeset	49 from .utils import hashutil
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	50
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43106 diff changeset	51 parsers = policy.importmod('parsers')
41980 a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	52 # how much bytes should be read from fncache in one read
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	53 # It is done to prevent loading large fncache files into memory
51762 ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51759 diff changeset	54 fncache_chunksize = 10**6
32411 df448de7cf3b parsers: switch to policy importer Yuya Nishihara <yuya@tcha.org> parents: 31371 diff changeset	55
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	56
51787 2e9e62242451 typing: disable some pytype errors in `mercurial.store` Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	57 def _match_tracked_entry(entry: "BaseStoreEntry", matcher):
40544 9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	58 """parses a fncache entry and returns whether the entry is tracking a path
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	59 matched by matcher or not.
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	60
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	61 If matcher is None, returns True"""
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	62
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	63 if matcher is None:
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	64 return True
51787 2e9e62242451 typing: disable some pytype errors in `mercurial.store` Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	65
2e9e62242451 typing: disable some pytype errors in `mercurial.store` Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	66 # TODO: make this safe for other entry types. Currently, the various
2e9e62242451 typing: disable some pytype errors in `mercurial.store` Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	67 # store.data_entry generators only yield RevlogStoreEntry, so the
2e9e62242451 typing: disable some pytype errors in `mercurial.store` Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	68 # attributes do exist on `entry`.
2e9e62242451 typing: disable some pytype errors in `mercurial.store` Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	69 # pytype: disable=attribute-error
50532 1b776f25302f store: use the boolean property in `store` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50531 diff changeset	70 if entry.is_filelog:
50528 b4953fad744e store: do the revlog matching on entry directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50527 diff changeset	71 return matcher(entry.target_id)
50532 1b776f25302f store: use the boolean property in `store` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50531 diff changeset	72 elif entry.is_manifestlog:
50528 b4953fad744e store: do the revlog matching on entry directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50527 diff changeset	73 return matcher.visitdir(entry.target_id.rstrip(b'/'))
51787 2e9e62242451 typing: disable some pytype errors in `mercurial.store` Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	74 # pytype: enable=attribute-error
50528 b4953fad744e store: do the revlog matching on entry directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50527 diff changeset	75 raise error.ProgrammingError(b"cannot process entry %r" % entry)
40668 66adfd58cb77 store: raise ProgrammingError if unable to decode a storage path Pulkit Goyal <pulkit@yandex-team.ru> parents: 40631 diff changeset	76
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	77
8531 810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	78 # This avoids a collision between a file named foo and a dir named
810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	79 # foo.i or foo.d
17607 cc58dc47cb5e store: use fast C implementation of encodedir() if it's available Adrian Buehlmann <adrian@cadifra.com> parents: 17605 diff changeset	80 def _encodedir(path):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	81 """
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	82 >>> _encodedir(b'data/foo.i')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	83 'data/foo.i'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	84 >>> _encodedir(b'data/foo.i/bla.i')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	85 'data/foo.i.hg/bla.i'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	86 >>> _encodedir(b'data/foo.i.hg/bla.i')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	87 'data/foo.i.hg.hg/bla.i'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	88 >>> _encodedir(b'data/foo.i\\ndata/foo.i/bla.i\\ndata/foo.i.hg/bla.i\\n')
17605 e9cc29be3305 store: add multiline doctest case for encodedir() Adrian Buehlmann <adrian@cadifra.com> parents: 17604 diff changeset	89 'data/foo.i\\ndata/foo.i.hg/bla.i\\ndata/foo.i.hg.hg/bla.i\\n'
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	90 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	91 return (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	92 path.replace(b".hg/", b".hg.hg/")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	93 .replace(b".i/", b".i.hg/")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	94 .replace(b".d/", b".d.hg/")
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	95 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	96
8531 810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	97
17607 cc58dc47cb5e store: use fast C implementation of encodedir() if it's available Adrian Buehlmann <adrian@cadifra.com> parents: 17605 diff changeset	98 encodedir = getattr(parsers, 'encodedir', _encodedir)
cc58dc47cb5e store: use fast C implementation of encodedir() if it's available Adrian Buehlmann <adrian@cadifra.com> parents: 17605 diff changeset	99
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	100
8531 810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	101 def decodedir(path):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	102 """
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	103 >>> decodedir(b'data/foo.i')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	104 'data/foo.i'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	105 >>> decodedir(b'data/foo.i.hg/bla.i')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	106 'data/foo.i/bla.i'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	107 >>> decodedir(b'data/foo.i.hg.hg/bla.i')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	108 'data/foo.i.hg/bla.i'
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	109 """
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	110 if b".hg/" not in path:
8531 810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	111 return path
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	112 return (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	113 path.replace(b".d.hg/", b".d/")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	114 .replace(b".i.hg/", b".i/")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	115 .replace(b".hg.hg/", b".hg/")
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	116 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	117
8531 810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	118
29071 2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	119 def _reserved():
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	120 """characters that are problematic for filesystems
29071 2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	121
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	122 * ascii escapes (0..31)
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	123 * ascii hi (126..255)
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	124 * windows specials
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	125
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	126 these characters will be escaped by encodefunctions
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	127 """
30076 400dfded8a29 py3: make the string unicode so its iterable in py3k Mateusz Kwapich <mitrandir@fb.com> parents: 29338 diff changeset	128 winreserved = [ord(x) for x in u'\\:*?"<>\|']
29071 2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	129 for x in range(32):
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	130 yield x
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	131 for x in range(126, 256):
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	132 yield x
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	133 for x in winreserved:
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	134 yield x
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	135
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	136
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	137 def _buildencodefun():
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	138 """
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	139 >>> enc, dec = _buildencodefun()
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	140
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	141 >>> enc(b'nothing/special.txt')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	142 'nothing/special.txt'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	143 >>> dec(b'nothing/special.txt')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	144 'nothing/special.txt'
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	145
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	146 >>> enc(b'HELLO')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	147 '_h_e_l_l_o'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	148 >>> dec(b'_h_e_l_l_o')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	149 'HELLO'
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	150
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	151 >>> enc(b'hello:world?')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	152 'hello~3aworld~3f'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	153 >>> dec(b'hello~3aworld~3f')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	154 'hello:world?'
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	155
34151 414a3513c2bd doctest: do not embed non-ascii characters in docstring Yuya Nishihara <yuya@tcha.org> parents: 34146 diff changeset	156 >>> enc(b'the\\x07quick\\xADshot')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	157 'the~07quick~adshot'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	158 >>> dec(b'the~07quick~adshot')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	159 'the\\x07quick\\xadshot'
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	160 """
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	161 e = b'_'
31263 64596338ba10 py3: factor out bytechr() function Yuya Nishihara <yuya@tcha.org> parents: 31244 diff changeset	162 xchr = pycompat.bytechr
64596338ba10 py3: factor out bytechr() function Yuya Nishihara <yuya@tcha.org> parents: 31244 diff changeset	163 asciistr = list(map(xchr, range(127)))
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	164 capitals = list(range(ord(b"A"), ord(b"Z") + 1))
30077 8f42d8c412c8 py3: make encodefun in store.py compatible with py3k Mateusz Kwapich <mitrandir@fb.com> parents: 30076 diff changeset	165
44470 9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things Augie Fackler <augie@google.com> parents: 44060 diff changeset	166 cmap = {x: x for x in asciistr}
29071 2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	167 for x in _reserved():
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	168 cmap[xchr(x)] = b"~%02x" % x
30077 8f42d8c412c8 py3: make encodefun in store.py compatible with py3k Mateusz Kwapich <mitrandir@fb.com> parents: 30076 diff changeset	169 for x in capitals + [ord(e)]:
8f42d8c412c8 py3: make encodefun in store.py compatible with py3k Mateusz Kwapich <mitrandir@fb.com> parents: 30076 diff changeset	170 cmap[xchr(x)] = e + xchr(x).lower()
8f42d8c412c8 py3: make encodefun in store.py compatible with py3k Mateusz Kwapich <mitrandir@fb.com> parents: 30076 diff changeset	171
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	172 dmap = {}
49004 f254fc73d956 global: bulk replace simple pycompat.iteritems(x) with x.items() Gregory Szorc <gregory.szorc@gmail.com> parents: 48966 diff changeset	173 for k, v in cmap.items():
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	174 dmap[v] = k
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	175
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	176 def decode(s):
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	177 i = 0
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	178 while i < len(s):
49292 d44e3c45f0e4 py3: replace `pycompat.xrange` by `range` Manuel Jacob <me@manueljacob.de> parents: 49037 diff changeset	179 for l in range(1, 4):
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	180 try:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	181 yield dmap[s[i : i + l]]
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	182 i += l
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	183 break
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	184 except KeyError:
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	185 pass
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	186 else:
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	187 raise KeyError
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	188
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	189 return (
49292 d44e3c45f0e4 py3: replace `pycompat.xrange` by `range` Manuel Jacob <me@manueljacob.de> parents: 49037 diff changeset	190 lambda s: b''.join([cmap[s[c : c + 1]] for c in range(len(s))]),
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	191 lambda s: b''.join(list(decode(s))),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	192 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	193
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	194
776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	195 _encodefname, _decodefname = _buildencodefun()
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	196
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	197
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	198 def encodefilename(s):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	199 """
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	200 >>> encodefilename(b'foo.i/bar.d/bla.hg/hi:world?/HELLO')
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	201 'foo.i.hg/bar.d.hg/bla.hg.hg/hi~3aworld~3f/_h_e_l_l_o'
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	202 """
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	203 return _encodefname(encodedir(s))
776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	204
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	205
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	206 def decodefilename(s):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	207 """
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	208 >>> decodefilename(b'foo.i.hg/bar.d.hg/bla.hg.hg/hi~3aworld~3f/_h_e_l_l_o')
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	209 'foo.i/bar.d/bla.hg/hi:world?/HELLO'
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	210 """
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	211 return decodedir(_decodefname(s))
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	212
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	213
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	214 def _buildlowerencodefun():
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	215 """
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	216 >>> f = _buildlowerencodefun()
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	217 >>> f(b'nothing/special.txt')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	218 'nothing/special.txt'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	219 >>> f(b'HELLO')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	220 'hello'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	221 >>> f(b'hello:world?')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	222 'hello~3aworld~3f'
34151 414a3513c2bd doctest: do not embed non-ascii characters in docstring Yuya Nishihara <yuya@tcha.org> parents: 34146 diff changeset	223 >>> f(b'the\\x07quick\\xADshot')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	224 'the~07quick~adshot'
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	225 """
34221 7e3f078b6f31 py3: use bytechr() in store._buildlowerencodefun() Yuya Nishihara <yuya@tcha.org> parents: 34220 diff changeset	226 xchr = pycompat.bytechr
49292 d44e3c45f0e4 py3: replace `pycompat.xrange` by `range` Manuel Jacob <me@manueljacob.de> parents: 49037 diff changeset	227 cmap = {xchr(x): xchr(x) for x in range(127)}
29071 2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	228 for x in _reserved():
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	229 cmap[xchr(x)] = b"~%02x" % x
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	230 for x in range(ord(b"A"), ord(b"Z") + 1):
34221 7e3f078b6f31 py3: use bytechr() in store._buildlowerencodefun() Yuya Nishihara <yuya@tcha.org> parents: 34220 diff changeset	231 cmap[xchr(x)] = xchr(x).lower()
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	232
34220 96808804b68f store: give name to lowerencode function Yuya Nishihara <yuya@tcha.org> parents: 34151 diff changeset	233 def lowerencode(s):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	234 return b"".join([cmap[c] for c in pycompat.iterbytestr(s)])
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	235
34220 96808804b68f store: give name to lowerencode function Yuya Nishihara <yuya@tcha.org> parents: 34151 diff changeset	236 return lowerencode
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	237
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	238
18430 0459c6555f69 store: implement lowerencode in C Bryan O'Sullivan <bryano@fb.com> parents: 18054 diff changeset	239 lowerencode = getattr(parsers, 'lowerencode', None) or _buildlowerencodefun()
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	240
17570 f53a7b256ca6 store: optimze _auxencode() a bit by grouping the reserved names by length Adrian Buehlmann <adrian@cadifra.com> parents: 17569 diff changeset	241 # Windows reserved names: con, prn, aux, nul, com1..com9, lpt1..lpt9
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	242 _winres3 = (b'aux', b'con', b'prn', b'nul') # length 3
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	243 _winres4 = (b'com', b'lpt') # length 4 (with trailing 1..9)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	244
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	245
12687 34d8247a4595 store: encode first period or space in filenames (issue1713) Adrian Buehlmann <adrian@cadifra.com> parents: 12171 diff changeset	246 def _auxencode(path, dotencode):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	247 """
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	248 Encodes filenames containing names reserved by Windows or which end in
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	249 period or space. Does not touch other single reserved characters c.
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	250 Specifically, c in '\\:?"<>\|' or ord(c) <= 31 are not* encoded here.
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	251 Additionally encodes space or period at the beginning, if dotencode is
17569 e9af2134825c store: explain "aux.foo" versus "foo.aux" in doc of _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17568 diff changeset	252 True. Parameter path is assumed to be all lowercase.
e9af2134825c store: explain "aux.foo" versus "foo.aux" in doc of _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17568 diff changeset	253 A segment only needs encoding if a reserved name appears as a
e9af2134825c store: explain "aux.foo" versus "foo.aux" in doc of _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17568 diff changeset	254 basename (e.g. "aux", "aux.foo"). A directory or file named "foo.aux"
e9af2134825c store: explain "aux.foo" versus "foo.aux" in doc of _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17568 diff changeset	255 doesn't need encoding.
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	256
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	257 >>> s = b'.foo/aux.txt/txt.aux/con/prn/nul/foo.'
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	258 >>> _auxencode(s.split(b'/'), True)
17574 81a033bb29bc store: let _auxencode() return the list of path segments Adrian Buehlmann <adrian@cadifra.com> parents: 17573 diff changeset	259 ['~2efoo', 'au~78.txt', 'txt.aux', 'co~6e', 'pr~6e', 'nu~6c', 'foo~2e']
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	260 >>> s = b'.com1com2/lpt9.lpt4.lpt1/conprn/com0/lpt0/foo.'
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	261 >>> _auxencode(s.split(b'/'), False)
17574 81a033bb29bc store: let _auxencode() return the list of path segments Adrian Buehlmann <adrian@cadifra.com> parents: 17573 diff changeset	262 ['.com1com2', 'lp~749.lpt4.lpt1', 'conprn', 'com0', 'lpt0', 'foo~2e']
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	263 >>> _auxencode([b'foo. '], True)
17574 81a033bb29bc store: let _auxencode() return the list of path segments Adrian Buehlmann <adrian@cadifra.com> parents: 17573 diff changeset	264 ['foo.~20']
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	265 >>> _auxencode([b' .foo'], True)
17574 81a033bb29bc store: let _auxencode() return the list of path segments Adrian Buehlmann <adrian@cadifra.com> parents: 17573 diff changeset	266 ['~20.foo']
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	267 """
17589 b11024849db6 store: parameter path of _auxencode is now a list of strings Adrian Buehlmann <adrian@cadifra.com> parents: 17588 diff changeset	268 for i, n in enumerate(path):
17572 b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	269 if not n:
b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	270 continue
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	271 if dotencode and n[0] in b'. ':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	272 n = b"~%02x" % ord(n[0:1]) + n[1:]
17589 b11024849db6 store: parameter path of _auxencode is now a list of strings Adrian Buehlmann <adrian@cadifra.com> parents: 17588 diff changeset	273 path[i] = n
17572 b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	274 else:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	275 l = n.find(b'.')
17572 b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	276 if l == -1:
b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	277 l = len(n)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	278 if (l == 3 and n[:3] in _winres3) or (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	279 l == 4
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	280 and n[3:4] <= b'9'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	281 and n[3:4] >= b'1'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	282 and n[:3] in _winres4
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	283 ):
17572 b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	284 # encode third letter ('aux' -> 'au~78')
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	285 ec = b"~%02x" % ord(n[2:3])
17572 b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	286 n = n[0:2] + ec + n[3:]
17589 b11024849db6 store: parameter path of _auxencode is now a list of strings Adrian Buehlmann <adrian@cadifra.com> parents: 17588 diff changeset	287 path[i] = n
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	288 if n[-1] in b'. ':
17572 b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	289 # encode last period or space ('foo...' -> 'foo..~2e')
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	290 path[i] = n[:-1] + b"~%02x" % ord(n[-1:])
17589 b11024849db6 store: parameter path of _auxencode is now a list of strings Adrian Buehlmann <adrian@cadifra.com> parents: 17588 diff changeset	291 return path
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	292
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	293
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	294 _maxstorepathlen = 120
00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	295 _dirprefixlen = 8
00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	296 _maxshortdirslen = 8 * (_dirprefixlen + 1) - 4
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	297
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	298
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	299 def _hashencode(path, dotencode):
46114 59fa3890d40a node: import symbols explicitly Joerg Sonnenberger <joerg@bec.de> parents: 45957 diff changeset	300 digest = hex(hashutil.sha1(path).digest())
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	301 le = lowerencode(path[5:]).split(b'/') # skips prefix 'data/' or 'meta/'
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	302 parts = _auxencode(le, dotencode)
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	303 basename = parts[-1]
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	304 _root, ext = os.path.splitext(basename)
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	305 sdirs = []
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	306 sdirslen = 0
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	307 for p in parts[:-1]:
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	308 d = p[:_dirprefixlen]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	309 if d[-1] in b'. ':
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	310 # Windows can't access dirs ending in period or space
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	311 d = d[:-1] + b'_'
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	312 if sdirslen == 0:
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	313 t = len(d)
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	314 else:
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	315 t = sdirslen + 1 + len(d)
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	316 if t > _maxshortdirslen:
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	317 break
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	318 sdirs.append(d)
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	319 sdirslen = t
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	320 dirs = b'/'.join(sdirs)
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	321 if len(dirs) > 0:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	322 dirs += b'/'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	323 res = b'dh/' + dirs + digest + ext
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	324 spaceleft = _maxstorepathlen - len(res)
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	325 if spaceleft > 0:
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	326 filler = basename[:spaceleft]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	327 res = b'dh/' + dirs + filler + digest + ext
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	328 return res
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	329
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	330
17590 eb0884680f5c store: eliminate one level of lambda functions on _hybridencode Adrian Buehlmann <adrian@cadifra.com> parents: 17589 diff changeset	331 def _hybridencode(path, dotencode):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	332 """encodes path with a length limit
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	333
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	334 Encodes all paths that begin with 'data/', according to the following.
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	335
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	336 Default encoding (reversible):
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	337
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	338 Encodes all uppercase letters 'X' as '_x'. All reserved or illegal
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	339 characters are encoded as '~xx', where xx is the two digit hex code
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	340 of the character (see encodefilename).
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	341 Relevant path components consisting of Windows reserved filenames are
17738 b8424c92ba2b spelling: fix minor spell checker issues Mads Kiilerich <mads@kiilerich.com> parents: 17731 diff changeset	342 masked by encoding the third character ('aux' -> 'au~78', see _auxencode).
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	343
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	344 Hashed encoding (not reversible):
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	345
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	346 If the default-encoded path is longer than _maxstorepathlen, a
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	347 non-reversible hybrid hashing of the path is done instead.
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	348 This encoding uses up to _dirprefixlen characters of all directory
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	349 levels of the lowerencoded path, but not more levels than can fit into
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	350 _maxshortdirslen.
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	351 Then follows the filler followed by the sha digest of the full path.
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	352 The filler is the beginning of the basename of the lowerencoded path
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	353 (the basename is everything after the last path separator). The filler
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	354 is as long as possible, filling in characters from the basename until
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	355 the encoded path has _maxstorepathlen characters (or all chars of the
00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	356 basename have been taken).
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	357 The extension (e.g. '.i' or '.d') is preserved.
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	358
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	359 The string 'data/' at the beginning is replaced with 'dh/', if the hashed
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	360 encoding was used.
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	361 """
17609 cbc180cfd60b store: reuse direncoded path in _hybridencode Adrian Buehlmann <adrian@cadifra.com> parents: 17608 diff changeset	362 path = encodedir(path)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	363 ef = _encodefname(path).split(b'/')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	364 res = b'/'.join(_auxencode(ef, dotencode))
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	365 if len(res) > _maxstorepathlen:
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	366 res = _hashencode(path, dotencode)
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	367 return res
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	368
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	369
17624 ae103510f6aa store: add a fallback _pathencode Python function Adrian Buehlmann <adrian@cadifra.com> parents: 17623 diff changeset	370 def _pathencode(path):
18435 8c019d2fd7c0 store: switch to C-based hashed path encoding Bryan O'Sullivan <bryano@fb.com> parents: 18430 diff changeset	371 de = encodedir(path)
17693 0c6de45e1212 store: optimize _pathencode by checking the length of the unencoded path Adrian Buehlmann <adrian@cadifra.com> parents: 17653 diff changeset	372 if len(path) > _maxstorepathlen:
18435 8c019d2fd7c0 store: switch to C-based hashed path encoding Bryan O'Sullivan <bryano@fb.com> parents: 18430 diff changeset	373 return _hashencode(de, True)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	374 ef = _encodefname(de).split(b'/')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	375 res = b'/'.join(_auxencode(ef, True))
17624 ae103510f6aa store: add a fallback _pathencode Python function Adrian Buehlmann <adrian@cadifra.com> parents: 17623 diff changeset	376 if len(res) > _maxstorepathlen:
18435 8c019d2fd7c0 store: switch to C-based hashed path encoding Bryan O'Sullivan <bryano@fb.com> parents: 18430 diff changeset	377 return _hashencode(de, True)
17624 ae103510f6aa store: add a fallback _pathencode Python function Adrian Buehlmann <adrian@cadifra.com> parents: 17623 diff changeset	378 return res
ae103510f6aa store: add a fallback _pathencode Python function Adrian Buehlmann <adrian@cadifra.com> parents: 17623 diff changeset	379
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	380
17624 ae103510f6aa store: add a fallback _pathencode Python function Adrian Buehlmann <adrian@cadifra.com> parents: 17623 diff changeset	381 _pathencode = getattr(parsers, 'pathencode', _pathencode)
ae103510f6aa store: add a fallback _pathencode Python function Adrian Buehlmann <adrian@cadifra.com> parents: 17623 diff changeset	382
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	383
17623 448e6ed7c557 store: move _plainhybridencode and _dothybridencode higher up in the file Adrian Buehlmann <adrian@cadifra.com> parents: 17621 diff changeset	384 def _plainhybridencode(f):
448e6ed7c557 store: move _plainhybridencode and _dothybridencode higher up in the file Adrian Buehlmann <adrian@cadifra.com> parents: 17621 diff changeset	385 return _hybridencode(f, False)
448e6ed7c557 store: move _plainhybridencode and _dothybridencode higher up in the file Adrian Buehlmann <adrian@cadifra.com> parents: 17621 diff changeset	386
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	387
17726 7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17725 diff changeset	388 def _calcmode(vfs):
6898 69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	389 try:
69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	390 # files in .hg/ will be created using this mode
17726 7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17725 diff changeset	391 mode = vfs.stat().st_mode
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	392 # avoid some useless chmods
25658 e93036747902 global: mass rewrite to use modern octal syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25091 diff changeset	393 if (0o777 & ~util.umask) == (0o777 & mode):
6898 69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	394 mode = None
69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	395 except OSError:
69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	396 mode = None
69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	397 return mode
69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	398
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	399
45372 909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	400 _data = [
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	401 b'bookmarks',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	402 b'narrowspec',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	403 b'data',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	404 b'meta',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	405 b'00manifest.d',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	406 b'00manifest.i',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	407 b'00changelog.d',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	408 b'00changelog.i',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	409 b'phaseroots',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	410 b'obsstore',
45494 d252f51ab032 share: introduce config option to store requires in .hg/store Pulkit Goyal <7895pulkit@gmail.com> parents: 45372 diff changeset	411 b'requires',
45372 909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	412 ]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	413
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	414 REVLOG_FILES_EXT = (
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	415 b'.i',
47332 0a3fa41fa719 revlogv2: use a unique filename for data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47245 diff changeset	416 b'.idx',
0a3fa41fa719 revlogv2: use a unique filename for data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47245 diff changeset	417 b'.d',
0a3fa41fa719 revlogv2: use a unique filename for data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47245 diff changeset	418 b'.dat',
0a3fa41fa719 revlogv2: use a unique filename for data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47245 diff changeset	419 b'.n',
0a3fa41fa719 revlogv2: use a unique filename for data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47245 diff changeset	420 b'.nd',
47395 e6292eb33384 revlog: store sidedata in their own file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47332 diff changeset	421 b'.sda',
47332 0a3fa41fa719 revlogv2: use a unique filename for data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47245 diff changeset	422 )
50513 5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	423 # file extension that also use a `-SOMELONGIDHASH.ext` form
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	424 REVLOG_FILES_LONG_EXT = (
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	425 b'.nd',
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	426 b'.idx',
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	427 b'.dat',
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	428 b'.sda',
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	429 )
46989 aed6ceaad6d7 streamclone: treat volatile file as "fullfile" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	430 # files that are "volatile" and might change between listing and streaming
aed6ceaad6d7 streamclone: treat volatile file as "fullfile" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	431 #
aed6ceaad6d7 streamclone: treat volatile file as "fullfile" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	432 # note: the ".nd" file are nodemap data and won't "change" but they might be
aed6ceaad6d7 streamclone: treat volatile file as "fullfile" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	433 # deleted.
aed6ceaad6d7 streamclone: treat volatile file as "fullfile" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	434 REVLOG_FILES_VOLATILE_EXT = (b'.n', b'.nd')
46895 6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	435
46990 0b569c75d180 store: exclude `undo.` nodemap's file from `walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46989 diff changeset	436 # some exception to the above matching
47659 f030c7d22032 walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47449 diff changeset	437 #
f030c7d22032 walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47449 diff changeset	438 # XXX This is currently not in use because of issue6542
50319 4be9ecc982e1 py3: fix for Python 3.12 emitting SyntaxWarning on invalid escape sequences Mads Kiilerich <mads@kiilerich.com> parents: 50315 diff changeset	439 EXCLUDED = re.compile(br'.*undo\.[^/]+\.(nd?\|i)$')
46990 0b569c75d180 store: exclude `undo.` nodemap's file from `walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46989 diff changeset	440
46895 6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	441
6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	442 def is_revlog(f, kind, st):
6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	443 if kind != stat.S_IFREG:
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	444 return False
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	445 if f.endswith(REVLOG_FILES_EXT):
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	446 return True
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	447 return False
46895 6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	448
6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	449
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	450 def is_revlog_file(f):
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	451 if f.endswith(REVLOG_FILES_EXT):
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	452 return True
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	453 return False
46455 374d7fff7cb5 store: use `endswith` to detect revlog extension Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46322 diff changeset	454
6903 0642d9d7ec80 clone: get a list of files to clone from store Matt Mackall <mpm@selenic.com> parents: 6902 diff changeset	455
50668 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	456 @attr.s(slots=True)
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	457 class StoreFile:
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	458 """a file matching a store entry"""
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	459
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	460 unencoded_path = attr.ib()
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	461 _file_size = attr.ib(default=None)
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	462 is_volatile = attr.ib(default=False)
52391 3f0cf7bb3086 stream: preserve volatile cache early Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52389 diff changeset	463 # Missing file can be safely ignored, used by "copy/hardlink" local clone
3f0cf7bb3086 stream: preserve volatile cache early Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52389 diff changeset	464 # for cache file not covered by lock.
3f0cf7bb3086 stream: preserve volatile cache early Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52389 diff changeset	465 optional = False
50668 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	466
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	467 def file_size(self, vfs):
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	468 if self._file_size is None:
50669 5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	469 if vfs is None:
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	470 msg = b"calling vfs-less file_size without prior call: %s"
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	471 msg %= self.unencoded_path
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	472 raise error.ProgrammingError(msg)
50668 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	473 try:
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	474 self._file_size = vfs.stat(self.unencoded_path).st_size
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	475 except FileNotFoundError:
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	476 self._file_size = 0
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	477 return self._file_size
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	478
51585 8e2a68e10f5c stream: in v3, skip the "size" fast path if the entries as some unknown size Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	479 @property
8e2a68e10f5c stream: in v3, skip the "size" fast path if the entries as some unknown size Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	480 def has_size(self):
8e2a68e10f5c stream: in v3, skip the "size" fast path if the entries as some unknown size Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	481 return self._file_size is not None
8e2a68e10f5c stream: in v3, skip the "size" fast path if the entries as some unknown size Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	482
52043 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51919 diff changeset	483 def get_stream(self, vfs, volatiles):
50669 5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	484 """return data "stream" information for this file
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	485
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	486 (unencoded_file_path, content_iterator, content_size)
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	487 """
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	488 size = self.file_size(None)
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	489
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	490 def get_stream():
52044 0ad269e24075 stream: open volatile file through the manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52043 diff changeset	491 path = vfs.join(self.unencoded_path)
0ad269e24075 stream: open volatile file through the manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52043 diff changeset	492 with volatiles.open(path) as fp:
50669 5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	493 yield None # ready to stream
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	494 if size <= 65536:
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	495 yield fp.read(size)
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	496 else:
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	497 yield from util.filechunkiter(fp, limit=size)
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	498
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	499 s = get_stream()
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	500 next(s)
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	501 return (self.unencoded_path, s, size)
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	502
50668 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	503
50508 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	504 @attr.s(slots=True, init=False)
50507 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	505 class BaseStoreEntry:
50505 521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50503 diff changeset	506 """An entry in the store
521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50503 diff changeset	507
521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50503 diff changeset	508 This is returned by `store.walk` and represent some data in the store."""
521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50503 diff changeset	509
51590 49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	510 maybe_volatile = True
49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	511
50668 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	512 def files(self) -> List[StoreFile]:
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	513 raise NotImplementedError
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	514
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	515 def get_streams(
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	516 self,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	517 repo=None,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	518 vfs=None,
52043 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51919 diff changeset	519 volatiles=None,
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	520 max_changeset=None,
50706 0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	521 preserve_file_count=False,
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	522 ):
50669 5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	523 """return a list of data stream associated to files for this entry
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	524
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	525 return [(unencoded_file_path, content_iterator, content_size), …]
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	526 """
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	527 assert vfs is not None
52043 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51919 diff changeset	528 return [f.get_stream(vfs, volatiles) for f in self.files()]
50669 5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	529
52389 a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52301 diff changeset	530 def preserve_volatiles(self, vfs, volatiles):
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52301 diff changeset	531 """Use a VolatileManager to preserve the state of any volatile file
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52301 diff changeset	532
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52301 diff changeset	533 This is useful for code that need a consistent view of the content like stream clone.
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52301 diff changeset	534 """
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52301 diff changeset	535 if self.maybe_volatile:
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52301 diff changeset	536 for f in self.files():
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52301 diff changeset	537 if f.is_volatile:
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52301 diff changeset	538 volatiles(vfs.join(f.unencoded_path))
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52301 diff changeset	539
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	540
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	541 @attr.s(slots=True, init=False)
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	542 class SimpleStoreEntry(BaseStoreEntry):
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	543 """A generic entry in the store"""
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	544
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	545 is_revlog = False
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	546
51590 49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	547 maybe_volatile = attr.ib()
50529 ed8cda1c18e1 store: rename `unencoded_path` to `entry_path` for StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50528 diff changeset	548 _entry_path = attr.ib()
50510 2b2284cf949b store: only access is_volatile information through the file object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50509 diff changeset	549 _is_volatile = attr.ib(default=False)
50509 7d4d2a160cf5 store: only access file_size information through the file object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50508 diff changeset	550 _file_size = attr.ib(default=None)
50661 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	551 _files = attr.ib(default=None)
50505 521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50503 diff changeset	552
50508 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	553 def __init__(
c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	554 self,
50529 ed8cda1c18e1 store: rename `unencoded_path` to `entry_path` for StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50528 diff changeset	555 entry_path,
50508 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	556 is_volatile=False,
c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	557 file_size=None,
c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	558 ):
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	559 super().__init__()
50529 ed8cda1c18e1 store: rename `unencoded_path` to `entry_path` for StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50528 diff changeset	560 self._entry_path = entry_path
50510 2b2284cf949b store: only access is_volatile information through the file object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50509 diff changeset	561 self._is_volatile = is_volatile
50509 7d4d2a160cf5 store: only access file_size information through the file object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50508 diff changeset	562 self._file_size = file_size
50661 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	563 self._files = None
51590 49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	564 self.maybe_volatile = is_volatile
50508 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	565
50668 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	566 def files(self) -> List[StoreFile]:
50661 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	567 if self._files is None:
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	568 self._files = [
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	569 StoreFile(
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	570 unencoded_path=self._entry_path,
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	571 file_size=self._file_size,
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	572 is_volatile=self._is_volatile,
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	573 )
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	574 ]
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	575 return self._files
50506 9fdc28e21b68 store: introduce a EntryFile object to actually access file info Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50505 diff changeset	576
9fdc28e21b68 store: introduce a EntryFile object to actually access file info Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50505 diff changeset	577
50508 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	578 @attr.s(slots=True, init=False)
50507 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	579 class RevlogStoreEntry(BaseStoreEntry):
5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	580 """A revlog entry in the store"""
5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	581
5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	582 is_revlog = True
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	583
50507 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	584 revlog_type = attr.ib(default=None)
50517 60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50516 diff changeset	585 target_id = attr.ib(default=None)
51590 49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	586 maybe_volatile = attr.ib(default=True)
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	587 _path_prefix = attr.ib(default=None)
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	588 _details = attr.ib(default=None)
50661 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	589 _files = attr.ib(default=None)
50507 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	590
50508 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	591 def __init__(
c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	592 self,
c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	593 revlog_type,
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	594 path_prefix,
50517 60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50516 diff changeset	595 target_id,
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	596 details,
50508 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	597 ):
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	598 super().__init__()
50508 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	599 self.revlog_type = revlog_type
50517 60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50516 diff changeset	600 self.target_id = target_id
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	601 self._path_prefix = path_prefix
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	602 assert b'.i' in details, (path_prefix, details)
51590 49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	603 for ext in details:
49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	604 if ext.endswith(REVLOG_FILES_VOLATILE_EXT):
49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	605 self.maybe_volatile = True
49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	606 break
49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	607 else:
49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	608 self.maybe_volatile = False
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	609 self._details = details
50661 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	610 self._files = None
50508 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	611
50531 66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50530 diff changeset	612 @property
66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50530 diff changeset	613 def is_changelog(self):
50688 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	614 return self.revlog_type == KIND_CHANGELOG
50531 66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50530 diff changeset	615
66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50530 diff changeset	616 @property
66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50530 diff changeset	617 def is_manifestlog(self):
50688 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	618 return self.revlog_type == KIND_MANIFESTLOG
50531 66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50530 diff changeset	619
66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50530 diff changeset	620 @property
66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50530 diff changeset	621 def is_filelog(self):
50688 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	622 return self.revlog_type == KIND_FILELOG
50531 66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50530 diff changeset	623
50526 3473d18c029a store: introduce a main_file_path method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50517 diff changeset	624 def main_file_path(self):
3473d18c029a store: introduce a main_file_path method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50517 diff changeset	625 """unencoded path of the main revlog file"""
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	626 return self._path_prefix + b'.i'
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	627
50668 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	628 def files(self) -> List[StoreFile]:
50661 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	629 if self._files is None:
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	630 self._files = []
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	631 for ext in sorted(self._details, key=_ext_key):
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	632 path = self._path_prefix + ext
50678 11562d72cb7b store: directly pass the filesize in the `details` of revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50677 diff changeset	633 file_size = self._details[ext]
50676 6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50671 diff changeset	634 # files that are "volatile" and might change between
6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50671 diff changeset	635 # listing and streaming
6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50671 diff changeset	636 #
6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50671 diff changeset	637 # note: the ".nd" file are nodemap data and won't "change"
6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50671 diff changeset	638 # but they might be deleted.
6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50671 diff changeset	639 volatile = ext.endswith(REVLOG_FILES_VOLATILE_EXT)
50677 cdb471c8ebcf store: explicitly pass file_size when creating StoreFile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50676 diff changeset	640 f = StoreFile(path, file_size, volatile)
50676 6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50671 diff changeset	641 self._files.append(f)
50661 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	642 return self._files
50526 3473d18c029a store: introduce a main_file_path method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50517 diff changeset	643
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	644 def get_streams(
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	645 self,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	646 repo=None,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	647 vfs=None,
52043 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51919 diff changeset	648 volatiles=None,
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	649 max_changeset=None,
50706 0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	650 preserve_file_count=False,
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	651 ):
51585 8e2a68e10f5c stream: in v3, skip the "size" fast path if the entries as some unknown size Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	652 pre_sized = all(f.has_size for f in self.files())
8e2a68e10f5c stream: in v3, skip the "size" fast path if the entries as some unknown size Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	653 if pre_sized and (
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	654 repo is None
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	655 or max_changeset is None
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	656 # This use revlog-v2, ignore for now
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	657 or any(k.endswith(b'.idx') for k in self._details.keys())
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	658 # This is not inline, no race expected
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	659 or b'.d' in self._details
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	660 ):
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	661 return super().get_streams(
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	662 repo=repo,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	663 vfs=vfs,
52043 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51919 diff changeset	664 volatiles=volatiles,
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	665 max_changeset=max_changeset,
50706 0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	666 preserve_file_count=preserve_file_count,
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	667 )
50706 0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	668 elif not preserve_file_count:
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	669 stream = [
52043 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51919 diff changeset	670 f.get_stream(vfs, volatiles)
50706 0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	671 for f in self.files()
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	672 if not f.unencoded_path.endswith((b'.i', b'.d'))
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	673 ]
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	674 rl = self.get_revlog_instance(repo).get_revlog()
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	675 rl_stream = rl.get_streams(max_changeset)
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	676 stream.extend(rl_stream)
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	677 return stream
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	678
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	679 name_to_size = {}
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	680 for f in self.files():
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	681 name_to_size[f.unencoded_path] = f.file_size(None)
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	682
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	683 stream = [
52043 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51919 diff changeset	684 f.get_stream(vfs, volatiles)
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	685 for f in self.files()
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	686 if not f.unencoded_path.endswith(b'.i')
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	687 ]
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	688
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	689 index_path = self._path_prefix + b'.i'
50671 5460424092e2 stream-clone: smoothly detect and handle a case were a revlog is split Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50670 diff changeset	690
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	691 index_file = None
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	692 try:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	693 index_file = vfs(index_path)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	694 header = index_file.read(INDEX_HEADER.size)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	695 if revlogmod.revlog.is_inline_index(header):
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	696 size = name_to_size[index_path]
50671 5460424092e2 stream-clone: smoothly detect and handle a case were a revlog is split Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50670 diff changeset	697
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	698 # no split underneath, just return the stream
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	699 def get_stream():
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	700 fp = index_file
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	701 try:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	702 fp.seek(0)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	703 yield None
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	704 if size <= 65536:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	705 yield fp.read(size)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	706 else:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	707 yield from util.filechunkiter(fp, limit=size)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	708 finally:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	709 fp.close()
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	710
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	711 s = get_stream()
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	712 next(s)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	713 index_file = None
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	714 stream.append((index_path, s, size))
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	715 else:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	716 rl = self.get_revlog_instance(repo).get_revlog()
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	717 rl_stream = rl.get_streams(max_changeset, force_inline=True)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	718 for name, s, size in rl_stream:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	719 if name_to_size.get(name, 0) != size:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	720 msg = _(b"expected %d bytes but %d provided for %s")
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	721 msg %= name_to_size.get(name, 0), size, name
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	722 raise error.Abort(msg)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	723 stream.extend(rl_stream)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	724 finally:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	725 if index_file is not None:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	726 index_file.close()
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	727
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	728 files = self.files()
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	729 assert len(stream) == len(files), (
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	730 stream,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	731 files,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	732 self._path_prefix,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	733 self.target_id,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	734 )
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	735 return stream
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	736
50657 e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	737 def get_revlog_instance(self, repo):
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	738 """Obtain a revlog instance from this store entry
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	739
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	740 An instance of the appropriate class is returned.
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	741 """
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	742 if self.is_changelog:
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	743 return changelog.changelog(repo.svfs)
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	744 elif self.is_manifestlog:
50658 3ea3767c23a4 store: do not drop the final `/` when creating manifestlog instance Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50657 diff changeset	745 mandir = self.target_id
50657 e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	746 return manifest.manifestrevlog(
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	747 repo.nodeconstants, repo.svfs, tree=mandir
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	748 )
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	749 else:
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	750 return filelog.filelog(repo.svfs, self.target_id)
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	751
50507 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	752
50513 5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	753 def _gather_revlog(files_data):
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	754 """group files per revlog prefix
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	755
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	756 The returns a two level nested dict. The top level key is the revlog prefix
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	757 without extension, the second level is all the file "suffix" that were
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	758 seen for this revlog and arbitrary file data as value.
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	759 """
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	760 revlogs = collections.defaultdict(dict)
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	761 for u, value in files_data:
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	762 name, ext = _split_revlog_ext(u)
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	763 revlogs[name][ext] = value
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	764 return sorted(revlogs.items())
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	765
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	766
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	767 def _split_revlog_ext(filename):
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	768 """split the revlog file prefix from the variable extension"""
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	769 if filename.endswith(REVLOG_FILES_LONG_EXT):
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	770 char = b'-'
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	771 else:
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	772 char = b'.'
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	773 idx = filename.rfind(char)
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	774 return filename[:idx], filename[idx:]
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	775
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	776
50515 b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	777 def _ext_key(ext):
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	778 """a key to order revlog suffix
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	779
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	780 important to issue .i after other entry."""
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	781 # the only important part of this order is to keep the `.i` last.
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	782 if ext.endswith(b'.n'):
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	783 return (0, ext)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	784 elif ext.endswith(b'.nd'):
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	785 return (10, ext)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	786 elif ext.endswith(b'.d'):
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	787 return (20, ext)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	788 elif ext.endswith(b'.i'):
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	789 return (50, ext)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	790 else:
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	791 return (40, ext)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	792
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	793
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49032 diff changeset	794 class basicstore:
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	795 '''base class for local repository stores'''
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	796
17651 3b49c28658f6 store: rename "openertype" argument to "vfstype" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17649 diff changeset	797 def __init__(self, path, vfstype):
17724 bf4b72d8dd4d store: initialize vfs field first to use it for initialization of others FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17722 diff changeset	798 vfs = vfstype(path)
bf4b72d8dd4d store: initialize vfs field first to use it for initialization of others FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17722 diff changeset	799 self.path = vfs.base
17726 7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17725 diff changeset	800 self.createmode = _calcmode(vfs)
17652 2c6f7231becc store: rename "op" variables to "vfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17651 diff changeset	801 vfs.createmode = self.createmode
17728 004bd533880d store: invoke "os.path.isdir()" via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17727 diff changeset	802 self.rawvfs = vfs
31244 9b7a2ef4f27c vfs: use 'vfs' module directly in 'mercurial.store' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31229 diff changeset	803 self.vfs = vfsmod.filtervfs(vfs, encodedir)
17653 dacb50696b75 store: initialize "vfs" fields by "vfs" constructors FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17652 diff changeset	804 self.opener = self.vfs
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	805
80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	806 def join(self, f):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	807 return self.path + b'/' + encodedir(f)
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	808
50527 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	809 def _walk(self, relpath, recurse, undecodable=None):
47918 2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	810 '''yields (revlog_type, unencoded, size)'''
13426 643b8212813e store: remove pointless pathjoiner parameter Adrian Buehlmann <adrian@cadifra.com> parents: 13391 diff changeset	811 path = self.path
643b8212813e store: remove pointless pathjoiner parameter Adrian Buehlmann <adrian@cadifra.com> parents: 13391 diff changeset	812 if relpath:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	813 path += b'/' + relpath
13426 643b8212813e store: remove pointless pathjoiner parameter Adrian Buehlmann <adrian@cadifra.com> parents: 13391 diff changeset	814 striplen = len(self.path) + 1
6899 56a7a54e074f store: simplify walking Matt Mackall <mpm@selenic.com> parents: 6898 diff changeset	815 l = []
17728 004bd533880d store: invoke "os.path.isdir()" via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17727 diff changeset	816 if self.rawvfs.isdir(path):
6899 56a7a54e074f store: simplify walking Matt Mackall <mpm@selenic.com> parents: 6898 diff changeset	817 visit = [path]
17747 aad3bce98f76 store: invoke "osutil.listdir()" via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17745 diff changeset	818 readdir = self.rawvfs.readdir
6899 56a7a54e074f store: simplify walking Matt Mackall <mpm@selenic.com> parents: 6898 diff changeset	819 while visit:
56a7a54e074f store: simplify walking Matt Mackall <mpm@selenic.com> parents: 6898 diff changeset	820 p = visit.pop()
17747 aad3bce98f76 store: invoke "osutil.listdir()" via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17745 diff changeset	821 for f, kind, st in readdir(p, stat=True):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	822 fp = p + b'/' + f
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	823 if is_revlog(f, kind, st):
6900 def492d1b592 store: change handling of decoding errors Matt Mackall <mpm@selenic.com> parents: 6899 diff changeset	824 n = util.pconvert(fp[striplen:])
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	825 l.append((decodedir(n), st.st_size))
6899 56a7a54e074f store: simplify walking Matt Mackall <mpm@selenic.com> parents: 6898 diff changeset	826 elif kind == stat.S_IFDIR and recurse:
56a7a54e074f store: simplify walking Matt Mackall <mpm@selenic.com> parents: 6898 diff changeset	827 visit.append(fp)
50512 1c0244a8cdaf store: change `_walk` return to `(filename, (type, size))` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50511 diff changeset	828
17054 125ff5654b72 store: sort filenames in place Bryan O'Sullivan <bryano@fb.com> parents: 16404 diff changeset	829 l.sort()
125ff5654b72 store: sort filenames in place Bryan O'Sullivan <bryano@fb.com> parents: 16404 diff changeset	830 return l
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	831
46646 e9901d01d135 revlog: add a mechanism to verify expected file position before appending Kyle Lippincott <spectral@google.com> parents: 46455 diff changeset	832 def changelog(self, trypending, concurrencychecker=None):
e9901d01d135 revlog: add a mechanism to verify expected file position before appending Kyle Lippincott <spectral@google.com> parents: 46455 diff changeset	833 return changelog.changelog(
e9901d01d135 revlog: add a mechanism to verify expected file position before appending Kyle Lippincott <spectral@google.com> parents: 46455 diff changeset	834 self.vfs,
e9901d01d135 revlog: add a mechanism to verify expected file position before appending Kyle Lippincott <spectral@google.com> parents: 46455 diff changeset	835 trypending=trypending,
e9901d01d135 revlog: add a mechanism to verify expected file position before appending Kyle Lippincott <spectral@google.com> parents: 46455 diff changeset	836 concurrencychecker=concurrencychecker,
e9901d01d135 revlog: add a mechanism to verify expected file position before appending Kyle Lippincott <spectral@google.com> parents: 46455 diff changeset	837 )
42911 3df3b139a43d localrepo: push manifestlog and changelog construction code into store Augie Fackler <augie@google.com> parents: 42664 diff changeset	838
52094 6641a3ae8313 manifest: drop the CamelCase name for `manifest.manifestlog` Matt Harbison <matt_harbison@yahoo.com> parents: 52044 diff changeset	839 def manifestlog(self, repo, storenarrowmatch) -> manifest.manifestlog:
46793 6266d19556ad node: introduce nodeconstants class Joerg Sonnenberger <joerg@bec.de> parents: 46646 diff changeset	840 rootstore = manifest.manifestrevlog(repo.nodeconstants, self.vfs)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	841 return manifest.manifestlog(self.vfs, repo, rootstore, storenarrowmatch)
42911 3df3b139a43d localrepo: push manifestlog and changelog construction code into store Augie Fackler <augie@google.com> parents: 42664 diff changeset	842
50538 862e3a13da44 store: rename `datafiles` to `data_entries` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50532 diff changeset	843 def data_entries(
50505 521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50503 diff changeset	844 self, matcher=None, undecodable=None
50507 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	845 ) -> Generator[BaseStoreEntry, None, None]:
47918 2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	846 """Like walk, but excluding the changelog and root manifest.
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	847
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	848 When [undecodable] is None, revlogs names that can't be
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	849 decoded cause an exception. When it is provided, it should
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	850 be a list and the filenames that can't be decoded are added
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	851 to it instead. This is very rarely needed."""
50516 1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50515 diff changeset	852 dirs = [
50688 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	853 (b'data', KIND_FILELOG, False),
e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	854 (b'meta', KIND_MANIFESTLOG, True),
50516 1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50515 diff changeset	855 ]
50659 309cbd8400ae store: properly compute the targer_id of manifestlog in no-fncache walk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50658 diff changeset	856 for base_dir, rl_type, strip_filename in dirs:
50527 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	857 files = self._walk(base_dir, True, undecodable=undecodable)
50516 1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50515 diff changeset	858 for revlog, details in _gather_revlog(files):
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	859 revlog_target_id = revlog.split(b'/', 1)[1]
50659 309cbd8400ae store: properly compute the targer_id of manifestlog in no-fncache walk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50658 diff changeset	860 if strip_filename and b'/' in revlog:
309cbd8400ae store: properly compute the targer_id of manifestlog in no-fncache walk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50658 diff changeset	861 revlog_target_id = revlog_target_id.rsplit(b'/', 1)[0]
309cbd8400ae store: properly compute the targer_id of manifestlog in no-fncache walk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50658 diff changeset	862 revlog_target_id += b'/'
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	863 yield RevlogStoreEntry(
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	864 path_prefix=revlog,
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	865 revlog_type=rl_type,
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	866 target_id=revlog_target_id,
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	867 details=details,
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	868 )
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	869
50548 0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	870 def top_entries(
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	871 self, phase=False, obsolescence=False
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	872 ) -> Generator[BaseStoreEntry, None, None]:
50547 5a62d56e3955 store: yield phases before changelog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50546 diff changeset	873 if phase and self.vfs.exists(b'phaseroots'):
5a62d56e3955 store: yield phases before changelog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50546 diff changeset	874 yield SimpleStoreEntry(
5a62d56e3955 store: yield phases before changelog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50546 diff changeset	875 entry_path=b'phaseroots',
5a62d56e3955 store: yield phases before changelog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50546 diff changeset	876 is_volatile=True,
5a62d56e3955 store: yield phases before changelog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50546 diff changeset	877 )
5a62d56e3955 store: yield phases before changelog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50546 diff changeset	878
50548 0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	879 if obsolescence and self.vfs.exists(b'obsstore'):
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	880 # XXX if we had the file size it could be non-volatile
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	881 yield SimpleStoreEntry(
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	882 entry_path=b'obsstore',
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	883 is_volatile=True,
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	884 )
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	885
50515 b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	886 files = reversed(self._walk(b'', False))
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	887
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	888 changelogs = collections.defaultdict(dict)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	889 manifestlogs = collections.defaultdict(dict)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	890
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	891 for u, s in files:
46895 6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	892 if u.startswith(b'00changelog'):
50515 b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	893 name, ext = _split_revlog_ext(u)
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	894 changelogs[name][ext] = s
46895 6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	895 elif u.startswith(b'00manifest'):
50515 b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	896 name, ext = _split_revlog_ext(u)
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	897 manifestlogs[name][ext] = s
46895 6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	898 else:
50507 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	899 yield SimpleStoreEntry(
50529 ed8cda1c18e1 store: rename `unencoded_path` to `entry_path` for StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50528 diff changeset	900 entry_path=u,
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	901 is_volatile=False,
50507 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	902 file_size=s,
5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	903 )
50515 b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	904 # yield manifest before changelog
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	905 top_rl = [
50688 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	906 (manifestlogs, KIND_MANIFESTLOG),
e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	907 (changelogs, KIND_CHANGELOG),
50515 b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	908 ]
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	909 assert len(manifestlogs) <= 1
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	910 assert len(changelogs) <= 1
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	911 for data, revlog_type in top_rl:
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	912 for revlog, details in sorted(data.items()):
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	913 yield RevlogStoreEntry(
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	914 path_prefix=revlog,
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	915 revlog_type=revlog_type,
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	916 target_id=b'',
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	917 details=details,
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	918 )
19177 1e104aaa4c44 store: move top file walk to a separate function Durham Goode <durham@fb.com> parents: 18435 diff changeset	919
50546 a32d739b0ffb store: make `walk` return an entry for phase if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	920 def walk(
50548 0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	921 self, matcher=None, phase=False, obsolescence=False
50546 a32d739b0ffb store: make `walk` return an entry for phase if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	922 ) -> Generator[BaseStoreEntry, None, None]:
50485 584ff1f97201 comments: fix spelling Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50319 diff changeset	923 """return files related to data storage (ie: revlogs)
46853 eed3e2b79b48 store: document the `walk` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	924
50546 a32d739b0ffb store: make `walk` return an entry for phase if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	925 yields instance from BaseStoreEntry subclasses
40340 2d45b549392f store: pass matcher to store.datafiles() Pulkit Goyal <pulkit@yandex-team.ru> parents: 39714 diff changeset	926
2d45b549392f store: pass matcher to store.datafiles() Pulkit Goyal <pulkit@yandex-team.ru> parents: 39714 diff changeset	927 if a matcher is passed, storage files of only those tracked paths
2d45b549392f store: pass matcher to store.datafiles() Pulkit Goyal <pulkit@yandex-team.ru> parents: 39714 diff changeset	928 are passed with matches the matcher
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	929 """
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	930 # yield data files first
50538 862e3a13da44 store: rename `datafiles` to `data_entries` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50532 diff changeset	931 for x in self.data_entries(matcher):
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	932 yield x
50548 0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	933 for x in self.top_entries(phase=phase, obsolescence=obsolescence):
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	934 yield x
80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	935
6903 0642d9d7ec80 clone: get a list of files to clone from store Matt Mackall <mpm@selenic.com> parents: 6902 diff changeset	936 def copylist(self):
45494 d252f51ab032 share: introduce config option to store requires in .hg/store Pulkit Goyal <7895pulkit@gmail.com> parents: 45372 diff changeset	937 return _data
6903 0642d9d7ec80 clone: get a list of files to clone from store Matt Mackall <mpm@selenic.com> parents: 6902 diff changeset	938
20883 cd443c7589cc fncache: move fncache writing to be in a transaction Durham Goode <durham@fb.com> parents: 20879 diff changeset	939 def write(self, tr):
13391 d00bbff8600e fncachestore: defer updating the fncache file to a single file open Adrian Buehlmann <adrian@cadifra.com> parents: 13169 diff changeset	940 pass
d00bbff8600e fncachestore: defer updating the fncache file to a single file open Adrian Buehlmann <adrian@cadifra.com> parents: 13169 diff changeset	941
20884 2efdd186925d caches: invalidate store caches when lock is taken Durham Goode <durham@fb.com> parents: 20883 diff changeset	942 def invalidatecaches(self):
2efdd186925d caches: invalidate store caches when lock is taken Durham Goode <durham@fb.com> parents: 20883 diff changeset	943 pass
2efdd186925d caches: invalidate store caches when lock is taken Durham Goode <durham@fb.com> parents: 20883 diff changeset	944
20885 f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	945 def markremoved(self, fn):
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	946 pass
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	947
17744 09d5b2055295 store: add a contains method to basicstore smuralid parents: 17738 diff changeset	948 def __contains__(self, path):
09d5b2055295 store: add a contains method to basicstore smuralid parents: 17738 diff changeset	949 '''Checks if the store contains path'''
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	950 path = b"/".join((b"data", path))
17744 09d5b2055295 store: add a contains method to basicstore smuralid parents: 17738 diff changeset	951 # file?
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	952 if self.vfs.exists(path + b".i"):
17744 09d5b2055295 store: add a contains method to basicstore smuralid parents: 17738 diff changeset	953 return True
09d5b2055295 store: add a contains method to basicstore smuralid parents: 17738 diff changeset	954 # dir?
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	955 if not path.endswith(b"/"):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	956 path = path + b"/"
19903 ca875b271ac3 store: use "vfs.exists()" instead of "os.path.exists()" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 19177 diff changeset	957 return self.vfs.exists(path)
17744 09d5b2055295 store: add a contains method to basicstore smuralid parents: 17738 diff changeset	958
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	959
6898 69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	960 class encodedstore(basicstore):
17651 3b49c28658f6 store: rename "openertype" argument to "vfstype" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17649 diff changeset	961 def __init__(self, path, vfstype):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	962 vfs = vfstype(path + b'/store')
17724 bf4b72d8dd4d store: initialize vfs field first to use it for initialization of others FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17722 diff changeset	963 self.path = vfs.base
17726 7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17725 diff changeset	964 self.createmode = _calcmode(vfs)
17652 2c6f7231becc store: rename "op" variables to "vfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17651 diff changeset	965 vfs.createmode = self.createmode
17728 004bd533880d store: invoke "os.path.isdir()" via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17727 diff changeset	966 self.rawvfs = vfs
31244 9b7a2ef4f27c vfs: use 'vfs' module directly in 'mercurial.store' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31229 diff changeset	967 self.vfs = vfsmod.filtervfs(vfs, encodefilename)
17653 dacb50696b75 store: initialize "vfs" fields by "vfs" constructors FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17652 diff changeset	968 self.opener = self.vfs
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	969
50527 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	970 def _walk(self, relpath, recurse, undecodable=None):
816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	971 old = super()._walk(relpath, recurse)
816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	972 new = []
816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	973 for f1, value in old:
6892 dab95717058d verify: check repo.store Adrian Buehlmann <adrian@cadifra.com> parents: 6890 diff changeset	974 try:
47918 2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	975 f2 = decodefilename(f1)
6892 dab95717058d verify: check repo.store Adrian Buehlmann <adrian@cadifra.com> parents: 6890 diff changeset	976 except KeyError:
47918 2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	977 if undecodable is None:
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	978 msg = _(b'undecodable revlog name %s') % f1
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	979 raise error.StorageError(msg)
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	980 else:
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	981 undecodable.append(f1)
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	982 continue
50527 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	983 new.append((f2, value))
816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	984 return new
816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	985
50538 862e3a13da44 store: rename `datafiles` to `data_entries` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50532 diff changeset	986 def data_entries(
50527 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	987 self, matcher=None, undecodable=None
816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	988 ) -> Generator[BaseStoreEntry, None, None]:
50538 862e3a13da44 store: rename `datafiles` to `data_entries` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50532 diff changeset	989 entries = super(encodedstore, self).data_entries(
862e3a13da44 store: rename `datafiles` to `data_entries` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50532 diff changeset	990 undecodable=undecodable
862e3a13da44 store: rename `datafiles` to `data_entries` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50532 diff changeset	991 )
50527 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	992 for entry in entries:
50528 b4953fad744e store: do the revlog matching on entry directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50527 diff changeset	993 if _match_tracked_entry(entry, matcher):
50527 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	994 yield entry
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	995
80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	996 def join(self, f):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	997 return self.path + b'/' + encodefilename(f)
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	998
6903 0642d9d7ec80 clone: get a list of files to clone from store Matt Mackall <mpm@selenic.com> parents: 6902 diff changeset	999 def copylist(self):
45372 909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1000 return [b'requires', b'00changelog.i'] + [b'store/' + f for f in _data]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1001
6903 0642d9d7ec80 clone: get a list of files to clone from store Matt Mackall <mpm@selenic.com> parents: 6902 diff changeset	1002
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49032 diff changeset	1003 class fncache:
8531 810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	1004 # the filename used to be partially encoded
810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	1005 # hence the encodedir/decodedir dance
17722 3b976051034d store: rename field name from "opener" to "vfs" in internal classes for fncache FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17721 diff changeset	1006 def __init__(self, vfs):
3b976051034d store: rename field name from "opener" to "vfs" in internal classes for fncache FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17721 diff changeset	1007 self.vfs = vfs
50315 cf6e1d535602 fncache: make it possible to ignore some file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49314 diff changeset	1008 self._ignores = set()
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1009 self.entries = None
13391 d00bbff8600e fncachestore: defer updating the fncache file to a single file open Adrian Buehlmann <adrian@cadifra.com> parents: 13169 diff changeset	1010 self._dirty = False
40770 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1011 # set of new additions to fncache
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1012 self.addls = set()
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1013
52299 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1014 @property
ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1015 def is_loaded(self):
ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1016 return self.entries is not None
ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1017
42664 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1018 def ensureloaded(self, warn=None):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	1019 """read the fncache file if not already read.
42664 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1020
f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1021 If the file on disk is corrupted, raise. If warn is provided,
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	1022 warn and keep going instead."""
52299 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1023 if not self.is_loaded:
42664 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1024 self._load(warn)
f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1025
f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1026 def _load(self, warn=None):
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1027 '''fill the entries from the fncache file'''
13391 d00bbff8600e fncachestore: defer updating the fncache file to a single file open Adrian Buehlmann <adrian@cadifra.com> parents: 13169 diff changeset	1028 self._dirty = False
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1029 try:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1030 fp = self.vfs(b'fncache', mode=b'rb')
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1031 except IOError:
03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1032 # skip nonexistent file
16404 9fca5b056c0a store: speed up read and write of large fncache files Bryan O'Sullivan <bryano@fb.com> parents: 15742 diff changeset	1033 self.entries = set()
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1034 return
41980 a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1035
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1036 self.entries = set()
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1037 chunk = b''
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1038 for c in iter(functools.partial(fp.read, fncache_chunksize), b''):
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1039 chunk += c
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1040 try:
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1041 p = chunk.rindex(b'\n')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1042 self.entries.update(decodedir(chunk[: p + 1]).splitlines())
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1043 chunk = chunk[p + 1 :]
41980 a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1044 except ValueError:
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1045 # substring '\n' not found, maybe the entry is bigger than the
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1046 # chunksize, so let's keep iterating
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1047 pass
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1048
41983 a920a9e1795a store: error out if fncache does not ends with a newline Pulkit Goyal <pulkit@yandex-team.ru> parents: 41980 diff changeset	1049 if chunk:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1050 msg = _(b"fncache does not ends with a newline")
42664 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1051 if warn:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1052 warn(msg + b'\n')
42664 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1053 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1054 raise error.Abort(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1055 msg,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1056 hint=_(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1057 b"use 'hg debugrebuildfncache' to "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1058 b"rebuild the fncache"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1059 ),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1060 )
42664 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1061 self._checkentries(fp, warn)
41978 d7ef84e595f8 store: move logic to check for invalid entry in fncache to own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41096 diff changeset	1062 fp.close()
d7ef84e595f8 store: move logic to check for invalid entry in fncache to own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41096 diff changeset	1063
42664 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1064 def _checkentries(self, fp, warn):
47031 f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1 Kyle Lippincott <spectral@google.com> parents: 46990 diff changeset	1065 """make sure there is no empty string in entries"""
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1066 if b'' in self.entries:
16404 9fca5b056c0a store: speed up read and write of large fncache files Bryan O'Sullivan <bryano@fb.com> parents: 15742 diff changeset	1067 fp.seek(0)
49032 fd5b8e696b75 py3: stop using util.iterfile() Gregory Szorc <gregory.szorc@gmail.com> parents: 49004 diff changeset	1068 for n, line in enumerate(fp):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1069 if not line.rstrip(b'\n'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1070 t = _(b'invalid entry in fncache, line %d') % (n + 1)
42664 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1071 if warn:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1072 warn(t + b'\n')
42664 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1073 else:
f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1074 raise error.Abort(t)
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1075
20883 cd443c7589cc fncache: move fncache writing to be in a transaction Durham Goode <durham@fb.com> parents: 20879 diff changeset	1076 def write(self, tr):
16404 9fca5b056c0a store: speed up read and write of large fncache files Bryan O'Sullivan <bryano@fb.com> parents: 15742 diff changeset	1077 if self._dirty:
52299 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1078 assert self.is_loaded
40782 df8ed31a8ad8 store: write fncache only once if there are both adds and removes Pulkit Goyal <pulkit@yandex-team.ru> parents: 40770 diff changeset	1079 self.entries = self.entries \| self.addls
df8ed31a8ad8 store: write fncache only once if there are both adds and removes Pulkit Goyal <pulkit@yandex-team.ru> parents: 40770 diff changeset	1080 self.addls = set()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1081 tr.addbackup(b'fncache')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1082 fp = self.vfs(b'fncache', mode=b'wb', atomictemp=True)
20879 cd03854a2e06 fncache: remove the rewriting logic Durham Goode <durham@fb.com> parents: 19903 diff changeset	1083 if self.entries:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1084 fp.write(encodedir(b'\n'.join(self.entries) + b'\n'))
20879 cd03854a2e06 fncache: remove the rewriting logic Durham Goode <durham@fb.com> parents: 19903 diff changeset	1085 fp.close()
cd03854a2e06 fncache: remove the rewriting logic Durham Goode <durham@fb.com> parents: 19903 diff changeset	1086 self._dirty = False
40770 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1087 if self.addls:
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1088 # if we have just new entries, let's append them to the fncache
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1089 tr.addbackup(b'fncache')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1090 fp = self.vfs(b'fncache', mode=b'ab', atomictemp=True)
40770 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1091 if self.addls:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1092 fp.write(encodedir(b'\n'.join(self.addls) + b'\n'))
40770 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1093 fp.close()
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1094 self.entries = None
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1095 self.addls = set()
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1096
50315 cf6e1d535602 fncache: make it possible to ignore some file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49314 diff changeset	1097 def addignore(self, fn):
cf6e1d535602 fncache: make it possible to ignore some file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49314 diff changeset	1098 self._ignores.add(fn)
cf6e1d535602 fncache: make it possible to ignore some file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49314 diff changeset	1099
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1100 def add(self, fn):
50315 cf6e1d535602 fncache: make it possible to ignore some file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49314 diff changeset	1101 if fn in self._ignores:
cf6e1d535602 fncache: make it possible to ignore some file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49314 diff changeset	1102 return
52299 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1103 if not self.is_loaded:
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1104 self._load()
10577 d5bd1beff794 store: only add new entries to the fncache file Adrian Buehlmann <adrian@cadifra.com> parents: 10339 diff changeset	1105 if fn not in self.entries:
40770 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1106 self.addls.add(fn)
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1107
20885 f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1108 def remove(self, fn):
52299 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1109 if not self.is_loaded:
20885 f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1110 self._load()
40770 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1111 if fn in self.addls:
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1112 self.addls.remove(fn)
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1113 return
20885 f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1114 try:
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1115 self.entries.remove(fn)
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1116 self._dirty = True
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1117 except KeyError:
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1118 pass
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1119
17782 8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1120 def __contains__(self, fn):
40770 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1121 if fn in self.addls:
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1122 return True
52299 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1123 if not self.is_loaded:
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1124 self._load()
17782 8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1125 return fn in self.entries
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1126
03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1127 def __iter__(self):
52299 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1128 if not self.is_loaded:
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1129 self._load()
40770 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1130 return iter(self.entries \| self.addls)
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1131
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1132
41096 6498f0e03526 vfs: fix proxyvfs inheritance Boris Feld <boris.feld@octobus.net> parents: 40782 diff changeset	1133 class _fncachevfs(vfsmod.proxyvfs):
17721 cf236e3501c3 store: rename argument name from "op"(ener) to "vfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17693 diff changeset	1134 def __init__(self, vfs, fnc, encode):
33412 a42369e04aee vfs: rename auditvfs to proxyvfs Yuya Nishihara <yuya@tcha.org> parents: 32411 diff changeset	1135 vfsmod.proxyvfs.__init__(self, vfs)
52300 9fbdf355c916 fncachestore: add typing information Rapha?l Gom?s <rgomes@octobus.net> parents: 52299 diff changeset	1136 self.fncache: fncache = fnc
14194 3a90fb3addc1 store: break up reference cycle introduced in 9cbff8a39a2a Adrian Buehlmann <adrian@cadifra.com> parents: 14166 diff changeset	1137 self.encode = encode
52301 bd43465af568 fncache: add attribute to check whether we're using dotencode Rapha?l Gom?s <rgomes@octobus.net> parents: 52300 diff changeset	1138 self.uses_dotencode = encode is _pathencode
14194 3a90fb3addc1 store: break up reference cycle introduced in 9cbff8a39a2a Adrian Buehlmann <adrian@cadifra.com> parents: 14166 diff changeset	1139
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1140 def __call__(self, path, mode=b'r', args, *kw):
38661 8ac0c9cd4c48 fncache: avoid loading the filename cache when not actually modifying it Martijn Pieters <mj@zopatista.com> parents: 37409 diff changeset	1141 encoded = self.encode(path)
50501 0dbab42adca5 store: do not record file that are not part of a revlog in fncache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50485 diff changeset	1142 if (
0dbab42adca5 store: do not record file that are not part of a revlog in fncache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50485 diff changeset	1143 mode not in (b'r', b'rb')
0dbab42adca5 store: do not record file that are not part of a revlog in fncache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50485 diff changeset	1144 and (path.startswith(b'data/') or path.startswith(b'meta/'))
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	1145 and is_revlog_file(path)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1146 ):
38661 8ac0c9cd4c48 fncache: avoid loading the filename cache when not actually modifying it Martijn Pieters <mj@zopatista.com> parents: 37409 diff changeset	1147 # do not trigger a fncache load when adding a file that already is
8ac0c9cd4c48 fncache: avoid loading the filename cache when not actually modifying it Martijn Pieters <mj@zopatista.com> parents: 37409 diff changeset	1148 # known to exist.
52299 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1149 notload = not self.fncache.is_loaded and (
50759 30381812708d fncache: fix a bug that corrupts the fncache after transaction rollback Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50706 diff changeset	1150 # if the file has size zero, it should be considered as missing.
30381812708d fncache: fix a bug that corrupts the fncache after transaction rollback Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50706 diff changeset	1151 # Such zero-size files are the result of truncation when a
30381812708d fncache: fix a bug that corrupts the fncache after transaction rollback Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50706 diff changeset	1152 # transaction is aborted.
30381812708d fncache: fix a bug that corrupts the fncache after transaction rollback Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50706 diff changeset	1153 self.vfs.exists(encoded)
30381812708d fncache: fix a bug that corrupts the fncache after transaction rollback Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50706 diff changeset	1154 and self.vfs.stat(encoded).st_size
30381812708d fncache: fix a bug that corrupts the fncache after transaction rollback Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50706 diff changeset	1155 )
38661 8ac0c9cd4c48 fncache: avoid loading the filename cache when not actually modifying it Martijn Pieters <mj@zopatista.com> parents: 37409 diff changeset	1156 if not notload:
8ac0c9cd4c48 fncache: avoid loading the filename cache when not actually modifying it Martijn Pieters <mj@zopatista.com> parents: 37409 diff changeset	1157 self.fncache.add(path)
8ac0c9cd4c48 fncache: avoid loading the filename cache when not actually modifying it Martijn Pieters <mj@zopatista.com> parents: 37409 diff changeset	1158 return self.vfs(encoded, mode, args, *kw)
14194 3a90fb3addc1 store: break up reference cycle introduced in 9cbff8a39a2a Adrian Buehlmann <adrian@cadifra.com> parents: 14166 diff changeset	1159
51919 2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	1160 def join(self, path: Optional[bytes], *insidef: bytes) -> bytes:
2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	1161 insidef = (self.encode(f) for f in insidef)
2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	1162
17725 ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17724 diff changeset	1163 if path:
51919 2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	1164 return self.vfs.join(self.encode(path), *insidef)
17725 ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17724 diff changeset	1165 else:
51919 2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	1166 return self.vfs.join(path, *insidef)
17725 ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17724 diff changeset	1167
47449 9ab54aa56982 vfs: add a `register_file` method on the vfs class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	1168 def register_file(self, path):
9ab54aa56982 vfs: add a `register_file` method on the vfs class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	1169 """generic hook point to lets fncache steer its stew"""
9ab54aa56982 vfs: add a `register_file` method on the vfs class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	1170 if path.startswith(b'data/') or path.startswith(b'meta/'):
9ab54aa56982 vfs: add a `register_file` method on the vfs class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	1171 self.fncache.add(path)
9ab54aa56982 vfs: add a `register_file` method on the vfs class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	1172
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1173
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1174 class fncachestore(basicstore):
17651 3b49c28658f6 store: rename "openertype" argument to "vfstype" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17649 diff changeset	1175 def __init__(self, path, vfstype, dotencode):
17591 9a5c2ecd1158 store: move encode lambda logic into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17590 diff changeset	1176 if dotencode:
18435 8c019d2fd7c0 store: switch to C-based hashed path encoding Bryan O'Sullivan <bryano@fb.com> parents: 18430 diff changeset	1177 encode = _pathencode
17591 9a5c2ecd1158 store: move encode lambda logic into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17590 diff changeset	1178 else:
9a5c2ecd1158 store: move encode lambda logic into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17590 diff changeset	1179 encode = _plainhybridencode
12687 34d8247a4595 store: encode first period or space in filenames (issue1713) Adrian Buehlmann <adrian@cadifra.com> parents: 12171 diff changeset	1180 self.encode = encode
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1181 vfs = vfstype(path + b'/store')
17724 bf4b72d8dd4d store: initialize vfs field first to use it for initialization of others FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17722 diff changeset	1182 self.path = vfs.base
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1183 self.pathsep = self.path + b'/'
17726 7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17725 diff changeset	1184 self.createmode = _calcmode(vfs)
17652 2c6f7231becc store: rename "op" variables to "vfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17651 diff changeset	1185 vfs.createmode = self.createmode
17727 6492b39a44d5 store: replace invocation of "getsize()" by "vfs.stat()" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17726 diff changeset	1186 self.rawvfs = vfs
17652 2c6f7231becc store: rename "op" variables to "vfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17651 diff changeset	1187 fnc = fncache(vfs)
9133 996c1cd8f530 store: eliminate reference cycle in fncachestore Simon Heimberg <simohe@besonet.ch> parents: 8778 diff changeset	1188 self.fncache = fnc
17653 dacb50696b75 store: initialize "vfs" fields by "vfs" constructors FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17652 diff changeset	1189 self.vfs = _fncachevfs(vfs, fnc, encode)
dacb50696b75 store: initialize "vfs" fields by "vfs" constructors FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17652 diff changeset	1190 self.opener = self.vfs
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1191
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1192 def join(self, f):
17562 b42b0729744d store: reduce string concatenation when joining Bryan O'Sullivan <bryano@fb.com> parents: 17555 diff changeset	1193 return self.pathsep + self.encode(f)
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1194
17731 c85dbae29684 store: restore getsize method Matt Mackall <mpm@selenic.com> parents: 17728 diff changeset	1195 def getsize(self, path):
c85dbae29684 store: restore getsize method Matt Mackall <mpm@selenic.com> parents: 17728 diff changeset	1196 return self.rawvfs.stat(path).st_size
c85dbae29684 store: restore getsize method Matt Mackall <mpm@selenic.com> parents: 17728 diff changeset	1197
50538 862e3a13da44 store: rename `datafiles` to `data_entries` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50532 diff changeset	1198 def data_entries(
50505 521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50503 diff changeset	1199 self, matcher=None, undecodable=None
50507 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	1200 ) -> Generator[BaseStoreEntry, None, None]:
50514 d4f54aded22e store: also group files by revlog in fncache version of datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50513 diff changeset	1201 # Note: all files in fncache should be revlog related, However the
d4f54aded22e store: also group files by revlog in fncache version of datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50513 diff changeset	1202 # fncache might contains such file added by previous version of
d4f54aded22e store: also group files by revlog in fncache version of datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50513 diff changeset	1203 # Mercurial.
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	1204 files = ((f, None) for f in self.fncache if is_revlog_file(f))
50514 d4f54aded22e store: also group files by revlog in fncache version of datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50513 diff changeset	1205 by_revlog = _gather_revlog(files)
d4f54aded22e store: also group files by revlog in fncache version of datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50513 diff changeset	1206 for revlog, details in by_revlog:
50516 1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50515 diff changeset	1207 if revlog.startswith(b'data/'):
50688 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	1208 rl_type = KIND_FILELOG
50517 60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50516 diff changeset	1209 revlog_target_id = revlog.split(b'/', 1)[1]
50516 1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50515 diff changeset	1210 elif revlog.startswith(b'meta/'):
50688 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	1211 rl_type = KIND_MANIFESTLOG
50517 60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50516 diff changeset	1212 # drop the initial directory and the `00manifest` file part
60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50516 diff changeset	1213 tmp = revlog.split(b'/', 1)[1]
60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50516 diff changeset	1214 revlog_target_id = tmp.rsplit(b'/', 1)[0] + b'/'
50516 1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50515 diff changeset	1215 else:
1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50515 diff changeset	1216 # unreachable
1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50515 diff changeset	1217 assert False, revlog
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	1218 entry = RevlogStoreEntry(
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	1219 path_prefix=revlog,
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	1220 revlog_type=rl_type,
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	1221 target_id=revlog_target_id,
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	1222 details=details,
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	1223 )
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	1224 if _match_tracked_entry(entry, matcher):
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	1225 yield entry
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1226
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1227 def copylist(self):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1228 d = (
45372 909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1229 b'bookmarks',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1230 b'narrowspec',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1231 b'data',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1232 b'meta',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1233 b'dh',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1234 b'fncache',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1235 b'phaseroots',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1236 b'obsstore',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1237 b'00manifest.d',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1238 b'00manifest.i',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1239 b'00changelog.d',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1240 b'00changelog.i',
45494 d252f51ab032 share: introduce config option to store requires in .hg/store Pulkit Goyal <7895pulkit@gmail.com> parents: 45372 diff changeset	1241 b'requires',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1242 )
45372 909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1243 return [b'requires', b'00changelog.i'] + [b'store/' + f for f in d]
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1244
20883 cd443c7589cc fncache: move fncache writing to be in a transaction Durham Goode <durham@fb.com> parents: 20879 diff changeset	1245 def write(self, tr):
cd443c7589cc fncache: move fncache writing to be in a transaction Durham Goode <durham@fb.com> parents: 20879 diff changeset	1246 self.fncache.write(tr)
13391 d00bbff8600e fncachestore: defer updating the fncache file to a single file open Adrian Buehlmann <adrian@cadifra.com> parents: 13169 diff changeset	1247
20884 2efdd186925d caches: invalidate store caches when lock is taken Durham Goode <durham@fb.com> parents: 20883 diff changeset	1248 def invalidatecaches(self):
2efdd186925d caches: invalidate store caches when lock is taken Durham Goode <durham@fb.com> parents: 20883 diff changeset	1249 self.fncache.entries = None
40770 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1250 self.fncache.addls = set()
20884 2efdd186925d caches: invalidate store caches when lock is taken Durham Goode <durham@fb.com> parents: 20883 diff changeset	1251
20885 f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1252 def markremoved(self, fn):
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1253 self.fncache.remove(fn)
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1254
17783 df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1255 def _exists(self, f):
df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1256 ef = self.encode(f)
df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1257 try:
df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1258 self.getsize(ef)
df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1259 return True
49314 2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT Manuel Jacob <me@manueljacob.de> parents: 49292 diff changeset	1260 except FileNotFoundError:
17783 df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1261 return False
df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1262
17745 b9a56b816ff2 store: add a contains method to fncachestore smuralid parents: 17744 diff changeset	1263 def __contains__(self, path):
b9a56b816ff2 store: add a contains method to fncachestore smuralid parents: 17744 diff changeset	1264 '''Checks if the store contains path'''
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1265 path = b"/".join((b"data", path))
17782 8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1266 # check for files (exact match)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1267 e = path + b'.i'
17784 73e1ab39792c store: fncache may contain non-existent entries (fixes b9a56b816ff2) Adrian Buehlmann <adrian@cadifra.com> parents: 17783 diff changeset	1268 if e in self.fncache and self._exists(e):
17782 8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1269 return True
8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1270 # now check for directories (prefix match)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1271 if not path.endswith(b'/'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1272 path += b'/'
17782 8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1273 for e in self.fncache:
17784 73e1ab39792c store: fncache may contain non-existent entries (fixes b9a56b816ff2) Adrian Buehlmann <adrian@cadifra.com> parents: 17783 diff changeset	1274 if e.startswith(path) and self._exists(e):
17782 8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1275 return True
8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1276 return False

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/store.py @ 52391:3f0cf7bb3086