mercurial-scm/hg-stable: mercurial/store.py annotate

annotate mercurial/store.py @ 52710:45dc0f874b8c

typing: lock in the type annotations that were lost with the pyupgrade changes For some reason, these reverted from a specific type to something less useful after the changes that culminated in 70a75d379daf. (e.g. the `remotefilectx` method went from `Generator[remotefilectx, Any, None]` to `Generator[nothing, Any, None]`.) The previous typing for `merge.filemap` was `Generator[Tuple[Any, Tuple[Any, Any, Any]], Any, None]`, and decayed to `Generator[nothing, Any, None]`. I don't feel like unravelling the specific types here, so restore the equivalent of that.

author	Matt Harbison <matt_harbison@yahoo.com>
date	Sun, 12 Jan 2025 21:35:30 -0500
parents	4cb75772818d
children

rev	line source
50688 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	1 # store.py - repository store handling for Mercurial)
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	2 #
46819 d4ba4d51f85f contributor: change mentions of mpm to olivia Rapha?l Gom?s <rgomes@octobus.net> parents: 46793 diff changeset	3 # Copyright 2008 Olivia Mackall <olivia@selenic.com>
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	4 #
8225 46293a0c7e9f updated license to be explicit about GPL version 2 Martin Geisler <mg@lazybytes.net> parents: 8210 diff changeset	5 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 9133 diff changeset	6 # GNU General Public License version 2 or any later version.
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	7
51901 f4733654f144 typing: add `from __future__ import annotations` to most files Matt Harbison <matt_harbison@yahoo.com> parents: 51868 diff changeset	8 from __future__ import annotations
f4733654f144 typing: add `from __future__ import annotations` to most files Matt Harbison <matt_harbison@yahoo.com> parents: 51868 diff changeset	9
50513 5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	10 import collections
41980 a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	11 import functools
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	12 import os
46990 0b569c75d180 store: exclude `undo.` nodemap's file from `walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46989 diff changeset	13 import re
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	14 import stat
51788 278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51787 diff changeset	15 import typing
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51787 diff changeset	16
51919 2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	17 from typing import (
2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	18 Generator,
52710 45dc0f874b8c typing: lock in the type annotations that were lost with the pyupgrade changes Matt Harbison <matt_harbison@yahoo.com> parents: 52670 diff changeset	19 Iterator,
51919 2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	20 List,
2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	21 Optional,
2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	22 )
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	23
509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	24 from .i18n import _
50505 521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50503 diff changeset	25 from .thirdparty import attr
51788 278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51787 diff changeset	26
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51787 diff changeset	27 # Force pytype to use the non-vendored package
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51787 diff changeset	28 if typing.TYPE_CHECKING:
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51787 diff changeset	29 # noinspection PyPackageRequirements
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51787 diff changeset	30 import attr
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51787 diff changeset	31
46114 59fa3890d40a node: import symbols explicitly Joerg Sonnenberger <joerg@bec.de> parents: 45957 diff changeset	32 from .node import hex
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	33 from .revlogutils.constants import (
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	34 INDEX_HEADER,
50688 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	35 KIND_CHANGELOG,
e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	36 KIND_FILELOG,
e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	37 KIND_MANIFESTLOG,
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	38 )
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	39 from . import (
42911 3df3b139a43d localrepo: push manifestlog and changelog construction code into store Augie Fackler <augie@google.com> parents: 42664 diff changeset	40 changelog,
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	41 error,
50657 e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	42 filelog,
42911 3df3b139a43d localrepo: push manifestlog and changelog construction code into store Augie Fackler <augie@google.com> parents: 42664 diff changeset	43 manifest,
32411 df448de7cf3b parsers: switch to policy importer Yuya Nishihara <yuya@tcha.org> parents: 31371 diff changeset	44 policy,
30077 8f42d8c412c8 py3: make encodefun in store.py compatible with py3k Mateusz Kwapich <mitrandir@fb.com> parents: 30076 diff changeset	45 pycompat,
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	46 revlog as revlogmod,
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	47 util,
31244 9b7a2ef4f27c vfs: use 'vfs' module directly in 'mercurial.store' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31229 diff changeset	48 vfs as vfsmod,
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	49 )
44060 a61287a95dc3 core: migrate uses of hashlib.sha1 to hashutil.sha1 Augie Fackler <augie@google.com> parents: 43554 diff changeset	50 from .utils import hashutil
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	51
43554 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43106 diff changeset	52 parsers = policy.importmod('parsers')
41980 a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	53 # how much bytes should be read from fncache in one read
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	54 # It is done to prevent loading large fncache files into memory
51762 ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51759 diff changeset	55 fncache_chunksize = 10**6
32411 df448de7cf3b parsers: switch to policy importer Yuya Nishihara <yuya@tcha.org> parents: 31371 diff changeset	56
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	57
52670 4cb75772818d pyupgrade: drop the quoting around type annotations Matt Harbison <matt_harbison@yahoo.com> parents: 52669 diff changeset	58 def _match_tracked_entry(entry: BaseStoreEntry, matcher):
40544 9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	59 """parses a fncache entry and returns whether the entry is tracking a path
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	60 matched by matcher or not.
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	61
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	62 If matcher is None, returns True"""
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	63
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	64 if matcher is None:
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	65 return True
51787 2e9e62242451 typing: disable some pytype errors in `mercurial.store` Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	66
2e9e62242451 typing: disable some pytype errors in `mercurial.store` Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	67 # TODO: make this safe for other entry types. Currently, the various
2e9e62242451 typing: disable some pytype errors in `mercurial.store` Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	68 # store.data_entry generators only yield RevlogStoreEntry, so the
2e9e62242451 typing: disable some pytype errors in `mercurial.store` Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	69 # attributes do exist on `entry`.
2e9e62242451 typing: disable some pytype errors in `mercurial.store` Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	70 # pytype: disable=attribute-error
50532 1b776f25302f store: use the boolean property in `store` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50531 diff changeset	71 if entry.is_filelog:
50528 b4953fad744e store: do the revlog matching on entry directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50527 diff changeset	72 return matcher(entry.target_id)
50532 1b776f25302f store: use the boolean property in `store` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50531 diff changeset	73 elif entry.is_manifestlog:
50528 b4953fad744e store: do the revlog matching on entry directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50527 diff changeset	74 return matcher.visitdir(entry.target_id.rstrip(b'/'))
51787 2e9e62242451 typing: disable some pytype errors in `mercurial.store` Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	75 # pytype: enable=attribute-error
50528 b4953fad744e store: do the revlog matching on entry directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50527 diff changeset	76 raise error.ProgrammingError(b"cannot process entry %r" % entry)
40668 66adfd58cb77 store: raise ProgrammingError if unable to decode a storage path Pulkit Goyal <pulkit@yandex-team.ru> parents: 40631 diff changeset	77
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	78
8531 810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	79 # This avoids a collision between a file named foo and a dir named
810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	80 # foo.i or foo.d
17607 cc58dc47cb5e store: use fast C implementation of encodedir() if it's available Adrian Buehlmann <adrian@cadifra.com> parents: 17605 diff changeset	81 def _encodedir(path):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	82 """
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	83 >>> _encodedir(b'data/foo.i')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	84 'data/foo.i'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	85 >>> _encodedir(b'data/foo.i/bla.i')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	86 'data/foo.i.hg/bla.i'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	87 >>> _encodedir(b'data/foo.i.hg/bla.i')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	88 'data/foo.i.hg.hg/bla.i'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	89 >>> _encodedir(b'data/foo.i\\ndata/foo.i/bla.i\\ndata/foo.i.hg/bla.i\\n')
17605 e9cc29be3305 store: add multiline doctest case for encodedir() Adrian Buehlmann <adrian@cadifra.com> parents: 17604 diff changeset	90 'data/foo.i\\ndata/foo.i.hg/bla.i\\ndata/foo.i.hg.hg/bla.i\\n'
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	91 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	92 return (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	93 path.replace(b".hg/", b".hg.hg/")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	94 .replace(b".i/", b".i.hg/")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	95 .replace(b".d/", b".d.hg/")
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	96 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	97
8531 810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	98
17607 cc58dc47cb5e store: use fast C implementation of encodedir() if it's available Adrian Buehlmann <adrian@cadifra.com> parents: 17605 diff changeset	99 encodedir = getattr(parsers, 'encodedir', _encodedir)
cc58dc47cb5e store: use fast C implementation of encodedir() if it's available Adrian Buehlmann <adrian@cadifra.com> parents: 17605 diff changeset	100
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	101
8531 810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	102 def decodedir(path):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	103 """
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	104 >>> decodedir(b'data/foo.i')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	105 'data/foo.i'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	106 >>> decodedir(b'data/foo.i.hg/bla.i')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	107 'data/foo.i/bla.i'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	108 >>> decodedir(b'data/foo.i.hg.hg/bla.i')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	109 'data/foo.i.hg/bla.i'
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	110 """
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	111 if b".hg/" not in path:
8531 810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	112 return path
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	113 return (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	114 path.replace(b".d.hg/", b".d/")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	115 .replace(b".i.hg/", b".i/")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	116 .replace(b".hg.hg/", b".hg/")
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	117 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	118
8531 810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	119
52710 45dc0f874b8c typing: lock in the type annotations that were lost with the pyupgrade changes Matt Harbison <matt_harbison@yahoo.com> parents: 52670 diff changeset	120 def _reserved() -> Iterator[int]:
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	121 """characters that are problematic for filesystems
29071 2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	122
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	123 * ascii escapes (0..31)
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	124 * ascii hi (126..255)
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	125 * windows specials
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	126
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	127 these characters will be escaped by encodefunctions
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	128 """
52664 9db77d46de79 py3: drop redundant `u''` prefixes on string literals Matt Harbison <matt_harbison@yahoo.com> parents: 52391 diff changeset	129 winreserved = [ord(x) for x in '\\:*?"<>\|']
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52668 diff changeset	130 yield from range(32)
e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52668 diff changeset	131 yield from range(126, 256)
e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52668 diff changeset	132 yield from winreserved
29071 2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	133
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	134
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	135 def _buildencodefun():
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	136 """
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	137 >>> enc, dec = _buildencodefun()
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	138
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	139 >>> enc(b'nothing/special.txt')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	140 'nothing/special.txt'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	141 >>> dec(b'nothing/special.txt')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	142 'nothing/special.txt'
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	143
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	144 >>> enc(b'HELLO')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	145 '_h_e_l_l_o'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	146 >>> dec(b'_h_e_l_l_o')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	147 'HELLO'
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	148
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	149 >>> enc(b'hello:world?')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	150 'hello~3aworld~3f'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	151 >>> dec(b'hello~3aworld~3f')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	152 'hello:world?'
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	153
34151 414a3513c2bd doctest: do not embed non-ascii characters in docstring Yuya Nishihara <yuya@tcha.org> parents: 34146 diff changeset	154 >>> enc(b'the\\x07quick\\xADshot')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	155 'the~07quick~adshot'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	156 >>> dec(b'the~07quick~adshot')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	157 'the\\x07quick\\xadshot'
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	158 """
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	159 e = b'_'
31263 64596338ba10 py3: factor out bytechr() function Yuya Nishihara <yuya@tcha.org> parents: 31244 diff changeset	160 xchr = pycompat.bytechr
64596338ba10 py3: factor out bytechr() function Yuya Nishihara <yuya@tcha.org> parents: 31244 diff changeset	161 asciistr = list(map(xchr, range(127)))
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	162 capitals = list(range(ord(b"A"), ord(b"Z") + 1))
30077 8f42d8c412c8 py3: make encodefun in store.py compatible with py3k Mateusz Kwapich <mitrandir@fb.com> parents: 30076 diff changeset	163
44470 9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things Augie Fackler <augie@google.com> parents: 44060 diff changeset	164 cmap = {x: x for x in asciistr}
29071 2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	165 for x in _reserved():
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	166 cmap[xchr(x)] = b"~%02x" % x
30077 8f42d8c412c8 py3: make encodefun in store.py compatible with py3k Mateusz Kwapich <mitrandir@fb.com> parents: 30076 diff changeset	167 for x in capitals + [ord(e)]:
8f42d8c412c8 py3: make encodefun in store.py compatible with py3k Mateusz Kwapich <mitrandir@fb.com> parents: 30076 diff changeset	168 cmap[xchr(x)] = e + xchr(x).lower()
8f42d8c412c8 py3: make encodefun in store.py compatible with py3k Mateusz Kwapich <mitrandir@fb.com> parents: 30076 diff changeset	169
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	170 dmap = {}
49004 f254fc73d956 global: bulk replace simple pycompat.iteritems(x) with x.items() Gregory Szorc <gregory.szorc@gmail.com> parents: 48966 diff changeset	171 for k, v in cmap.items():
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	172 dmap[v] = k
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	173
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	174 def decode(s):
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	175 i = 0
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	176 while i < len(s):
49292 d44e3c45f0e4 py3: replace `pycompat.xrange` by `range` Manuel Jacob <me@manueljacob.de> parents: 49037 diff changeset	177 for l in range(1, 4):
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	178 try:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	179 yield dmap[s[i : i + l]]
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	180 i += l
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	181 break
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	182 except KeyError:
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	183 pass
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	184 else:
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	185 raise KeyError
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	186
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	187 return (
49292 d44e3c45f0e4 py3: replace `pycompat.xrange` by `range` Manuel Jacob <me@manueljacob.de> parents: 49037 diff changeset	188 lambda s: b''.join([cmap[s[c : c + 1]] for c in range(len(s))]),
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	189 lambda s: b''.join(list(decode(s))),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	190 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	191
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	192
776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	193 _encodefname, _decodefname = _buildencodefun()
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	194
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	195
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	196 def encodefilename(s):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	197 """
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	198 >>> encodefilename(b'foo.i/bar.d/bla.hg/hi:world?/HELLO')
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	199 'foo.i.hg/bar.d.hg/bla.hg.hg/hi~3aworld~3f/_h_e_l_l_o'
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	200 """
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	201 return _encodefname(encodedir(s))
776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	202
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	203
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	204 def decodefilename(s):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	205 """
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	206 >>> decodefilename(b'foo.i.hg/bar.d.hg/bla.hg.hg/hi~3aworld~3f/_h_e_l_l_o')
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	207 'foo.i/bar.d/bla.hg/hi:world?/HELLO'
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	208 """
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	209 return decodedir(_decodefname(s))
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	210
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	211
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	212 def _buildlowerencodefun():
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	213 """
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	214 >>> f = _buildlowerencodefun()
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	215 >>> f(b'nothing/special.txt')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	216 'nothing/special.txt'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	217 >>> f(b'HELLO')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	218 'hello'
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	219 >>> f(b'hello:world?')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	220 'hello~3aworld~3f'
34151 414a3513c2bd doctest: do not embed non-ascii characters in docstring Yuya Nishihara <yuya@tcha.org> parents: 34146 diff changeset	221 >>> f(b'the\\x07quick\\xADshot')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	222 'the~07quick~adshot'
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	223 """
34221 7e3f078b6f31 py3: use bytechr() in store._buildlowerencodefun() Yuya Nishihara <yuya@tcha.org> parents: 34220 diff changeset	224 xchr = pycompat.bytechr
49292 d44e3c45f0e4 py3: replace `pycompat.xrange` by `range` Manuel Jacob <me@manueljacob.de> parents: 49037 diff changeset	225 cmap = {xchr(x): xchr(x) for x in range(127)}
29071 2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	226 for x in _reserved():
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	227 cmap[xchr(x)] = b"~%02x" % x
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	228 for x in range(ord(b"A"), ord(b"Z") + 1):
34221 7e3f078b6f31 py3: use bytechr() in store._buildlowerencodefun() Yuya Nishihara <yuya@tcha.org> parents: 34220 diff changeset	229 cmap[xchr(x)] = xchr(x).lower()
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	230
34220 96808804b68f store: give name to lowerencode function Yuya Nishihara <yuya@tcha.org> parents: 34151 diff changeset	231 def lowerencode(s):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	232 return b"".join([cmap[c] for c in pycompat.iterbytestr(s)])
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	233
34220 96808804b68f store: give name to lowerencode function Yuya Nishihara <yuya@tcha.org> parents: 34151 diff changeset	234 return lowerencode
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	235
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	236
18430 0459c6555f69 store: implement lowerencode in C Bryan O'Sullivan <bryano@fb.com> parents: 18054 diff changeset	237 lowerencode = getattr(parsers, 'lowerencode', None) or _buildlowerencodefun()
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	238
17570 f53a7b256ca6 store: optimze _auxencode() a bit by grouping the reserved names by length Adrian Buehlmann <adrian@cadifra.com> parents: 17569 diff changeset	239 # Windows reserved names: con, prn, aux, nul, com1..com9, lpt1..lpt9
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	240 _winres3 = (b'aux', b'con', b'prn', b'nul') # length 3
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	241 _winres4 = (b'com', b'lpt') # length 4 (with trailing 1..9)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	242
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	243
12687 34d8247a4595 store: encode first period or space in filenames (issue1713) Adrian Buehlmann <adrian@cadifra.com> parents: 12171 diff changeset	244 def _auxencode(path, dotencode):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	245 """
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	246 Encodes filenames containing names reserved by Windows or which end in
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	247 period or space. Does not touch other single reserved characters c.
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	248 Specifically, c in '\\:?"<>\|' or ord(c) <= 31 are not* encoded here.
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	249 Additionally encodes space or period at the beginning, if dotencode is
17569 e9af2134825c store: explain "aux.foo" versus "foo.aux" in doc of _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17568 diff changeset	250 True. Parameter path is assumed to be all lowercase.
e9af2134825c store: explain "aux.foo" versus "foo.aux" in doc of _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17568 diff changeset	251 A segment only needs encoding if a reserved name appears as a
e9af2134825c store: explain "aux.foo" versus "foo.aux" in doc of _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17568 diff changeset	252 basename (e.g. "aux", "aux.foo"). A directory or file named "foo.aux"
e9af2134825c store: explain "aux.foo" versus "foo.aux" in doc of _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17568 diff changeset	253 doesn't need encoding.
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	254
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	255 >>> s = b'.foo/aux.txt/txt.aux/con/prn/nul/foo.'
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	256 >>> _auxencode(s.split(b'/'), True)
17574 81a033bb29bc store: let _auxencode() return the list of path segments Adrian Buehlmann <adrian@cadifra.com> parents: 17573 diff changeset	257 ['~2efoo', 'au~78.txt', 'txt.aux', 'co~6e', 'pr~6e', 'nu~6c', 'foo~2e']
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	258 >>> s = b'.com1com2/lpt9.lpt4.lpt1/conprn/com0/lpt0/foo.'
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	259 >>> _auxencode(s.split(b'/'), False)
17574 81a033bb29bc store: let _auxencode() return the list of path segments Adrian Buehlmann <adrian@cadifra.com> parents: 17573 diff changeset	260 ['.com1com2', 'lp~749.lpt4.lpt1', 'conprn', 'com0', 'lpt0', 'foo~2e']
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	261 >>> _auxencode([b'foo. '], True)
17574 81a033bb29bc store: let _auxencode() return the list of path segments Adrian Buehlmann <adrian@cadifra.com> parents: 17573 diff changeset	262 ['foo.~20']
34146 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	263 >>> _auxencode([b' .foo'], True)
17574 81a033bb29bc store: let _auxencode() return the list of path segments Adrian Buehlmann <adrian@cadifra.com> parents: 17573 diff changeset	264 ['~20.foo']
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	265 """
17589 b11024849db6 store: parameter path of _auxencode is now a list of strings Adrian Buehlmann <adrian@cadifra.com> parents: 17588 diff changeset	266 for i, n in enumerate(path):
17572 b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	267 if not n:
b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	268 continue
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	269 if dotencode and n[0] in b'. ':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	270 n = b"~%02x" % ord(n[0:1]) + n[1:]
17589 b11024849db6 store: parameter path of _auxencode is now a list of strings Adrian Buehlmann <adrian@cadifra.com> parents: 17588 diff changeset	271 path[i] = n
17572 b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	272 else:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	273 l = n.find(b'.')
17572 b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	274 if l == -1:
b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	275 l = len(n)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	276 if (l == 3 and n[:3] in _winres3) or (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	277 l == 4
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	278 and n[3:4] <= b'9'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	279 and n[3:4] >= b'1'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	280 and n[:3] in _winres4
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	281 ):
17572 b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	282 # encode third letter ('aux' -> 'au~78')
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	283 ec = b"~%02x" % ord(n[2:3])
17572 b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	284 n = n[0:2] + ec + n[3:]
17589 b11024849db6 store: parameter path of _auxencode is now a list of strings Adrian Buehlmann <adrian@cadifra.com> parents: 17588 diff changeset	285 path[i] = n
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	286 if n[-1] in b'. ':
17572 b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	287 # encode last period or space ('foo...' -> 'foo..~2e')
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	288 path[i] = n[:-1] + b"~%02x" % ord(n[-1:])
17589 b11024849db6 store: parameter path of _auxencode is now a list of strings Adrian Buehlmann <adrian@cadifra.com> parents: 17588 diff changeset	289 return path
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	290
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	291
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	292 _maxstorepathlen = 120
00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	293 _dirprefixlen = 8
00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	294 _maxshortdirslen = 8 * (_dirprefixlen + 1) - 4
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	295
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	296
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	297 def _hashencode(path, dotencode):
46114 59fa3890d40a node: import symbols explicitly Joerg Sonnenberger <joerg@bec.de> parents: 45957 diff changeset	298 digest = hex(hashutil.sha1(path).digest())
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	299 le = lowerencode(path[5:]).split(b'/') # skips prefix 'data/' or 'meta/'
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	300 parts = _auxencode(le, dotencode)
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	301 basename = parts[-1]
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	302 _root, ext = os.path.splitext(basename)
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	303 sdirs = []
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	304 sdirslen = 0
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	305 for p in parts[:-1]:
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	306 d = p[:_dirprefixlen]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	307 if d[-1] in b'. ':
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	308 # Windows can't access dirs ending in period or space
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	309 d = d[:-1] + b'_'
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	310 if sdirslen == 0:
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	311 t = len(d)
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	312 else:
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	313 t = sdirslen + 1 + len(d)
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	314 if t > _maxshortdirslen:
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	315 break
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	316 sdirs.append(d)
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	317 sdirslen = t
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	318 dirs = b'/'.join(sdirs)
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	319 if len(dirs) > 0:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	320 dirs += b'/'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	321 res = b'dh/' + dirs + digest + ext
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	322 spaceleft = _maxstorepathlen - len(res)
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	323 if spaceleft > 0:
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	324 filler = basename[:spaceleft]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	325 res = b'dh/' + dirs + filler + digest + ext
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	326 return res
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	327
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	328
17590 eb0884680f5c store: eliminate one level of lambda functions on _hybridencode Adrian Buehlmann <adrian@cadifra.com> parents: 17589 diff changeset	329 def _hybridencode(path, dotencode):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	330 """encodes path with a length limit
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	331
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	332 Encodes all paths that begin with 'data/', according to the following.
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	333
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	334 Default encoding (reversible):
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	335
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	336 Encodes all uppercase letters 'X' as '_x'. All reserved or illegal
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	337 characters are encoded as '~xx', where xx is the two digit hex code
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	338 of the character (see encodefilename).
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	339 Relevant path components consisting of Windows reserved filenames are
17738 b8424c92ba2b spelling: fix minor spell checker issues Mads Kiilerich <mads@kiilerich.com> parents: 17731 diff changeset	340 masked by encoding the third character ('aux' -> 'au~78', see _auxencode).
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	341
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	342 Hashed encoding (not reversible):
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	343
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	344 If the default-encoded path is longer than _maxstorepathlen, a
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	345 non-reversible hybrid hashing of the path is done instead.
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	346 This encoding uses up to _dirprefixlen characters of all directory
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	347 levels of the lowerencoded path, but not more levels than can fit into
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	348 _maxshortdirslen.
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	349 Then follows the filler followed by the sha digest of the full path.
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	350 The filler is the beginning of the basename of the lowerencoded path
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	351 (the basename is everything after the last path separator). The filler
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	352 is as long as possible, filling in characters from the basename until
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	353 the encoded path has _maxstorepathlen characters (or all chars of the
00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	354 basename have been taken).
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	355 The extension (e.g. '.i' or '.d') is preserved.
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	356
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	357 The string 'data/' at the beginning is replaced with 'dh/', if the hashed
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	358 encoding was used.
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	359 """
17609 cbc180cfd60b store: reuse direncoded path in _hybridencode Adrian Buehlmann <adrian@cadifra.com> parents: 17608 diff changeset	360 path = encodedir(path)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	361 ef = _encodefname(path).split(b'/')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	362 res = b'/'.join(_auxencode(ef, dotencode))
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	363 if len(res) > _maxstorepathlen:
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	364 res = _hashencode(path, dotencode)
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	365 return res
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	366
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	367
17624 ae103510f6aa store: add a fallback _pathencode Python function Adrian Buehlmann <adrian@cadifra.com> parents: 17623 diff changeset	368 def _pathencode(path):
18435 8c019d2fd7c0 store: switch to C-based hashed path encoding Bryan O'Sullivan <bryano@fb.com> parents: 18430 diff changeset	369 de = encodedir(path)
17693 0c6de45e1212 store: optimize _pathencode by checking the length of the unencoded path Adrian Buehlmann <adrian@cadifra.com> parents: 17653 diff changeset	370 if len(path) > _maxstorepathlen:
18435 8c019d2fd7c0 store: switch to C-based hashed path encoding Bryan O'Sullivan <bryano@fb.com> parents: 18430 diff changeset	371 return _hashencode(de, True)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	372 ef = _encodefname(de).split(b'/')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	373 res = b'/'.join(_auxencode(ef, True))
17624 ae103510f6aa store: add a fallback _pathencode Python function Adrian Buehlmann <adrian@cadifra.com> parents: 17623 diff changeset	374 if len(res) > _maxstorepathlen:
18435 8c019d2fd7c0 store: switch to C-based hashed path encoding Bryan O'Sullivan <bryano@fb.com> parents: 18430 diff changeset	375 return _hashencode(de, True)
17624 ae103510f6aa store: add a fallback _pathencode Python function Adrian Buehlmann <adrian@cadifra.com> parents: 17623 diff changeset	376 return res
ae103510f6aa store: add a fallback _pathencode Python function Adrian Buehlmann <adrian@cadifra.com> parents: 17623 diff changeset	377
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	378
17624 ae103510f6aa store: add a fallback _pathencode Python function Adrian Buehlmann <adrian@cadifra.com> parents: 17623 diff changeset	379 _pathencode = getattr(parsers, 'pathencode', _pathencode)
ae103510f6aa store: add a fallback _pathencode Python function Adrian Buehlmann <adrian@cadifra.com> parents: 17623 diff changeset	380
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	381
17623 448e6ed7c557 store: move _plainhybridencode and _dothybridencode higher up in the file Adrian Buehlmann <adrian@cadifra.com> parents: 17621 diff changeset	382 def _plainhybridencode(f):
448e6ed7c557 store: move _plainhybridencode and _dothybridencode higher up in the file Adrian Buehlmann <adrian@cadifra.com> parents: 17621 diff changeset	383 return _hybridencode(f, False)
448e6ed7c557 store: move _plainhybridencode and _dothybridencode higher up in the file Adrian Buehlmann <adrian@cadifra.com> parents: 17621 diff changeset	384
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	385
17726 7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17725 diff changeset	386 def _calcmode(vfs):
6898 69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	387 try:
69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	388 # files in .hg/ will be created using this mode
17726 7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17725 diff changeset	389 mode = vfs.stat().st_mode
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	390 # avoid some useless chmods
25658 e93036747902 global: mass rewrite to use modern octal syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25091 diff changeset	391 if (0o777 & ~util.umask) == (0o777 & mode):
6898 69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	392 mode = None
69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	393 except OSError:
69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	394 mode = None
69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	395 return mode
69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	396
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	397
45372 909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	398 _data = [
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	399 b'bookmarks',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	400 b'narrowspec',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	401 b'data',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	402 b'meta',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	403 b'00manifest.d',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	404 b'00manifest.i',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	405 b'00changelog.d',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	406 b'00changelog.i',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	407 b'phaseroots',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	408 b'obsstore',
45494 d252f51ab032 share: introduce config option to store requires in .hg/store Pulkit Goyal <7895pulkit@gmail.com> parents: 45372 diff changeset	409 b'requires',
45372 909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	410 ]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	411
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	412 REVLOG_FILES_EXT = (
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	413 b'.i',
47332 0a3fa41fa719 revlogv2: use a unique filename for data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47245 diff changeset	414 b'.idx',
0a3fa41fa719 revlogv2: use a unique filename for data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47245 diff changeset	415 b'.d',
0a3fa41fa719 revlogv2: use a unique filename for data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47245 diff changeset	416 b'.dat',
0a3fa41fa719 revlogv2: use a unique filename for data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47245 diff changeset	417 b'.n',
0a3fa41fa719 revlogv2: use a unique filename for data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47245 diff changeset	418 b'.nd',
47395 e6292eb33384 revlog: store sidedata in their own file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47332 diff changeset	419 b'.sda',
47332 0a3fa41fa719 revlogv2: use a unique filename for data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47245 diff changeset	420 )
50513 5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	421 # file extension that also use a `-SOMELONGIDHASH.ext` form
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	422 REVLOG_FILES_LONG_EXT = (
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	423 b'.nd',
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	424 b'.idx',
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	425 b'.dat',
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	426 b'.sda',
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	427 )
46989 aed6ceaad6d7 streamclone: treat volatile file as "fullfile" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	428 # files that are "volatile" and might change between listing and streaming
aed6ceaad6d7 streamclone: treat volatile file as "fullfile" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	429 #
aed6ceaad6d7 streamclone: treat volatile file as "fullfile" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	430 # note: the ".nd" file are nodemap data and won't "change" but they might be
aed6ceaad6d7 streamclone: treat volatile file as "fullfile" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	431 # deleted.
aed6ceaad6d7 streamclone: treat volatile file as "fullfile" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	432 REVLOG_FILES_VOLATILE_EXT = (b'.n', b'.nd')
46895 6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	433
46990 0b569c75d180 store: exclude `undo.` nodemap's file from `walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46989 diff changeset	434 # some exception to the above matching
47659 f030c7d22032 walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47449 diff changeset	435 #
f030c7d22032 walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47449 diff changeset	436 # XXX This is currently not in use because of issue6542
50319 4be9ecc982e1 py3: fix for Python 3.12 emitting SyntaxWarning on invalid escape sequences Mads Kiilerich <mads@kiilerich.com> parents: 50315 diff changeset	437 EXCLUDED = re.compile(br'.*undo\.[^/]+\.(nd?\|i)$')
46990 0b569c75d180 store: exclude `undo.` nodemap's file from `walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46989 diff changeset	438
46895 6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	439
6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	440 def is_revlog(f, kind, st):
6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	441 if kind != stat.S_IFREG:
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	442 return False
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	443 if f.endswith(REVLOG_FILES_EXT):
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	444 return True
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	445 return False
46895 6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	446
6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	447
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	448 def is_revlog_file(f):
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	449 if f.endswith(REVLOG_FILES_EXT):
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	450 return True
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	451 return False
46455 374d7fff7cb5 store: use `endswith` to detect revlog extension Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46322 diff changeset	452
6903 0642d9d7ec80 clone: get a list of files to clone from store Matt Mackall <mpm@selenic.com> parents: 6902 diff changeset	453
50668 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	454 @attr.s(slots=True)
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	455 class StoreFile:
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	456 """a file matching a store entry"""
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	457
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	458 unencoded_path = attr.ib()
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	459 _file_size = attr.ib(default=None)
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	460 is_volatile = attr.ib(default=False)
52391 3f0cf7bb3086 stream: preserve volatile cache early Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52389 diff changeset	461 # Missing file can be safely ignored, used by "copy/hardlink" local clone
3f0cf7bb3086 stream: preserve volatile cache early Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52389 diff changeset	462 # for cache file not covered by lock.
3f0cf7bb3086 stream: preserve volatile cache early Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52389 diff changeset	463 optional = False
50668 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	464
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	465 def file_size(self, vfs):
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	466 if self._file_size is None:
50669 5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	467 if vfs is None:
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	468 msg = b"calling vfs-less file_size without prior call: %s"
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	469 msg %= self.unencoded_path
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	470 raise error.ProgrammingError(msg)
50668 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	471 try:
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	472 self._file_size = vfs.stat(self.unencoded_path).st_size
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	473 except FileNotFoundError:
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	474 self._file_size = 0
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	475 return self._file_size
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	476
51585 8e2a68e10f5c stream: in v3, skip the "size" fast path if the entries as some unknown size Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	477 @property
8e2a68e10f5c stream: in v3, skip the "size" fast path if the entries as some unknown size Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	478 def has_size(self):
8e2a68e10f5c stream: in v3, skip the "size" fast path if the entries as some unknown size Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	479 return self._file_size is not None
8e2a68e10f5c stream: in v3, skip the "size" fast path if the entries as some unknown size Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	480
52043 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51919 diff changeset	481 def get_stream(self, vfs, volatiles):
50669 5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	482 """return data "stream" information for this file
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	483
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	484 (unencoded_file_path, content_iterator, content_size)
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	485 """
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	486 size = self.file_size(None)
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	487
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	488 def get_stream():
52044 0ad269e24075 stream: open volatile file through the manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52043 diff changeset	489 path = vfs.join(self.unencoded_path)
0ad269e24075 stream: open volatile file through the manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52043 diff changeset	490 with volatiles.open(path) as fp:
50669 5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	491 yield None # ready to stream
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	492 if size <= 65536:
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	493 yield fp.read(size)
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	494 else:
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	495 yield from util.filechunkiter(fp, limit=size)
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	496
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	497 s = get_stream()
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	498 next(s)
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	499 return (self.unencoded_path, s, size)
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	500
50668 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	501
50508 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	502 @attr.s(slots=True, init=False)
50507 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	503 class BaseStoreEntry:
50505 521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50503 diff changeset	504 """An entry in the store
521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50503 diff changeset	505
521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50503 diff changeset	506 This is returned by `store.walk` and represent some data in the store."""
521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50503 diff changeset	507
51590 49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	508 maybe_volatile = True
49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	509
50668 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	510 def files(self) -> List[StoreFile]:
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	511 raise NotImplementedError
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	512
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	513 def get_streams(
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	514 self,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	515 repo=None,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	516 vfs=None,
52043 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51919 diff changeset	517 volatiles=None,
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	518 max_changeset=None,
50706 0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	519 preserve_file_count=False,
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	520 ):
50669 5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	521 """return a list of data stream associated to files for this entry
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	522
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	523 return [(unencoded_file_path, content_iterator, content_size), …]
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	524 """
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	525 assert vfs is not None
52043 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51919 diff changeset	526 return [f.get_stream(vfs, volatiles) for f in self.files()]
50669 5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50668 diff changeset	527
52389 a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52301 diff changeset	528 def preserve_volatiles(self, vfs, volatiles):
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52301 diff changeset	529 """Use a VolatileManager to preserve the state of any volatile file
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52301 diff changeset	530
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52301 diff changeset	531 This is useful for code that need a consistent view of the content like stream clone.
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52301 diff changeset	532 """
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52301 diff changeset	533 if self.maybe_volatile:
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52301 diff changeset	534 for f in self.files():
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52301 diff changeset	535 if f.is_volatile:
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52301 diff changeset	536 volatiles(vfs.join(f.unencoded_path))
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52301 diff changeset	537
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	538
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	539 @attr.s(slots=True, init=False)
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	540 class SimpleStoreEntry(BaseStoreEntry):
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	541 """A generic entry in the store"""
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	542
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	543 is_revlog = False
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	544
51590 49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	545 maybe_volatile = attr.ib()
50529 ed8cda1c18e1 store: rename `unencoded_path` to `entry_path` for StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50528 diff changeset	546 _entry_path = attr.ib()
50510 2b2284cf949b store: only access is_volatile information through the file object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50509 diff changeset	547 _is_volatile = attr.ib(default=False)
50509 7d4d2a160cf5 store: only access file_size information through the file object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50508 diff changeset	548 _file_size = attr.ib(default=None)
50661 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	549 _files = attr.ib(default=None)
50505 521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50503 diff changeset	550
50508 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	551 def __init__(
c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	552 self,
50529 ed8cda1c18e1 store: rename `unencoded_path` to `entry_path` for StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50528 diff changeset	553 entry_path,
50508 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	554 is_volatile=False,
c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	555 file_size=None,
c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	556 ):
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	557 super().__init__()
50529 ed8cda1c18e1 store: rename `unencoded_path` to `entry_path` for StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50528 diff changeset	558 self._entry_path = entry_path
50510 2b2284cf949b store: only access is_volatile information through the file object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50509 diff changeset	559 self._is_volatile = is_volatile
50509 7d4d2a160cf5 store: only access file_size information through the file object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50508 diff changeset	560 self._file_size = file_size
50661 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	561 self._files = None
51590 49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	562 self.maybe_volatile = is_volatile
50508 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	563
50668 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	564 def files(self) -> List[StoreFile]:
50661 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	565 if self._files is None:
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	566 self._files = [
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	567 StoreFile(
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	568 unencoded_path=self._entry_path,
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	569 file_size=self._file_size,
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	570 is_volatile=self._is_volatile,
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	571 )
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	572 ]
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	573 return self._files
50506 9fdc28e21b68 store: introduce a EntryFile object to actually access file info Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50505 diff changeset	574
9fdc28e21b68 store: introduce a EntryFile object to actually access file info Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50505 diff changeset	575
50508 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	576 @attr.s(slots=True, init=False)
50507 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	577 class RevlogStoreEntry(BaseStoreEntry):
5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	578 """A revlog entry in the store"""
5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	579
5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	580 is_revlog = True
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	581
50507 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	582 revlog_type = attr.ib(default=None)
50517 60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50516 diff changeset	583 target_id = attr.ib(default=None)
51590 49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	584 maybe_volatile = attr.ib(default=True)
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	585 _path_prefix = attr.ib(default=None)
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	586 _details = attr.ib(default=None)
50661 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	587 _files = attr.ib(default=None)
50507 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	588
50508 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	589 def __init__(
c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	590 self,
c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	591 revlog_type,
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	592 path_prefix,
50517 60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50516 diff changeset	593 target_id,
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	594 details,
50508 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	595 ):
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	596 super().__init__()
50508 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	597 self.revlog_type = revlog_type
50517 60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50516 diff changeset	598 self.target_id = target_id
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	599 self._path_prefix = path_prefix
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	600 assert b'.i' in details, (path_prefix, details)
51590 49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	601 for ext in details:
49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	602 if ext.endswith(REVLOG_FILES_VOLATILE_EXT):
49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	603 self.maybe_volatile = True
49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	604 break
49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	605 else:
49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51585 diff changeset	606 self.maybe_volatile = False
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	607 self._details = details
50661 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	608 self._files = None
50508 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50507 diff changeset	609
50531 66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50530 diff changeset	610 @property
66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50530 diff changeset	611 def is_changelog(self):
50688 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	612 return self.revlog_type == KIND_CHANGELOG
50531 66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50530 diff changeset	613
66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50530 diff changeset	614 @property
66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50530 diff changeset	615 def is_manifestlog(self):
50688 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	616 return self.revlog_type == KIND_MANIFESTLOG
50531 66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50530 diff changeset	617
66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50530 diff changeset	618 @property
66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50530 diff changeset	619 def is_filelog(self):
50688 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	620 return self.revlog_type == KIND_FILELOG
50531 66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50530 diff changeset	621
50526 3473d18c029a store: introduce a main_file_path method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50517 diff changeset	622 def main_file_path(self):
3473d18c029a store: introduce a main_file_path method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50517 diff changeset	623 """unencoded path of the main revlog file"""
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	624 return self._path_prefix + b'.i'
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	625
50668 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50662 diff changeset	626 def files(self) -> List[StoreFile]:
50661 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	627 if self._files is None:
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	628 self._files = []
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	629 for ext in sorted(self._details, key=_ext_key):
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	630 path = self._path_prefix + ext
50678 11562d72cb7b store: directly pass the filesize in the `details` of revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50677 diff changeset	631 file_size = self._details[ext]
50676 6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50671 diff changeset	632 # files that are "volatile" and might change between
6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50671 diff changeset	633 # listing and streaming
6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50671 diff changeset	634 #
6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50671 diff changeset	635 # note: the ".nd" file are nodemap data and won't "change"
6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50671 diff changeset	636 # but they might be deleted.
6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50671 diff changeset	637 volatile = ext.endswith(REVLOG_FILES_VOLATILE_EXT)
50677 cdb471c8ebcf store: explicitly pass file_size when creating StoreFile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50676 diff changeset	638 f = StoreFile(path, file_size, volatile)
50676 6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50671 diff changeset	639 self._files.append(f)
50661 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50659 diff changeset	640 return self._files
50526 3473d18c029a store: introduce a main_file_path method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50517 diff changeset	641
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	642 def get_streams(
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	643 self,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	644 repo=None,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	645 vfs=None,
52043 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51919 diff changeset	646 volatiles=None,
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	647 max_changeset=None,
50706 0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	648 preserve_file_count=False,
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	649 ):
51585 8e2a68e10f5c stream: in v3, skip the "size" fast path if the entries as some unknown size Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	650 pre_sized = all(f.has_size for f in self.files())
8e2a68e10f5c stream: in v3, skip the "size" fast path if the entries as some unknown size Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50952 diff changeset	651 if pre_sized and (
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	652 repo is None
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	653 or max_changeset is None
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	654 # This use revlog-v2, ignore for now
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	655 or any(k.endswith(b'.idx') for k in self._details.keys())
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	656 # This is not inline, no race expected
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	657 or b'.d' in self._details
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	658 ):
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	659 return super().get_streams(
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	660 repo=repo,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	661 vfs=vfs,
52043 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51919 diff changeset	662 volatiles=volatiles,
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	663 max_changeset=max_changeset,
50706 0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	664 preserve_file_count=preserve_file_count,
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	665 )
50706 0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	666 elif not preserve_file_count:
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	667 stream = [
52043 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51919 diff changeset	668 f.get_stream(vfs, volatiles)
50706 0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	669 for f in self.files()
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	670 if not f.unencoded_path.endswith((b'.i', b'.d'))
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	671 ]
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	672 rl = self.get_revlog_instance(repo).get_revlog()
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	673 rl_stream = rl.get_streams(max_changeset)
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	674 stream.extend(rl_stream)
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50688 diff changeset	675 return stream
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	676
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	677 name_to_size = {}
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	678 for f in self.files():
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	679 name_to_size[f.unencoded_path] = f.file_size(None)
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	680
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	681 stream = [
52043 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51919 diff changeset	682 f.get_stream(vfs, volatiles)
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	683 for f in self.files()
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	684 if not f.unencoded_path.endswith(b'.i')
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	685 ]
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	686
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	687 index_path = self._path_prefix + b'.i'
50671 5460424092e2 stream-clone: smoothly detect and handle a case were a revlog is split Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50670 diff changeset	688
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	689 index_file = None
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	690 try:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	691 index_file = vfs(index_path)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	692 header = index_file.read(INDEX_HEADER.size)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	693 if revlogmod.revlog.is_inline_index(header):
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	694 size = name_to_size[index_path]
50671 5460424092e2 stream-clone: smoothly detect and handle a case were a revlog is split Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50670 diff changeset	695
50680 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	696 # no split underneath, just return the stream
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	697 def get_stream():
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	698 fp = index_file
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	699 try:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	700 fp.seek(0)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	701 yield None
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	702 if size <= 65536:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	703 yield fp.read(size)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	704 else:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	705 yield from util.filechunkiter(fp, limit=size)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	706 finally:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	707 fp.close()
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	708
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	709 s = get_stream()
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	710 next(s)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	711 index_file = None
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	712 stream.append((index_path, s, size))
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	713 else:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	714 rl = self.get_revlog_instance(repo).get_revlog()
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	715 rl_stream = rl.get_streams(max_changeset, force_inline=True)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	716 for name, s, size in rl_stream:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	717 if name_to_size.get(name, 0) != size:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	718 msg = _(b"expected %d bytes but %d provided for %s")
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	719 msg %= name_to_size.get(name, 0), size, name
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	720 raise error.Abort(msg)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	721 stream.extend(rl_stream)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	722 finally:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	723 if index_file is not None:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	724 index_file.close()
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50679 diff changeset	725
50670 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	726 files = self.files()
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	727 assert len(stream) == len(files), (
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	728 stream,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	729 files,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	730 self._path_prefix,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	731 self.target_id,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	732 )
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	733 return stream
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	734
50657 e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	735 def get_revlog_instance(self, repo):
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	736 """Obtain a revlog instance from this store entry
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	737
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	738 An instance of the appropriate class is returned.
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	739 """
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	740 if self.is_changelog:
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	741 return changelog.changelog(repo.svfs)
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	742 elif self.is_manifestlog:
50658 3ea3767c23a4 store: do not drop the final `/` when creating manifestlog instance Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50657 diff changeset	743 mandir = self.target_id
50657 e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	744 return manifest.manifestrevlog(
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	745 repo.nodeconstants, repo.svfs, tree=mandir
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	746 )
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	747 else:
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	748 return filelog.filelog(repo.svfs, self.target_id)
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50548 diff changeset	749
50507 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	750
50513 5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	751 def _gather_revlog(files_data):
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	752 """group files per revlog prefix
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	753
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	754 The returns a two level nested dict. The top level key is the revlog prefix
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	755 without extension, the second level is all the file "suffix" that were
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	756 seen for this revlog and arbitrary file data as value.
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	757 """
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	758 revlogs = collections.defaultdict(dict)
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	759 for u, value in files_data:
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	760 name, ext = _split_revlog_ext(u)
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	761 revlogs[name][ext] = value
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	762 return sorted(revlogs.items())
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	763
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	764
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	765 def _split_revlog_ext(filename):
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	766 """split the revlog file prefix from the variable extension"""
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	767 if filename.endswith(REVLOG_FILES_LONG_EXT):
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	768 char = b'-'
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	769 else:
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	770 char = b'.'
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	771 idx = filename.rfind(char)
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	772 return filename[:idx], filename[idx:]
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	773
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50512 diff changeset	774
50515 b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	775 def _ext_key(ext):
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	776 """a key to order revlog suffix
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	777
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	778 important to issue .i after other entry."""
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	779 # the only important part of this order is to keep the `.i` last.
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	780 if ext.endswith(b'.n'):
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	781 return (0, ext)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	782 elif ext.endswith(b'.nd'):
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	783 return (10, ext)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	784 elif ext.endswith(b'.d'):
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	785 return (20, ext)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	786 elif ext.endswith(b'.i'):
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	787 return (50, ext)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	788 else:
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	789 return (40, ext)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	790
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	791
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49032 diff changeset	792 class basicstore:
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	793 '''base class for local repository stores'''
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	794
17651 3b49c28658f6 store: rename "openertype" argument to "vfstype" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17649 diff changeset	795 def __init__(self, path, vfstype):
17724 bf4b72d8dd4d store: initialize vfs field first to use it for initialization of others FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17722 diff changeset	796 vfs = vfstype(path)
bf4b72d8dd4d store: initialize vfs field first to use it for initialization of others FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17722 diff changeset	797 self.path = vfs.base
17726 7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17725 diff changeset	798 self.createmode = _calcmode(vfs)
17652 2c6f7231becc store: rename "op" variables to "vfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17651 diff changeset	799 vfs.createmode = self.createmode
17728 004bd533880d store: invoke "os.path.isdir()" via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17727 diff changeset	800 self.rawvfs = vfs
31244 9b7a2ef4f27c vfs: use 'vfs' module directly in 'mercurial.store' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31229 diff changeset	801 self.vfs = vfsmod.filtervfs(vfs, encodedir)
17653 dacb50696b75 store: initialize "vfs" fields by "vfs" constructors FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17652 diff changeset	802 self.opener = self.vfs
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	803
80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	804 def join(self, f):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	805 return self.path + b'/' + encodedir(f)
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	806
50527 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	807 def _walk(self, relpath, recurse, undecodable=None):
47918 2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	808 '''yields (revlog_type, unencoded, size)'''
13426 643b8212813e store: remove pointless pathjoiner parameter Adrian Buehlmann <adrian@cadifra.com> parents: 13391 diff changeset	809 path = self.path
643b8212813e store: remove pointless pathjoiner parameter Adrian Buehlmann <adrian@cadifra.com> parents: 13391 diff changeset	810 if relpath:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	811 path += b'/' + relpath
13426 643b8212813e store: remove pointless pathjoiner parameter Adrian Buehlmann <adrian@cadifra.com> parents: 13391 diff changeset	812 striplen = len(self.path) + 1
6899 56a7a54e074f store: simplify walking Matt Mackall <mpm@selenic.com> parents: 6898 diff changeset	813 l = []
17728 004bd533880d store: invoke "os.path.isdir()" via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17727 diff changeset	814 if self.rawvfs.isdir(path):
6899 56a7a54e074f store: simplify walking Matt Mackall <mpm@selenic.com> parents: 6898 diff changeset	815 visit = [path]
17747 aad3bce98f76 store: invoke "osutil.listdir()" via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17745 diff changeset	816 readdir = self.rawvfs.readdir
6899 56a7a54e074f store: simplify walking Matt Mackall <mpm@selenic.com> parents: 6898 diff changeset	817 while visit:
56a7a54e074f store: simplify walking Matt Mackall <mpm@selenic.com> parents: 6898 diff changeset	818 p = visit.pop()
17747 aad3bce98f76 store: invoke "osutil.listdir()" via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17745 diff changeset	819 for f, kind, st in readdir(p, stat=True):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	820 fp = p + b'/' + f
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	821 if is_revlog(f, kind, st):
6900 def492d1b592 store: change handling of decoding errors Matt Mackall <mpm@selenic.com> parents: 6899 diff changeset	822 n = util.pconvert(fp[striplen:])
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	823 l.append((decodedir(n), st.st_size))
6899 56a7a54e074f store: simplify walking Matt Mackall <mpm@selenic.com> parents: 6898 diff changeset	824 elif kind == stat.S_IFDIR and recurse:
56a7a54e074f store: simplify walking Matt Mackall <mpm@selenic.com> parents: 6898 diff changeset	825 visit.append(fp)
50512 1c0244a8cdaf store: change `_walk` return to `(filename, (type, size))` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50511 diff changeset	826
17054 125ff5654b72 store: sort filenames in place Bryan O'Sullivan <bryano@fb.com> parents: 16404 diff changeset	827 l.sort()
125ff5654b72 store: sort filenames in place Bryan O'Sullivan <bryano@fb.com> parents: 16404 diff changeset	828 return l
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	829
46646 e9901d01d135 revlog: add a mechanism to verify expected file position before appending Kyle Lippincott <spectral@google.com> parents: 46455 diff changeset	830 def changelog(self, trypending, concurrencychecker=None):
e9901d01d135 revlog: add a mechanism to verify expected file position before appending Kyle Lippincott <spectral@google.com> parents: 46455 diff changeset	831 return changelog.changelog(
e9901d01d135 revlog: add a mechanism to verify expected file position before appending Kyle Lippincott <spectral@google.com> parents: 46455 diff changeset	832 self.vfs,
e9901d01d135 revlog: add a mechanism to verify expected file position before appending Kyle Lippincott <spectral@google.com> parents: 46455 diff changeset	833 trypending=trypending,
e9901d01d135 revlog: add a mechanism to verify expected file position before appending Kyle Lippincott <spectral@google.com> parents: 46455 diff changeset	834 concurrencychecker=concurrencychecker,
e9901d01d135 revlog: add a mechanism to verify expected file position before appending Kyle Lippincott <spectral@google.com> parents: 46455 diff changeset	835 )
42911 3df3b139a43d localrepo: push manifestlog and changelog construction code into store Augie Fackler <augie@google.com> parents: 42664 diff changeset	836
52094 6641a3ae8313 manifest: drop the CamelCase name for `manifest.manifestlog` Matt Harbison <matt_harbison@yahoo.com> parents: 52044 diff changeset	837 def manifestlog(self, repo, storenarrowmatch) -> manifest.manifestlog:
46793 6266d19556ad node: introduce nodeconstants class Joerg Sonnenberger <joerg@bec.de> parents: 46646 diff changeset	838 rootstore = manifest.manifestrevlog(repo.nodeconstants, self.vfs)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	839 return manifest.manifestlog(self.vfs, repo, rootstore, storenarrowmatch)
42911 3df3b139a43d localrepo: push manifestlog and changelog construction code into store Augie Fackler <augie@google.com> parents: 42664 diff changeset	840
50538 862e3a13da44 store: rename `datafiles` to `data_entries` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50532 diff changeset	841 def data_entries(
50505 521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50503 diff changeset	842 self, matcher=None, undecodable=None
50507 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	843 ) -> Generator[BaseStoreEntry, None, None]:
47918 2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	844 """Like walk, but excluding the changelog and root manifest.
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	845
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	846 When [undecodable] is None, revlogs names that can't be
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	847 decoded cause an exception. When it is provided, it should
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	848 be a list and the filenames that can't be decoded are added
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	849 to it instead. This is very rarely needed."""
50516 1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50515 diff changeset	850 dirs = [
50688 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	851 (b'data', KIND_FILELOG, False),
e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	852 (b'meta', KIND_MANIFESTLOG, True),
50516 1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50515 diff changeset	853 ]
50659 309cbd8400ae store: properly compute the targer_id of manifestlog in no-fncache walk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50658 diff changeset	854 for base_dir, rl_type, strip_filename in dirs:
50527 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	855 files = self._walk(base_dir, True, undecodable=undecodable)
50516 1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50515 diff changeset	856 for revlog, details in _gather_revlog(files):
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	857 revlog_target_id = revlog.split(b'/', 1)[1]
50659 309cbd8400ae store: properly compute the targer_id of manifestlog in no-fncache walk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50658 diff changeset	858 if strip_filename and b'/' in revlog:
309cbd8400ae store: properly compute the targer_id of manifestlog in no-fncache walk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50658 diff changeset	859 revlog_target_id = revlog_target_id.rsplit(b'/', 1)[0]
309cbd8400ae store: properly compute the targer_id of manifestlog in no-fncache walk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50658 diff changeset	860 revlog_target_id += b'/'
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	861 yield RevlogStoreEntry(
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	862 path_prefix=revlog,
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	863 revlog_type=rl_type,
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	864 target_id=revlog_target_id,
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	865 details=details,
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	866 )
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	867
50548 0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	868 def top_entries(
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	869 self, phase=False, obsolescence=False
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	870 ) -> Generator[BaseStoreEntry, None, None]:
50547 5a62d56e3955 store: yield phases before changelog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50546 diff changeset	871 if phase and self.vfs.exists(b'phaseroots'):
5a62d56e3955 store: yield phases before changelog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50546 diff changeset	872 yield SimpleStoreEntry(
5a62d56e3955 store: yield phases before changelog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50546 diff changeset	873 entry_path=b'phaseroots',
5a62d56e3955 store: yield phases before changelog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50546 diff changeset	874 is_volatile=True,
5a62d56e3955 store: yield phases before changelog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50546 diff changeset	875 )
5a62d56e3955 store: yield phases before changelog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50546 diff changeset	876
50548 0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	877 if obsolescence and self.vfs.exists(b'obsstore'):
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	878 # XXX if we had the file size it could be non-volatile
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	879 yield SimpleStoreEntry(
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	880 entry_path=b'obsstore',
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	881 is_volatile=True,
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	882 )
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	883
50515 b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	884 files = reversed(self._walk(b'', False))
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	885
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	886 changelogs = collections.defaultdict(dict)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	887 manifestlogs = collections.defaultdict(dict)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	888
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	889 for u, s in files:
46895 6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	890 if u.startswith(b'00changelog'):
50515 b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	891 name, ext = _split_revlog_ext(u)
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	892 changelogs[name][ext] = s
46895 6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	893 elif u.startswith(b'00manifest'):
50515 b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	894 name, ext = _split_revlog_ext(u)
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	895 manifestlogs[name][ext] = s
46895 6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	896 else:
50507 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	897 yield SimpleStoreEntry(
50529 ed8cda1c18e1 store: rename `unencoded_path` to `entry_path` for StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50528 diff changeset	898 entry_path=u,
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	899 is_volatile=False,
50507 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	900 file_size=s,
5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	901 )
50515 b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	902 # yield manifest before changelog
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	903 top_rl = [
50688 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	904 (manifestlogs, KIND_MANIFESTLOG),
e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	905 (changelogs, KIND_CHANGELOG),
50515 b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	906 ]
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	907 assert len(manifestlogs) <= 1
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	908 assert len(changelogs) <= 1
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	909 for data, revlog_type in top_rl:
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50514 diff changeset	910 for revlog, details in sorted(data.items()):
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	911 yield RevlogStoreEntry(
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	912 path_prefix=revlog,
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	913 revlog_type=revlog_type,
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	914 target_id=b'',
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	915 details=details,
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	916 )
19177 1e104aaa4c44 store: move top file walk to a separate function Durham Goode <durham@fb.com> parents: 18435 diff changeset	917
50546 a32d739b0ffb store: make `walk` return an entry for phase if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	918 def walk(
50548 0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50547 diff changeset	919 self, matcher=None, phase=False, obsolescence=False
50546 a32d739b0ffb store: make `walk` return an entry for phase if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	920 ) -> Generator[BaseStoreEntry, None, None]:
50485 584ff1f97201 comments: fix spelling Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50319 diff changeset	921 """return files related to data storage (ie: revlogs)
46853 eed3e2b79b48 store: document the `walk` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	922
50546 a32d739b0ffb store: make `walk` return an entry for phase if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50539 diff changeset	923 yields instance from BaseStoreEntry subclasses
40340 2d45b549392f store: pass matcher to store.datafiles() Pulkit Goyal <pulkit@yandex-team.ru> parents: 39714 diff changeset	924
2d45b549392f store: pass matcher to store.datafiles() Pulkit Goyal <pulkit@yandex-team.ru> parents: 39714 diff changeset	925 if a matcher is passed, storage files of only those tracked paths
2d45b549392f store: pass matcher to store.datafiles() Pulkit Goyal <pulkit@yandex-team.ru> parents: 39714 diff changeset	926 are passed with matches the matcher
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	927 """
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	928 # yield data files first
52669 e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52668 diff changeset	929 yield from self.data_entries(matcher)
e627cc25b6f3 pyupgrade: rewrite `yield` statements in a loop to `yield from` Matt Harbison <matt_harbison@yahoo.com> parents: 52668 diff changeset	930 yield from self.top_entries(phase=phase, obsolescence=obsolescence)
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	931
6903 0642d9d7ec80 clone: get a list of files to clone from store Matt Mackall <mpm@selenic.com> parents: 6902 diff changeset	932 def copylist(self):
45494 d252f51ab032 share: introduce config option to store requires in .hg/store Pulkit Goyal <7895pulkit@gmail.com> parents: 45372 diff changeset	933 return _data
6903 0642d9d7ec80 clone: get a list of files to clone from store Matt Mackall <mpm@selenic.com> parents: 6902 diff changeset	934
20883 cd443c7589cc fncache: move fncache writing to be in a transaction Durham Goode <durham@fb.com> parents: 20879 diff changeset	935 def write(self, tr):
13391 d00bbff8600e fncachestore: defer updating the fncache file to a single file open Adrian Buehlmann <adrian@cadifra.com> parents: 13169 diff changeset	936 pass
d00bbff8600e fncachestore: defer updating the fncache file to a single file open Adrian Buehlmann <adrian@cadifra.com> parents: 13169 diff changeset	937
20884 2efdd186925d caches: invalidate store caches when lock is taken Durham Goode <durham@fb.com> parents: 20883 diff changeset	938 def invalidatecaches(self):
2efdd186925d caches: invalidate store caches when lock is taken Durham Goode <durham@fb.com> parents: 20883 diff changeset	939 pass
2efdd186925d caches: invalidate store caches when lock is taken Durham Goode <durham@fb.com> parents: 20883 diff changeset	940
20885 f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	941 def markremoved(self, fn):
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	942 pass
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	943
17744 09d5b2055295 store: add a contains method to basicstore smuralid parents: 17738 diff changeset	944 def __contains__(self, path):
09d5b2055295 store: add a contains method to basicstore smuralid parents: 17738 diff changeset	945 '''Checks if the store contains path'''
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	946 path = b"/".join((b"data", path))
17744 09d5b2055295 store: add a contains method to basicstore smuralid parents: 17738 diff changeset	947 # file?
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	948 if self.vfs.exists(path + b".i"):
17744 09d5b2055295 store: add a contains method to basicstore smuralid parents: 17738 diff changeset	949 return True
09d5b2055295 store: add a contains method to basicstore smuralid parents: 17738 diff changeset	950 # dir?
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	951 if not path.endswith(b"/"):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	952 path = path + b"/"
19903 ca875b271ac3 store: use "vfs.exists()" instead of "os.path.exists()" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 19177 diff changeset	953 return self.vfs.exists(path)
17744 09d5b2055295 store: add a contains method to basicstore smuralid parents: 17738 diff changeset	954
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	955
6898 69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	956 class encodedstore(basicstore):
17651 3b49c28658f6 store: rename "openertype" argument to "vfstype" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17649 diff changeset	957 def __init__(self, path, vfstype):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	958 vfs = vfstype(path + b'/store')
17724 bf4b72d8dd4d store: initialize vfs field first to use it for initialization of others FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17722 diff changeset	959 self.path = vfs.base
17726 7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17725 diff changeset	960 self.createmode = _calcmode(vfs)
17652 2c6f7231becc store: rename "op" variables to "vfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17651 diff changeset	961 vfs.createmode = self.createmode
17728 004bd533880d store: invoke "os.path.isdir()" via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17727 diff changeset	962 self.rawvfs = vfs
31244 9b7a2ef4f27c vfs: use 'vfs' module directly in 'mercurial.store' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31229 diff changeset	963 self.vfs = vfsmod.filtervfs(vfs, encodefilename)
17653 dacb50696b75 store: initialize "vfs" fields by "vfs" constructors FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17652 diff changeset	964 self.opener = self.vfs
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	965
50527 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	966 def _walk(self, relpath, recurse, undecodable=None):
816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	967 old = super()._walk(relpath, recurse)
816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	968 new = []
816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	969 for f1, value in old:
6892 dab95717058d verify: check repo.store Adrian Buehlmann <adrian@cadifra.com> parents: 6890 diff changeset	970 try:
47918 2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	971 f2 = decodefilename(f1)
6892 dab95717058d verify: check repo.store Adrian Buehlmann <adrian@cadifra.com> parents: 6890 diff changeset	972 except KeyError:
47918 2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	973 if undecodable is None:
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	974 msg = _(b'undecodable revlog name %s') % f1
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	975 raise error.StorageError(msg)
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	976 else:
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	977 undecodable.append(f1)
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47784 diff changeset	978 continue
50527 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	979 new.append((f2, value))
816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	980 return new
816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	981
50538 862e3a13da44 store: rename `datafiles` to `data_entries` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50532 diff changeset	982 def data_entries(
50527 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	983 self, matcher=None, undecodable=None
816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	984 ) -> Generator[BaseStoreEntry, None, None]:
52668 5cc8deb96b48 pyupgrade: modernize calls to superclass methods Matt Harbison <matt_harbison@yahoo.com> parents: 52665 diff changeset	985 entries = super().data_entries(undecodable=undecodable)
50527 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	986 for entry in entries:
50528 b4953fad744e store: do the revlog matching on entry directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50527 diff changeset	987 if _match_tracked_entry(entry, matcher):
50527 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50526 diff changeset	988 yield entry
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	989
80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	990 def join(self, f):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	991 return self.path + b'/' + encodefilename(f)
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	992
6903 0642d9d7ec80 clone: get a list of files to clone from store Matt Mackall <mpm@selenic.com> parents: 6902 diff changeset	993 def copylist(self):
45372 909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	994 return [b'requires', b'00changelog.i'] + [b'store/' + f for f in _data]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	995
6903 0642d9d7ec80 clone: get a list of files to clone from store Matt Mackall <mpm@selenic.com> parents: 6902 diff changeset	996
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 49032 diff changeset	997 class fncache:
8531 810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	998 # the filename used to be partially encoded
810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	999 # hence the encodedir/decodedir dance
17722 3b976051034d store: rename field name from "opener" to "vfs" in internal classes for fncache FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17721 diff changeset	1000 def __init__(self, vfs):
3b976051034d store: rename field name from "opener" to "vfs" in internal classes for fncache FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17721 diff changeset	1001 self.vfs = vfs
50315 cf6e1d535602 fncache: make it possible to ignore some file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49314 diff changeset	1002 self._ignores = set()
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1003 self.entries = None
13391 d00bbff8600e fncachestore: defer updating the fncache file to a single file open Adrian Buehlmann <adrian@cadifra.com> parents: 13169 diff changeset	1004 self._dirty = False
40770 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1005 # set of new additions to fncache
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1006 self.addls = set()
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1007
52299 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1008 @property
ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1009 def is_loaded(self):
ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1010 return self.entries is not None
ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1011
42664 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1012 def ensureloaded(self, warn=None):
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	1013 """read the fncache file if not already read.
42664 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1014
f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1015 If the file on disk is corrupted, raise. If warn is provided,
45957 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45494 diff changeset	1016 warn and keep going instead."""
52299 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1017 if not self.is_loaded:
42664 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1018 self._load(warn)
f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1019
f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1020 def _load(self, warn=None):
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1021 '''fill the entries from the fncache file'''
13391 d00bbff8600e fncachestore: defer updating the fncache file to a single file open Adrian Buehlmann <adrian@cadifra.com> parents: 13169 diff changeset	1022 self._dirty = False
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1023 try:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1024 fp = self.vfs(b'fncache', mode=b'rb')
52665 24ee91ba9aa8 pyupgrade: drop usage of py3 aliases for `OSError` Matt Harbison <matt_harbison@yahoo.com> parents: 52664 diff changeset	1025 except OSError:
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1026 # skip nonexistent file
16404 9fca5b056c0a store: speed up read and write of large fncache files Bryan O'Sullivan <bryano@fb.com> parents: 15742 diff changeset	1027 self.entries = set()
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1028 return
41980 a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1029
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1030 self.entries = set()
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1031 chunk = b''
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1032 for c in iter(functools.partial(fp.read, fncache_chunksize), b''):
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1033 chunk += c
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1034 try:
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1035 p = chunk.rindex(b'\n')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1036 self.entries.update(decodedir(chunk[: p + 1]).splitlines())
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1037 chunk = chunk[p + 1 :]
41980 a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1038 except ValueError:
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1039 # substring '\n' not found, maybe the entry is bigger than the
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1040 # chunksize, so let's keep iterating
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1041 pass
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1042
41983 a920a9e1795a store: error out if fncache does not ends with a newline Pulkit Goyal <pulkit@yandex-team.ru> parents: 41980 diff changeset	1043 if chunk:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1044 msg = _(b"fncache does not ends with a newline")
42664 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1045 if warn:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1046 warn(msg + b'\n')
42664 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1047 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1048 raise error.Abort(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1049 msg,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1050 hint=_(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1051 b"use 'hg debugrebuildfncache' to "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1052 b"rebuild the fncache"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1053 ),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1054 )
42664 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1055 self._checkentries(fp, warn)
41978 d7ef84e595f8 store: move logic to check for invalid entry in fncache to own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41096 diff changeset	1056 fp.close()
d7ef84e595f8 store: move logic to check for invalid entry in fncache to own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41096 diff changeset	1057
42664 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1058 def _checkentries(self, fp, warn):
47031 f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1 Kyle Lippincott <spectral@google.com> parents: 46990 diff changeset	1059 """make sure there is no empty string in entries"""
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1060 if b'' in self.entries:
16404 9fca5b056c0a store: speed up read and write of large fncache files Bryan O'Sullivan <bryano@fb.com> parents: 15742 diff changeset	1061 fp.seek(0)
49032 fd5b8e696b75 py3: stop using util.iterfile() Gregory Szorc <gregory.szorc@gmail.com> parents: 49004 diff changeset	1062 for n, line in enumerate(fp):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1063 if not line.rstrip(b'\n'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1064 t = _(b'invalid entry in fncache, line %d') % (n + 1)
42664 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1065 if warn:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1066 warn(t + b'\n')
42664 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1067 else:
f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42363 diff changeset	1068 raise error.Abort(t)
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1069
20883 cd443c7589cc fncache: move fncache writing to be in a transaction Durham Goode <durham@fb.com> parents: 20879 diff changeset	1070 def write(self, tr):
16404 9fca5b056c0a store: speed up read and write of large fncache files Bryan O'Sullivan <bryano@fb.com> parents: 15742 diff changeset	1071 if self._dirty:
52299 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1072 assert self.is_loaded
40782 df8ed31a8ad8 store: write fncache only once if there are both adds and removes Pulkit Goyal <pulkit@yandex-team.ru> parents: 40770 diff changeset	1073 self.entries = self.entries \| self.addls
df8ed31a8ad8 store: write fncache only once if there are both adds and removes Pulkit Goyal <pulkit@yandex-team.ru> parents: 40770 diff changeset	1074 self.addls = set()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1075 tr.addbackup(b'fncache')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1076 fp = self.vfs(b'fncache', mode=b'wb', atomictemp=True)
20879 cd03854a2e06 fncache: remove the rewriting logic Durham Goode <durham@fb.com> parents: 19903 diff changeset	1077 if self.entries:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1078 fp.write(encodedir(b'\n'.join(self.entries) + b'\n'))
20879 cd03854a2e06 fncache: remove the rewriting logic Durham Goode <durham@fb.com> parents: 19903 diff changeset	1079 fp.close()
cd03854a2e06 fncache: remove the rewriting logic Durham Goode <durham@fb.com> parents: 19903 diff changeset	1080 self._dirty = False
40770 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1081 if self.addls:
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1082 # if we have just new entries, let's append them to the fncache
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1083 tr.addbackup(b'fncache')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1084 fp = self.vfs(b'fncache', mode=b'ab', atomictemp=True)
40770 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1085 if self.addls:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1086 fp.write(encodedir(b'\n'.join(self.addls) + b'\n'))
40770 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1087 fp.close()
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1088 self.entries = None
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1089 self.addls = set()
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1090
50315 cf6e1d535602 fncache: make it possible to ignore some file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49314 diff changeset	1091 def addignore(self, fn):
cf6e1d535602 fncache: make it possible to ignore some file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49314 diff changeset	1092 self._ignores.add(fn)
cf6e1d535602 fncache: make it possible to ignore some file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49314 diff changeset	1093
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1094 def add(self, fn):
50315 cf6e1d535602 fncache: make it possible to ignore some file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49314 diff changeset	1095 if fn in self._ignores:
cf6e1d535602 fncache: make it possible to ignore some file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49314 diff changeset	1096 return
52299 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1097 if not self.is_loaded:
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1098 self._load()
10577 d5bd1beff794 store: only add new entries to the fncache file Adrian Buehlmann <adrian@cadifra.com> parents: 10339 diff changeset	1099 if fn not in self.entries:
40770 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1100 self.addls.add(fn)
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1101
20885 f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1102 def remove(self, fn):
52299 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1103 if not self.is_loaded:
20885 f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1104 self._load()
40770 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1105 if fn in self.addls:
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1106 self.addls.remove(fn)
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1107 return
20885 f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1108 try:
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1109 self.entries.remove(fn)
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1110 self._dirty = True
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1111 except KeyError:
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1112 pass
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1113
17782 8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1114 def __contains__(self, fn):
40770 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1115 if fn in self.addls:
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1116 return True
52299 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1117 if not self.is_loaded:
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1118 self._load()
17782 8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1119 return fn in self.entries
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1120
03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1121 def __iter__(self):
52299 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1122 if not self.is_loaded:
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1123 self._load()
40770 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1124 return iter(self.entries \| self.addls)
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1125
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1126
41096 6498f0e03526 vfs: fix proxyvfs inheritance Boris Feld <boris.feld@octobus.net> parents: 40782 diff changeset	1127 class _fncachevfs(vfsmod.proxyvfs):
17721 cf236e3501c3 store: rename argument name from "op"(ener) to "vfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17693 diff changeset	1128 def __init__(self, vfs, fnc, encode):
33412 a42369e04aee vfs: rename auditvfs to proxyvfs Yuya Nishihara <yuya@tcha.org> parents: 32411 diff changeset	1129 vfsmod.proxyvfs.__init__(self, vfs)
52300 9fbdf355c916 fncachestore: add typing information Rapha?l Gom?s <rgomes@octobus.net> parents: 52299 diff changeset	1130 self.fncache: fncache = fnc
14194 3a90fb3addc1 store: break up reference cycle introduced in 9cbff8a39a2a Adrian Buehlmann <adrian@cadifra.com> parents: 14166 diff changeset	1131 self.encode = encode
52301 bd43465af568 fncache: add attribute to check whether we're using dotencode Rapha?l Gom?s <rgomes@octobus.net> parents: 52300 diff changeset	1132 self.uses_dotencode = encode is _pathencode
14194 3a90fb3addc1 store: break up reference cycle introduced in 9cbff8a39a2a Adrian Buehlmann <adrian@cadifra.com> parents: 14166 diff changeset	1133
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1134 def __call__(self, path, mode=b'r', args, *kw):
38661 8ac0c9cd4c48 fncache: avoid loading the filename cache when not actually modifying it Martijn Pieters <mj@zopatista.com> parents: 37409 diff changeset	1135 encoded = self.encode(path)
50501 0dbab42adca5 store: do not record file that are not part of a revlog in fncache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50485 diff changeset	1136 if (
0dbab42adca5 store: do not record file that are not part of a revlog in fncache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50485 diff changeset	1137 mode not in (b'r', b'rb')
0dbab42adca5 store: do not record file that are not part of a revlog in fncache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50485 diff changeset	1138 and (path.startswith(b'data/') or path.startswith(b'meta/'))
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	1139 and is_revlog_file(path)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1140 ):
38661 8ac0c9cd4c48 fncache: avoid loading the filename cache when not actually modifying it Martijn Pieters <mj@zopatista.com> parents: 37409 diff changeset	1141 # do not trigger a fncache load when adding a file that already is
8ac0c9cd4c48 fncache: avoid loading the filename cache when not actually modifying it Martijn Pieters <mj@zopatista.com> parents: 37409 diff changeset	1142 # known to exist.
52299 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52094 diff changeset	1143 notload = not self.fncache.is_loaded and (
50759 30381812708d fncache: fix a bug that corrupts the fncache after transaction rollback Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50706 diff changeset	1144 # if the file has size zero, it should be considered as missing.
30381812708d fncache: fix a bug that corrupts the fncache after transaction rollback Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50706 diff changeset	1145 # Such zero-size files are the result of truncation when a
30381812708d fncache: fix a bug that corrupts the fncache after transaction rollback Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50706 diff changeset	1146 # transaction is aborted.
30381812708d fncache: fix a bug that corrupts the fncache after transaction rollback Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50706 diff changeset	1147 self.vfs.exists(encoded)
30381812708d fncache: fix a bug that corrupts the fncache after transaction rollback Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50706 diff changeset	1148 and self.vfs.stat(encoded).st_size
30381812708d fncache: fix a bug that corrupts the fncache after transaction rollback Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50706 diff changeset	1149 )
38661 8ac0c9cd4c48 fncache: avoid loading the filename cache when not actually modifying it Martijn Pieters <mj@zopatista.com> parents: 37409 diff changeset	1150 if not notload:
8ac0c9cd4c48 fncache: avoid loading the filename cache when not actually modifying it Martijn Pieters <mj@zopatista.com> parents: 37409 diff changeset	1151 self.fncache.add(path)
8ac0c9cd4c48 fncache: avoid loading the filename cache when not actually modifying it Martijn Pieters <mj@zopatista.com> parents: 37409 diff changeset	1152 return self.vfs(encoded, mode, args, *kw)
14194 3a90fb3addc1 store: break up reference cycle introduced in 9cbff8a39a2a Adrian Buehlmann <adrian@cadifra.com> parents: 14166 diff changeset	1153
51919 2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	1154 def join(self, path: Optional[bytes], *insidef: bytes) -> bytes:
2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	1155 insidef = (self.encode(f) for f in insidef)
2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	1156
17725 ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17724 diff changeset	1157 if path:
51919 2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	1158 return self.vfs.join(self.encode(path), *insidef)
17725 ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17724 diff changeset	1159 else:
51919 2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	1160 return self.vfs.join(path, *insidef)
17725 ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17724 diff changeset	1161
47449 9ab54aa56982 vfs: add a `register_file` method on the vfs class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	1162 def register_file(self, path):
9ab54aa56982 vfs: add a `register_file` method on the vfs class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	1163 """generic hook point to lets fncache steer its stew"""
9ab54aa56982 vfs: add a `register_file` method on the vfs class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	1164 if path.startswith(b'data/') or path.startswith(b'meta/'):
9ab54aa56982 vfs: add a `register_file` method on the vfs class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	1165 self.fncache.add(path)
9ab54aa56982 vfs: add a `register_file` method on the vfs class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47395 diff changeset	1166
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1167
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1168 class fncachestore(basicstore):
17651 3b49c28658f6 store: rename "openertype" argument to "vfstype" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17649 diff changeset	1169 def __init__(self, path, vfstype, dotencode):
17591 9a5c2ecd1158 store: move encode lambda logic into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17590 diff changeset	1170 if dotencode:
18435 8c019d2fd7c0 store: switch to C-based hashed path encoding Bryan O'Sullivan <bryano@fb.com> parents: 18430 diff changeset	1171 encode = _pathencode
17591 9a5c2ecd1158 store: move encode lambda logic into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17590 diff changeset	1172 else:
9a5c2ecd1158 store: move encode lambda logic into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17590 diff changeset	1173 encode = _plainhybridencode
12687 34d8247a4595 store: encode first period or space in filenames (issue1713) Adrian Buehlmann <adrian@cadifra.com> parents: 12171 diff changeset	1174 self.encode = encode
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1175 vfs = vfstype(path + b'/store')
17724 bf4b72d8dd4d store: initialize vfs field first to use it for initialization of others FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17722 diff changeset	1176 self.path = vfs.base
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1177 self.pathsep = self.path + b'/'
17726 7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17725 diff changeset	1178 self.createmode = _calcmode(vfs)
17652 2c6f7231becc store: rename "op" variables to "vfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17651 diff changeset	1179 vfs.createmode = self.createmode
17727 6492b39a44d5 store: replace invocation of "getsize()" by "vfs.stat()" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17726 diff changeset	1180 self.rawvfs = vfs
17652 2c6f7231becc store: rename "op" variables to "vfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17651 diff changeset	1181 fnc = fncache(vfs)
9133 996c1cd8f530 store: eliminate reference cycle in fncachestore Simon Heimberg <simohe@besonet.ch> parents: 8778 diff changeset	1182 self.fncache = fnc
17653 dacb50696b75 store: initialize "vfs" fields by "vfs" constructors FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17652 diff changeset	1183 self.vfs = _fncachevfs(vfs, fnc, encode)
dacb50696b75 store: initialize "vfs" fields by "vfs" constructors FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17652 diff changeset	1184 self.opener = self.vfs
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1185
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1186 def join(self, f):
17562 b42b0729744d store: reduce string concatenation when joining Bryan O'Sullivan <bryano@fb.com> parents: 17555 diff changeset	1187 return self.pathsep + self.encode(f)
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1188
17731 c85dbae29684 store: restore getsize method Matt Mackall <mpm@selenic.com> parents: 17728 diff changeset	1189 def getsize(self, path):
c85dbae29684 store: restore getsize method Matt Mackall <mpm@selenic.com> parents: 17728 diff changeset	1190 return self.rawvfs.stat(path).st_size
c85dbae29684 store: restore getsize method Matt Mackall <mpm@selenic.com> parents: 17728 diff changeset	1191
50538 862e3a13da44 store: rename `datafiles` to `data_entries` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50532 diff changeset	1192 def data_entries(
50505 521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50503 diff changeset	1193 self, matcher=None, undecodable=None
50507 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50506 diff changeset	1194 ) -> Generator[BaseStoreEntry, None, None]:
50514 d4f54aded22e store: also group files by revlog in fncache version of datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50513 diff changeset	1195 # Note: all files in fncache should be revlog related, However the
d4f54aded22e store: also group files by revlog in fncache version of datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50513 diff changeset	1196 # fncache might contains such file added by previous version of
d4f54aded22e store: also group files by revlog in fncache version of datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50513 diff changeset	1197 # Mercurial.
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	1198 files = ((f, None) for f in self.fncache if is_revlog_file(f))
50514 d4f54aded22e store: also group files by revlog in fncache version of datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50513 diff changeset	1199 by_revlog = _gather_revlog(files)
d4f54aded22e store: also group files by revlog in fncache version of datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50513 diff changeset	1200 for revlog, details in by_revlog:
50516 1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50515 diff changeset	1201 if revlog.startswith(b'data/'):
50688 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	1202 rl_type = KIND_FILELOG
50517 60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50516 diff changeset	1203 revlog_target_id = revlog.split(b'/', 1)[1]
50516 1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50515 diff changeset	1204 elif revlog.startswith(b'meta/'):
50688 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50680 diff changeset	1205 rl_type = KIND_MANIFESTLOG
50517 60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50516 diff changeset	1206 # drop the initial directory and the `00manifest` file part
60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50516 diff changeset	1207 tmp = revlog.split(b'/', 1)[1]
60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50516 diff changeset	1208 revlog_target_id = tmp.rsplit(b'/', 1)[0] + b'/'
50516 1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50515 diff changeset	1209 else:
1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50515 diff changeset	1210 # unreachable
1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50515 diff changeset	1211 assert False, revlog
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	1212 entry = RevlogStoreEntry(
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	1213 path_prefix=revlog,
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	1214 revlog_type=rl_type,
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	1215 target_id=revlog_target_id,
50679 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50678 diff changeset	1216 details=details,
50530 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	1217 )
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	1218 if _match_tracked_entry(entry, matcher):
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50529 diff changeset	1219 yield entry
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1220
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1221 def copylist(self):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1222 d = (
45372 909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1223 b'bookmarks',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1224 b'narrowspec',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1225 b'data',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1226 b'meta',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1227 b'dh',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1228 b'fncache',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1229 b'phaseroots',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1230 b'obsstore',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1231 b'00manifest.d',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1232 b'00manifest.i',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1233 b'00changelog.d',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1234 b'00changelog.i',
45494 d252f51ab032 share: introduce config option to store requires in .hg/store Pulkit Goyal <7895pulkit@gmail.com> parents: 45372 diff changeset	1235 b'requires',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42911 diff changeset	1236 )
45372 909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44470 diff changeset	1237 return [b'requires', b'00changelog.i'] + [b'store/' + f for f in d]
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1238
20883 cd443c7589cc fncache: move fncache writing to be in a transaction Durham Goode <durham@fb.com> parents: 20879 diff changeset	1239 def write(self, tr):
cd443c7589cc fncache: move fncache writing to be in a transaction Durham Goode <durham@fb.com> parents: 20879 diff changeset	1240 self.fncache.write(tr)
13391 d00bbff8600e fncachestore: defer updating the fncache file to a single file open Adrian Buehlmann <adrian@cadifra.com> parents: 13169 diff changeset	1241
20884 2efdd186925d caches: invalidate store caches when lock is taken Durham Goode <durham@fb.com> parents: 20883 diff changeset	1242 def invalidatecaches(self):
2efdd186925d caches: invalidate store caches when lock is taken Durham Goode <durham@fb.com> parents: 20883 diff changeset	1243 self.fncache.entries = None
40770 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40668 diff changeset	1244 self.fncache.addls = set()
20884 2efdd186925d caches: invalidate store caches when lock is taken Durham Goode <durham@fb.com> parents: 20883 diff changeset	1245
20885 f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1246 def markremoved(self, fn):
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1247 self.fncache.remove(fn)
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1248
17783 df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1249 def _exists(self, f):
df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1250 ef = self.encode(f)
df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1251 try:
df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1252 self.getsize(ef)
df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1253 return True
49314 2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT Manuel Jacob <me@manueljacob.de> parents: 49292 diff changeset	1254 except FileNotFoundError:
17783 df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1255 return False
df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1256
17745 b9a56b816ff2 store: add a contains method to fncachestore smuralid parents: 17744 diff changeset	1257 def __contains__(self, path):
b9a56b816ff2 store: add a contains method to fncachestore smuralid parents: 17744 diff changeset	1258 '''Checks if the store contains path'''
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1259 path = b"/".join((b"data", path))
17782 8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1260 # check for files (exact match)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1261 e = path + b'.i'
17784 73e1ab39792c store: fncache may contain non-existent entries (fixes b9a56b816ff2) Adrian Buehlmann <adrian@cadifra.com> parents: 17783 diff changeset	1262 if e in self.fncache and self._exists(e):
17782 8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1263 return True
8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1264 # now check for directories (prefix match)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1265 if not path.endswith(b'/'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1266 path += b'/'
17782 8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1267 for e in self.fncache:
17784 73e1ab39792c store: fncache may contain non-existent entries (fixes b9a56b816ff2) Adrian Buehlmann <adrian@cadifra.com> parents: 17783 diff changeset	1268 if e.startswith(path) and self._exists(e):
17782 8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1269 return True
8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1270 return False

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/store.py @ 52710:45dc0f874b8c