mercurial-scm/hg: mercurial/store.py annotate

annotate mercurial/store.py @ 52639:9db77d46de79

py3: drop redundant `u''` prefixes on string literals Strings are unicode on Python 3. These were rewritten by `pyupgrade`. It's arguably better to fix the `contrib` stuff upstream and then re-vendor it, but I don't feel like waiting for that, and then all of the regression testing involved to get a minor improvement in the codebase. It was last vendored 5 years ago, and will likely be a large change anyway to drop py2 support. Also, we've already made minor formatting changes to it locally.

author	Matt Harbison <matt_harbison@yahoo.com>
date	Mon, 06 Jan 2025 14:15:40 -0500
parents	3f0cf7bb3086
children	24ee91ba9aa8

rev	line source
50677 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	1 # store.py - repository store handling for Mercurial)
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	2 #
46819 d4ba4d51f85f contributor: change mentions of mpm to olivia Rapha?l Gom?s <rgomes@octobus.net> parents: 46780 diff changeset	3 # Copyright 2008 Olivia Mackall <olivia@selenic.com>
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	4 #
8225 46293a0c7e9f updated license to be explicit about GPL version 2 Martin Geisler <mg@lazybytes.net> parents: 8210 diff changeset	5 # This software may be used and distributed according to the terms of the
10263 25e572394f5c Update license to GPLv2+ Matt Mackall <mpm@selenic.com> parents: 9133 diff changeset	6 # GNU General Public License version 2 or any later version.
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	7
51859 f4733654f144 typing: add `from __future__ import annotations` to most files Matt Harbison <matt_harbison@yahoo.com> parents: 51822 diff changeset	8 from __future__ import annotations
f4733654f144 typing: add `from __future__ import annotations` to most files Matt Harbison <matt_harbison@yahoo.com> parents: 51822 diff changeset	9
50476 5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	10 import collections
41978 a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41973 diff changeset	11 import functools
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	12 import os
46990 0b569c75d180 store: exclude `undo.` nodemap's file from `walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46989 diff changeset	13 import re
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	14 import stat
51725 278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51724 diff changeset	15 import typing
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51724 diff changeset	16
51878 2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51859 diff changeset	17 from typing import (
2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51859 diff changeset	18 Generator,
2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51859 diff changeset	19 List,
2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51859 diff changeset	20 Optional,
2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51859 diff changeset	21 )
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	22
509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	23 from .i18n import _
50468 521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50466 diff changeset	24 from .thirdparty import attr
51725 278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51724 diff changeset	25
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51724 diff changeset	26 # Force pytype to use the non-vendored package
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51724 diff changeset	27 if typing.TYPE_CHECKING:
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51724 diff changeset	28 # noinspection PyPackageRequirements
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51724 diff changeset	29 import attr
278af66e6595 typing: induce pytype to use the standard `attr` instead of the vendored copy Matt Harbison <matt_harbison@yahoo.com> parents: 51724 diff changeset	30
46113 59fa3890d40a node: import symbols explicitly Joerg Sonnenberger <joerg@bec.de> parents: 45942 diff changeset	31 from .node import hex
50669 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	32 from .revlogutils.constants import (
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	33 INDEX_HEADER,
50677 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	34 KIND_CHANGELOG,
e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	35 KIND_FILELOG,
e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	36 KIND_MANIFESTLOG,
50669 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	37 )
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	38 from . import (
42905 3df3b139a43d localrepo: push manifestlog and changelog construction code into store Augie Fackler <augie@google.com> parents: 42715 diff changeset	39 changelog,
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	40 error,
50624 e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50511 diff changeset	41 filelog,
42905 3df3b139a43d localrepo: push manifestlog and changelog construction code into store Augie Fackler <augie@google.com> parents: 42715 diff changeset	42 manifest,
32372 df448de7cf3b parsers: switch to policy importer Yuya Nishihara <yuya@tcha.org> parents: 31362 diff changeset	43 policy,
30077 8f42d8c412c8 py3: make encodefun in store.py compatible with py3k Mateusz Kwapich <mitrandir@fb.com> parents: 30076 diff changeset	44 pycompat,
50669 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	45 revlog as revlogmod,
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	46 util,
31234 9b7a2ef4f27c vfs: use 'vfs' module directly in 'mercurial.store' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31219 diff changeset	47 vfs as vfsmod,
27480 509159675cdb store: use absolute_import Gregory Szorc <gregory.szorc@gmail.com> parents: 26778 diff changeset	48 )
44060 a61287a95dc3 core: migrate uses of hashlib.sha1 to hashutil.sha1 Augie Fackler <augie@google.com> parents: 43506 diff changeset	49 from .utils import hashutil
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	50
43506 9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings Augie Fackler <augie@google.com> parents: 43106 diff changeset	51 parsers = policy.importmod('parsers')
41978 a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41973 diff changeset	52 # how much bytes should be read from fncache in one read
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41973 diff changeset	53 # It is done to prevent loading large fncache files into memory
51699 ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51696 diff changeset	54 fncache_chunksize = 10**6
32372 df448de7cf3b parsers: switch to policy importer Yuya Nishihara <yuya@tcha.org> parents: 31362 diff changeset	55
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	56
51724 2e9e62242451 typing: disable some pytype errors in `mercurial.store` Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	57 def _match_tracked_entry(entry: "BaseStoreEntry", matcher):
40494 9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	58 """parses a fncache entry and returns whether the entry is tracking a path
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	59 matched by matcher or not.
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	60
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	61 If matcher is None, returns True"""
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	62
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	63 if matcher is None:
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files Pulkit Goyal <pulkit@yandex-team.ru> parents: 40340 diff changeset	64 return True
51724 2e9e62242451 typing: disable some pytype errors in `mercurial.store` Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	65
2e9e62242451 typing: disable some pytype errors in `mercurial.store` Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	66 # TODO: make this safe for other entry types. Currently, the various
2e9e62242451 typing: disable some pytype errors in `mercurial.store` Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	67 # store.data_entry generators only yield RevlogStoreEntry, so the
2e9e62242451 typing: disable some pytype errors in `mercurial.store` Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	68 # attributes do exist on `entry`.
2e9e62242451 typing: disable some pytype errors in `mercurial.store` Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	69 # pytype: disable=attribute-error
50495 1b776f25302f store: use the boolean property in `store` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50494 diff changeset	70 if entry.is_filelog:
50491 b4953fad744e store: do the revlog matching on entry directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50490 diff changeset	71 return matcher(entry.target_id)
50495 1b776f25302f store: use the boolean property in `store` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50494 diff changeset	72 elif entry.is_manifestlog:
50491 b4953fad744e store: do the revlog matching on entry directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50490 diff changeset	73 return matcher.visitdir(entry.target_id.rstrip(b'/'))
51724 2e9e62242451 typing: disable some pytype errors in `mercurial.store` Matt Harbison <matt_harbison@yahoo.com> parents: 51699 diff changeset	74 # pytype: enable=attribute-error
50491 b4953fad744e store: do the revlog matching on entry directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50490 diff changeset	75 raise error.ProgrammingError(b"cannot process entry %r" % entry)
40624 66adfd58cb77 store: raise ProgrammingError if unable to decode a storage path Pulkit Goyal <pulkit@yandex-team.ru> parents: 40584 diff changeset	76
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	77
8531 810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	78 # This avoids a collision between a file named foo and a dir named
810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	79 # foo.i or foo.d
17607 cc58dc47cb5e store: use fast C implementation of encodedir() if it's available Adrian Buehlmann <adrian@cadifra.com> parents: 17605 diff changeset	80 def _encodedir(path):
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	81 """
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	82 >>> _encodedir(b'data/foo.i')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	83 'data/foo.i'
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	84 >>> _encodedir(b'data/foo.i/bla.i')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	85 'data/foo.i.hg/bla.i'
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	86 >>> _encodedir(b'data/foo.i.hg/bla.i')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	87 'data/foo.i.hg.hg/bla.i'
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	88 >>> _encodedir(b'data/foo.i\\ndata/foo.i/bla.i\\ndata/foo.i.hg/bla.i\\n')
17605 e9cc29be3305 store: add multiline doctest case for encodedir() Adrian Buehlmann <adrian@cadifra.com> parents: 17604 diff changeset	89 'data/foo.i\\ndata/foo.i.hg/bla.i\\ndata/foo.i.hg.hg/bla.i\\n'
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	90 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	91 return (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	92 path.replace(b".hg/", b".hg.hg/")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	93 .replace(b".i/", b".i.hg/")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	94 .replace(b".d/", b".d.hg/")
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	95 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	96
8531 810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	97
17607 cc58dc47cb5e store: use fast C implementation of encodedir() if it's available Adrian Buehlmann <adrian@cadifra.com> parents: 17605 diff changeset	98 encodedir = getattr(parsers, 'encodedir', _encodedir)
cc58dc47cb5e store: use fast C implementation of encodedir() if it's available Adrian Buehlmann <adrian@cadifra.com> parents: 17605 diff changeset	99
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	100
8531 810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	101 def decodedir(path):
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	102 """
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	103 >>> decodedir(b'data/foo.i')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	104 'data/foo.i'
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	105 >>> decodedir(b'data/foo.i.hg/bla.i')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	106 'data/foo.i/bla.i'
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	107 >>> decodedir(b'data/foo.i.hg.hg/bla.i')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	108 'data/foo.i.hg/bla.i'
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	109 """
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	110 if b".hg/" not in path:
8531 810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	111 return path
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	112 return (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	113 path.replace(b".d.hg/", b".d/")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	114 .replace(b".i.hg/", b".i/")
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	115 .replace(b".hg.hg/", b".hg/")
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	116 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	117
8531 810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	118
29071 2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	119 def _reserved():
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	120 """characters that are problematic for filesystems
29071 2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	121
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	122 * ascii escapes (0..31)
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	123 * ascii hi (126..255)
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	124 * windows specials
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	125
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	126 these characters will be escaped by encodefunctions
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	127 """
52639 9db77d46de79 py3: drop redundant `u''` prefixes on string literals Matt Harbison <matt_harbison@yahoo.com> parents: 52359 diff changeset	128 winreserved = [ord(x) for x in '\\:*?"<>\|']
29071 2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	129 for x in range(32):
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	130 yield x
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	131 for x in range(126, 256):
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	132 yield x
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	133 for x in winreserved:
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	134 yield x
2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	135
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	136
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	137 def _buildencodefun():
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	138 """
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	139 >>> enc, dec = _buildencodefun()
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	140
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	141 >>> enc(b'nothing/special.txt')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	142 'nothing/special.txt'
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	143 >>> dec(b'nothing/special.txt')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	144 'nothing/special.txt'
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	145
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	146 >>> enc(b'HELLO')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	147 '_h_e_l_l_o'
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	148 >>> dec(b'_h_e_l_l_o')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	149 'HELLO'
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	150
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	151 >>> enc(b'hello:world?')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	152 'hello~3aworld~3f'
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	153 >>> dec(b'hello~3aworld~3f')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	154 'hello:world?'
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	155
34136 414a3513c2bd doctest: do not embed non-ascii characters in docstring Yuya Nishihara <yuya@tcha.org> parents: 34131 diff changeset	156 >>> enc(b'the\\x07quick\\xADshot')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	157 'the~07quick~adshot'
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	158 >>> dec(b'the~07quick~adshot')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	159 'the\\x07quick\\xadshot'
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	160 """
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	161 e = b'_'
31253 64596338ba10 py3: factor out bytechr() function Yuya Nishihara <yuya@tcha.org> parents: 31234 diff changeset	162 xchr = pycompat.bytechr
64596338ba10 py3: factor out bytechr() function Yuya Nishihara <yuya@tcha.org> parents: 31234 diff changeset	163 asciistr = list(map(xchr, range(127)))
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	164 capitals = list(range(ord(b"A"), ord(b"Z") + 1))
30077 8f42d8c412c8 py3: make encodefun in store.py compatible with py3k Mateusz Kwapich <mitrandir@fb.com> parents: 30076 diff changeset	165
44452 9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things Augie Fackler <augie@google.com> parents: 44060 diff changeset	166 cmap = {x: x for x in asciistr}
29071 2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	167 for x in _reserved():
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	168 cmap[xchr(x)] = b"~%02x" % x
30077 8f42d8c412c8 py3: make encodefun in store.py compatible with py3k Mateusz Kwapich <mitrandir@fb.com> parents: 30076 diff changeset	169 for x in capitals + [ord(e)]:
8f42d8c412c8 py3: make encodefun in store.py compatible with py3k Mateusz Kwapich <mitrandir@fb.com> parents: 30076 diff changeset	170 cmap[xchr(x)] = e + xchr(x).lower()
8f42d8c412c8 py3: make encodefun in store.py compatible with py3k Mateusz Kwapich <mitrandir@fb.com> parents: 30076 diff changeset	171
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	172 dmap = {}
48913 f254fc73d956 global: bulk replace simple pycompat.iteritems(x) with x.items() Gregory Szorc <gregory.szorc@gmail.com> parents: 48875 diff changeset	173 for k, v in cmap.items():
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	174 dmap[v] = k
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	175
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	176 def decode(s):
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	177 i = 0
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	178 while i < len(s):
49284 d44e3c45f0e4 py3: replace `pycompat.xrange` by `range` Manuel Jacob <me@manueljacob.de> parents: 48946 diff changeset	179 for l in range(1, 4):
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	180 try:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	181 yield dmap[s[i : i + l]]
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	182 i += l
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	183 break
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	184 except KeyError:
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	185 pass
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	186 else:
01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	187 raise KeyError
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	188
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	189 return (
49284 d44e3c45f0e4 py3: replace `pycompat.xrange` by `range` Manuel Jacob <me@manueljacob.de> parents: 48946 diff changeset	190 lambda s: b''.join([cmap[s[c : c + 1]] for c in range(len(s))]),
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	191 lambda s: b''.join(list(decode(s))),
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	192 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	193
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	194
776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	195 _encodefname, _decodefname = _buildencodefun()
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	196
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	197
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	198 def encodefilename(s):
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	199 """
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	200 >>> encodefilename(b'foo.i/bar.d/bla.hg/hi:world?/HELLO')
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	201 'foo.i.hg/bar.d.hg/bla.hg.hg/hi~3aworld~3f/_h_e_l_l_o'
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	202 """
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	203 return _encodefname(encodedir(s))
776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	204
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	205
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	206 def decodefilename(s):
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	207 """
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	208 >>> decodefilename(b'foo.i.hg/bar.d.hg/bla.hg.hg/hi~3aworld~3f/_h_e_l_l_o')
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	209 'foo.i/bar.d/bla.hg/hi:world?/HELLO'
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	210 """
17608 776240123525 store: extract functions _encodefname and _decodefname Adrian Buehlmann <adrian@cadifra.com> parents: 17607 diff changeset	211 return decodedir(_decodefname(s))
6839 01db3e101362 move filename encoding functions from util.py to new store.py Adrian Buehlmann <adrian@cadifra.com> parents: diff changeset	212
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	213
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	214 def _buildlowerencodefun():
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	215 """
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	216 >>> f = _buildlowerencodefun()
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	217 >>> f(b'nothing/special.txt')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	218 'nothing/special.txt'
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	219 >>> f(b'HELLO')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	220 'hello'
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	221 >>> f(b'hello:world?')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	222 'hello~3aworld~3f'
34136 414a3513c2bd doctest: do not embed non-ascii characters in docstring Yuya Nishihara <yuya@tcha.org> parents: 34131 diff changeset	223 >>> f(b'the\\x07quick\\xADshot')
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	224 'the~07quick~adshot'
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	225 """
34214 7e3f078b6f31 py3: use bytechr() in store._buildlowerencodefun() Yuya Nishihara <yuya@tcha.org> parents: 34213 diff changeset	226 xchr = pycompat.bytechr
49284 d44e3c45f0e4 py3: replace `pycompat.xrange` by `range` Manuel Jacob <me@manueljacob.de> parents: 48946 diff changeset	227 cmap = {xchr(x): xchr(x) for x in range(127)}
29071 2f58975eb2cb store: treat range as a generator instead of a list for py3 compat timeless <timeless@mozdev.org> parents: 28007 diff changeset	228 for x in _reserved():
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	229 cmap[xchr(x)] = b"~%02x" % x
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	230 for x in range(ord(b"A"), ord(b"Z") + 1):
34214 7e3f078b6f31 py3: use bytechr() in store._buildlowerencodefun() Yuya Nishihara <yuya@tcha.org> parents: 34213 diff changeset	231 cmap[xchr(x)] = xchr(x).lower()
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	232
34213 96808804b68f store: give name to lowerencode function Yuya Nishihara <yuya@tcha.org> parents: 34136 diff changeset	233 def lowerencode(s):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	234 return b"".join([cmap[c] for c in pycompat.iterbytestr(s)])
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	235
34213 96808804b68f store: give name to lowerencode function Yuya Nishihara <yuya@tcha.org> parents: 34136 diff changeset	236 return lowerencode
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	237
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	238
18430 0459c6555f69 store: implement lowerencode in C Bryan O'Sullivan <bryano@fb.com> parents: 18054 diff changeset	239 lowerencode = getattr(parsers, 'lowerencode', None) or _buildlowerencodefun()
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	240
17570 f53a7b256ca6 store: optimze _auxencode() a bit by grouping the reserved names by length Adrian Buehlmann <adrian@cadifra.com> parents: 17569 diff changeset	241 # Windows reserved names: con, prn, aux, nul, com1..com9, lpt1..lpt9
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	242 _winres3 = (b'aux', b'con', b'prn', b'nul') # length 3
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	243 _winres4 = (b'com', b'lpt') # length 4 (with trailing 1..9)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	244
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	245
12687 34d8247a4595 store: encode first period or space in filenames (issue1713) Adrian Buehlmann <adrian@cadifra.com> parents: 12171 diff changeset	246 def _auxencode(path, dotencode):
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	247 """
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	248 Encodes filenames containing names reserved by Windows or which end in
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	249 period or space. Does not touch other single reserved characters c.
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	250 Specifically, c in '\\:?"<>\|' or ord(c) <= 31 are not* encoded here.
ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	251 Additionally encodes space or period at the beginning, if dotencode is
17569 e9af2134825c store: explain "aux.foo" versus "foo.aux" in doc of _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17568 diff changeset	252 True. Parameter path is assumed to be all lowercase.
e9af2134825c store: explain "aux.foo" versus "foo.aux" in doc of _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17568 diff changeset	253 A segment only needs encoding if a reserved name appears as a
e9af2134825c store: explain "aux.foo" versus "foo.aux" in doc of _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17568 diff changeset	254 basename (e.g. "aux", "aux.foo"). A directory or file named "foo.aux"
e9af2134825c store: explain "aux.foo" versus "foo.aux" in doc of _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17568 diff changeset	255 doesn't need encoding.
13949 ba43aa1e173c store: add some doctests Adrian Buehlmann <adrian@cadifra.com> parents: 13426 diff changeset	256
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	257 >>> s = b'.foo/aux.txt/txt.aux/con/prn/nul/foo.'
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	258 >>> _auxencode(s.split(b'/'), True)
17574 81a033bb29bc store: let _auxencode() return the list of path segments Adrian Buehlmann <adrian@cadifra.com> parents: 17573 diff changeset	259 ['~2efoo', 'au~78.txt', 'txt.aux', 'co~6e', 'pr~6e', 'nu~6c', 'foo~2e']
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	260 >>> s = b'.com1com2/lpt9.lpt4.lpt1/conprn/com0/lpt0/foo.'
0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	261 >>> _auxencode(s.split(b'/'), False)
17574 81a033bb29bc store: let _auxencode() return the list of path segments Adrian Buehlmann <adrian@cadifra.com> parents: 17573 diff changeset	262 ['.com1com2', 'lp~749.lpt4.lpt1', 'conprn', 'com0', 'lpt0', 'foo~2e']
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	263 >>> _auxencode([b'foo. '], True)
17574 81a033bb29bc store: let _auxencode() return the list of path segments Adrian Buehlmann <adrian@cadifra.com> parents: 17573 diff changeset	264 ['foo.~20']
34131 0fa781320203 doctest: bulk-replace string literals with b'' for Python 3 Yuya Nishihara <yuya@tcha.org> parents: 33412 diff changeset	265 >>> _auxencode([b' .foo'], True)
17574 81a033bb29bc store: let _auxencode() return the list of path segments Adrian Buehlmann <adrian@cadifra.com> parents: 17573 diff changeset	266 ['~20.foo']
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	267 """
17589 b11024849db6 store: parameter path of _auxencode is now a list of strings Adrian Buehlmann <adrian@cadifra.com> parents: 17588 diff changeset	268 for i, n in enumerate(path):
17572 b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	269 if not n:
b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	270 continue
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	271 if dotencode and n[0] in b'. ':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	272 n = b"~%02x" % ord(n[0:1]) + n[1:]
17589 b11024849db6 store: parameter path of _auxencode is now a list of strings Adrian Buehlmann <adrian@cadifra.com> parents: 17588 diff changeset	273 path[i] = n
17572 b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	274 else:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	275 l = n.find(b'.')
17572 b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	276 if l == -1:
b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	277 l = len(n)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	278 if (l == 3 and n[:3] in _winres3) or (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	279 l == 4
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	280 and n[3:4] <= b'9'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	281 and n[3:4] >= b'1'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	282 and n[:3] in _winres4
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	283 ):
17572 b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	284 # encode third letter ('aux' -> 'au~78')
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	285 ec = b"~%02x" % ord(n[2:3])
17572 b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	286 n = n[0:2] + ec + n[3:]
17589 b11024849db6 store: parameter path of _auxencode is now a list of strings Adrian Buehlmann <adrian@cadifra.com> parents: 17588 diff changeset	287 path[i] = n
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	288 if n[-1] in b'. ':
17572 b644287e79a8 store: unindent most of the contents of the for loop in _auxencode() Adrian Buehlmann <adrian@cadifra.com> parents: 17571 diff changeset	289 # encode last period or space ('foo...' -> 'foo..~2e')
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	290 path[i] = n[:-1] + b"~%02x" % ord(n[-1:])
17589 b11024849db6 store: parameter path of _auxencode is now a list of strings Adrian Buehlmann <adrian@cadifra.com> parents: 17588 diff changeset	291 return path
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	292
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	293
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	294 _maxstorepathlen = 120
00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	295 _dirprefixlen = 8
00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	296 _maxshortdirslen = 8 * (_dirprefixlen + 1) - 4
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	297
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	298
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	299 def _hashencode(path, dotencode):
46113 59fa3890d40a node: import symbols explicitly Joerg Sonnenberger <joerg@bec.de> parents: 45942 diff changeset	300 digest = hex(hashutil.sha1(path).digest())
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	301 le = lowerencode(path[5:]).split(b'/') # skips prefix 'data/' or 'meta/'
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	302 parts = _auxencode(le, dotencode)
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	303 basename = parts[-1]
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	304 _root, ext = os.path.splitext(basename)
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	305 sdirs = []
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	306 sdirslen = 0
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	307 for p in parts[:-1]:
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	308 d = p[:_dirprefixlen]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	309 if d[-1] in b'. ':
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	310 # Windows can't access dirs ending in period or space
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	311 d = d[:-1] + b'_'
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	312 if sdirslen == 0:
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	313 t = len(d)
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	314 else:
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	315 t = sdirslen + 1 + len(d)
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	316 if t > _maxshortdirslen:
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	317 break
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	318 sdirs.append(d)
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	319 sdirslen = t
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	320 dirs = b'/'.join(sdirs)
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	321 if len(dirs) > 0:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	322 dirs += b'/'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	323 res = b'dh/' + dirs + digest + ext
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	324 spaceleft = _maxstorepathlen - len(res)
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	325 if spaceleft > 0:
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	326 filler = basename[:spaceleft]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	327 res = b'dh/' + dirs + filler + digest + ext
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	328 return res
d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	329
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	330
17590 eb0884680f5c store: eliminate one level of lambda functions on _hybridencode Adrian Buehlmann <adrian@cadifra.com> parents: 17589 diff changeset	331 def _hybridencode(path, dotencode):
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	332 """encodes path with a length limit
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	333
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	334 Encodes all paths that begin with 'data/', according to the following.
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	335
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	336 Default encoding (reversible):
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	337
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	338 Encodes all uppercase letters 'X' as '_x'. All reserved or illegal
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	339 characters are encoded as '~xx', where xx is the two digit hex code
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	340 of the character (see encodefilename).
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	341 Relevant path components consisting of Windows reserved filenames are
17738 b8424c92ba2b spelling: fix minor spell checker issues Mads Kiilerich <mads@kiilerich.com> parents: 17731 diff changeset	342 masked by encoding the third character ('aux' -> 'au~78', see _auxencode).
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	343
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	344 Hashed encoding (not reversible):
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	345
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	346 If the default-encoded path is longer than _maxstorepathlen, a
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	347 non-reversible hybrid hashing of the path is done instead.
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	348 This encoding uses up to _dirprefixlen characters of all directory
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	349 levels of the lowerencoded path, but not more levels than can fit into
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	350 _maxshortdirslen.
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	351 Then follows the filler followed by the sha digest of the full path.
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	352 The filler is the beginning of the basename of the lowerencoded path
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	353 (the basename is everything after the last path separator). The filler
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	354 is as long as possible, filling in characters from the basename until
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	355 the encoded path has _maxstorepathlen characters (or all chars of the
00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	356 basename have been taken).
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	357 The extension (e.g. '.i' or '.d') is preserved.
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	358
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	359 The string 'data/' at the beginning is replaced with 'dh/', if the hashed
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	360 encoding was used.
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	361 """
17609 cbc180cfd60b store: reuse direncoded path in _hybridencode Adrian Buehlmann <adrian@cadifra.com> parents: 17608 diff changeset	362 path = encodedir(path)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	363 ef = _encodefname(path).split(b'/')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	364 res = b'/'.join(_auxencode(ef, dotencode))
14288 00a0ab08f986 store: change names to comply with project coding standards Adrian Buehlmann <adrian@cadifra.com> parents: 14194 diff changeset	365 if len(res) > _maxstorepathlen:
17610 d0afa149e059 store: refactor hashed encoding into its own function Bryan O'Sullivan <bryano@fb.com> parents: 17609 diff changeset	366 res = _hashencode(path, dotencode)
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	367 return res
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	368
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	369
17624 ae103510f6aa store: add a fallback _pathencode Python function Adrian Buehlmann <adrian@cadifra.com> parents: 17623 diff changeset	370 def _pathencode(path):
18435 8c019d2fd7c0 store: switch to C-based hashed path encoding Bryan O'Sullivan <bryano@fb.com> parents: 18430 diff changeset	371 de = encodedir(path)
17693 0c6de45e1212 store: optimize _pathencode by checking the length of the unencoded path Adrian Buehlmann <adrian@cadifra.com> parents: 17653 diff changeset	372 if len(path) > _maxstorepathlen:
18435 8c019d2fd7c0 store: switch to C-based hashed path encoding Bryan O'Sullivan <bryano@fb.com> parents: 18430 diff changeset	373 return _hashencode(de, True)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	374 ef = _encodefname(de).split(b'/')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	375 res = b'/'.join(_auxencode(ef, True))
17624 ae103510f6aa store: add a fallback _pathencode Python function Adrian Buehlmann <adrian@cadifra.com> parents: 17623 diff changeset	376 if len(res) > _maxstorepathlen:
18435 8c019d2fd7c0 store: switch to C-based hashed path encoding Bryan O'Sullivan <bryano@fb.com> parents: 18430 diff changeset	377 return _hashencode(de, True)
17624 ae103510f6aa store: add a fallback _pathencode Python function Adrian Buehlmann <adrian@cadifra.com> parents: 17623 diff changeset	378 return res
ae103510f6aa store: add a fallback _pathencode Python function Adrian Buehlmann <adrian@cadifra.com> parents: 17623 diff changeset	379
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	380
17624 ae103510f6aa store: add a fallback _pathencode Python function Adrian Buehlmann <adrian@cadifra.com> parents: 17623 diff changeset	381 _pathencode = getattr(parsers, 'pathencode', _pathencode)
ae103510f6aa store: add a fallback _pathencode Python function Adrian Buehlmann <adrian@cadifra.com> parents: 17623 diff changeset	382
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	383
17623 448e6ed7c557 store: move _plainhybridencode and _dothybridencode higher up in the file Adrian Buehlmann <adrian@cadifra.com> parents: 17621 diff changeset	384 def _plainhybridencode(f):
448e6ed7c557 store: move _plainhybridencode and _dothybridencode higher up in the file Adrian Buehlmann <adrian@cadifra.com> parents: 17621 diff changeset	385 return _hybridencode(f, False)
448e6ed7c557 store: move _plainhybridencode and _dothybridencode higher up in the file Adrian Buehlmann <adrian@cadifra.com> parents: 17621 diff changeset	386
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	387
17726 7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17725 diff changeset	388 def _calcmode(vfs):
6898 69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	389 try:
69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	390 # files in .hg/ will be created using this mode
17726 7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17725 diff changeset	391 mode = vfs.stat().st_mode
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	392 # avoid some useless chmods
25658 e93036747902 global: mass rewrite to use modern octal syntax Gregory Szorc <gregory.szorc@gmail.com> parents: 25091 diff changeset	393 if (0o777 & ~util.umask) == (0o777 & mode):
6898 69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	394 mode = None
69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	395 except OSError:
69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	396 mode = None
69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	397 return mode
69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	398
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	399
45351 909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	400 _data = [
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	401 b'bookmarks',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	402 b'narrowspec',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	403 b'data',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	404 b'meta',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	405 b'00manifest.d',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	406 b'00manifest.i',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	407 b'00changelog.d',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	408 b'00changelog.i',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	409 b'phaseroots',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	410 b'obsstore',
45483 d252f51ab032 share: introduce config option to store requires in .hg/store Pulkit Goyal <7895pulkit@gmail.com> parents: 45351 diff changeset	411 b'requires',
45351 909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	412 ]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	413
50668 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	414 REVLOG_FILES_EXT = (
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	415 b'.i',
47324 0a3fa41fa719 revlogv2: use a unique filename for data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47234 diff changeset	416 b'.idx',
0a3fa41fa719 revlogv2: use a unique filename for data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47234 diff changeset	417 b'.d',
0a3fa41fa719 revlogv2: use a unique filename for data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47234 diff changeset	418 b'.dat',
0a3fa41fa719 revlogv2: use a unique filename for data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47234 diff changeset	419 b'.n',
0a3fa41fa719 revlogv2: use a unique filename for data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47234 diff changeset	420 b'.nd',
47389 e6292eb33384 revlog: store sidedata in their own file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47324 diff changeset	421 b'.sda',
47324 0a3fa41fa719 revlogv2: use a unique filename for data Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47234 diff changeset	422 )
50476 5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	423 # file extension that also use a `-SOMELONGIDHASH.ext` form
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	424 REVLOG_FILES_LONG_EXT = (
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	425 b'.nd',
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	426 b'.idx',
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	427 b'.dat',
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	428 b'.sda',
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	429 )
46989 aed6ceaad6d7 streamclone: treat volatile file as "fullfile" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	430 # files that are "volatile" and might change between listing and streaming
aed6ceaad6d7 streamclone: treat volatile file as "fullfile" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	431 #
aed6ceaad6d7 streamclone: treat volatile file as "fullfile" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	432 # note: the ".nd" file are nodemap data and won't "change" but they might be
aed6ceaad6d7 streamclone: treat volatile file as "fullfile" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	433 # deleted.
aed6ceaad6d7 streamclone: treat volatile file as "fullfile" Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46895 diff changeset	434 REVLOG_FILES_VOLATILE_EXT = (b'.n', b'.nd')
46895 6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	435
46990 0b569c75d180 store: exclude `undo.` nodemap's file from `walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46989 diff changeset	436 # some exception to the above matching
47659 f030c7d22032 walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47443 diff changeset	437 #
f030c7d22032 walk: no longer ignore revlogs of files starting with `undo.` (issue6542) Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47443 diff changeset	438 # XXX This is currently not in use because of issue6542
50319 4be9ecc982e1 py3: fix for Python 3.12 emitting SyntaxWarning on invalid escape sequences Mads Kiilerich <mads@kiilerich.com> parents: 50315 diff changeset	439 EXCLUDED = re.compile(br'.*undo\.[^/]+\.(nd?\|i)$')
46990 0b569c75d180 store: exclude `undo.` nodemap's file from `walk` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46989 diff changeset	440
46895 6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	441
6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	442 def is_revlog(f, kind, st):
6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	443 if kind != stat.S_IFREG:
50668 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	444 return False
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	445 if f.endswith(REVLOG_FILES_EXT):
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	446 return True
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	447 return False
46895 6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	448
6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	449
50668 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	450 def is_revlog_file(f):
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	451 if f.endswith(REVLOG_FILES_EXT):
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	452 return True
e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	453 return False
46383 374d7fff7cb5 store: use `endswith` to detect revlog extension Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46322 diff changeset	454
6903 0642d9d7ec80 clone: get a list of files to clone from store Matt Mackall <mpm@selenic.com> parents: 6902 diff changeset	455
50635 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	456 @attr.s(slots=True)
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	457 class StoreFile:
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	458 """a file matching a store entry"""
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	459
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	460 unencoded_path = attr.ib()
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	461 _file_size = attr.ib(default=None)
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	462 is_volatile = attr.ib(default=False)
52359 3f0cf7bb3086 stream: preserve volatile cache early Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52357 diff changeset	463 # Missing file can be safely ignored, used by "copy/hardlink" local clone
3f0cf7bb3086 stream: preserve volatile cache early Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52357 diff changeset	464 # for cache file not covered by lock.
3f0cf7bb3086 stream: preserve volatile cache early Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52357 diff changeset	465 optional = False
50635 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	466
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	467 def file_size(self, vfs):
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	468 if self._file_size is None:
50636 5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	469 if vfs is None:
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	470 msg = b"calling vfs-less file_size without prior call: %s"
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	471 msg %= self.unencoded_path
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	472 raise error.ProgrammingError(msg)
50635 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	473 try:
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	474 self._file_size = vfs.stat(self.unencoded_path).st_size
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	475 except FileNotFoundError:
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	476 self._file_size = 0
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	477 return self._file_size
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	478
51544 8e2a68e10f5c stream: in v3, skip the "size" fast path if the entries as some unknown size Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	479 @property
8e2a68e10f5c stream: in v3, skip the "size" fast path if the entries as some unknown size Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	480 def has_size(self):
8e2a68e10f5c stream: in v3, skip the "size" fast path if the entries as some unknown size Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	481 return self._file_size is not None
8e2a68e10f5c stream: in v3, skip the "size" fast path if the entries as some unknown size Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	482
52012 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51878 diff changeset	483 def get_stream(self, vfs, volatiles):
50636 5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	484 """return data "stream" information for this file
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	485
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	486 (unencoded_file_path, content_iterator, content_size)
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	487 """
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	488 size = self.file_size(None)
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	489
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	490 def get_stream():
52013 0ad269e24075 stream: open volatile file through the manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52012 diff changeset	491 path = vfs.join(self.unencoded_path)
0ad269e24075 stream: open volatile file through the manager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52012 diff changeset	492 with volatiles.open(path) as fp:
50636 5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	493 yield None # ready to stream
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	494 if size <= 65536:
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	495 yield fp.read(size)
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	496 else:
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	497 yield from util.filechunkiter(fp, limit=size)
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	498
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	499 s = get_stream()
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	500 next(s)
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	501 return (self.unencoded_path, s, size)
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	502
50635 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	503
50471 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50470 diff changeset	504 @attr.s(slots=True, init=False)
50470 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50469 diff changeset	505 class BaseStoreEntry:
50468 521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50466 diff changeset	506 """An entry in the store
521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50466 diff changeset	507
521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50466 diff changeset	508 This is returned by `store.walk` and represent some data in the store."""
521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50466 diff changeset	509
51549 49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51544 diff changeset	510 maybe_volatile = True
49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51544 diff changeset	511
50635 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	512 def files(self) -> List[StoreFile]:
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	513 raise NotImplementedError
8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	514
50637 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	515 def get_streams(
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	516 self,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	517 repo=None,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	518 vfs=None,
52012 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51878 diff changeset	519 volatiles=None,
50637 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	520 max_changeset=None,
50703 0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50677 diff changeset	521 preserve_file_count=False,
50637 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	522 ):
50636 5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	523 """return a list of data stream associated to files for this entry
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	524
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	525 return [(unencoded_file_path, content_iterator, content_size), …]
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	526 """
5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	527 assert vfs is not None
52012 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51878 diff changeset	528 return [f.get_stream(vfs, volatiles) for f in self.files()]
50636 5e60abf811f3 stream-clone: make it the responsability of the store entry to stream content Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50635 diff changeset	529
52357 a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52171 diff changeset	530 def preserve_volatiles(self, vfs, volatiles):
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52171 diff changeset	531 """Use a VolatileManager to preserve the state of any volatile file
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52171 diff changeset	532
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52171 diff changeset	533 This is useful for code that need a consistent view of the content like stream clone.
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52171 diff changeset	534 """
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52171 diff changeset	535 if self.maybe_volatile:
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52171 diff changeset	536 for f in self.files():
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52171 diff changeset	537 if f.is_volatile:
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52171 diff changeset	538 volatiles(vfs.join(f.unencoded_path))
a260d326458f stream: create a `e.preserve_volatiles` method directly on StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 52171 diff changeset	539
50493 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	540
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	541 @attr.s(slots=True, init=False)
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	542 class SimpleStoreEntry(BaseStoreEntry):
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	543 """A generic entry in the store"""
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	544
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	545 is_revlog = False
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	546
51549 49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51544 diff changeset	547 maybe_volatile = attr.ib()
50492 ed8cda1c18e1 store: rename `unencoded_path` to `entry_path` for StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50491 diff changeset	548 _entry_path = attr.ib()
50473 2b2284cf949b store: only access is_volatile information through the file object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50472 diff changeset	549 _is_volatile = attr.ib(default=False)
50472 7d4d2a160cf5 store: only access file_size information through the file object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50471 diff changeset	550 _file_size = attr.ib(default=None)
50628 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50626 diff changeset	551 _files = attr.ib(default=None)
50468 521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50466 diff changeset	552
50471 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50470 diff changeset	553 def __init__(
c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50470 diff changeset	554 self,
50492 ed8cda1c18e1 store: rename `unencoded_path` to `entry_path` for StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50491 diff changeset	555 entry_path,
50471 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50470 diff changeset	556 is_volatile=False,
c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50470 diff changeset	557 file_size=None,
c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50470 diff changeset	558 ):
50493 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	559 super().__init__()
50492 ed8cda1c18e1 store: rename `unencoded_path` to `entry_path` for StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50491 diff changeset	560 self._entry_path = entry_path
50473 2b2284cf949b store: only access is_volatile information through the file object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50472 diff changeset	561 self._is_volatile = is_volatile
50472 7d4d2a160cf5 store: only access file_size information through the file object Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50471 diff changeset	562 self._file_size = file_size
50628 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50626 diff changeset	563 self._files = None
51549 49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51544 diff changeset	564 self.maybe_volatile = is_volatile
50471 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50470 diff changeset	565
50635 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	566 def files(self) -> List[StoreFile]:
50628 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50626 diff changeset	567 if self._files is None:
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50626 diff changeset	568 self._files = [
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50626 diff changeset	569 StoreFile(
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50626 diff changeset	570 unencoded_path=self._entry_path,
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50626 diff changeset	571 file_size=self._file_size,
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50626 diff changeset	572 is_volatile=self._is_volatile,
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50626 diff changeset	573 )
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50626 diff changeset	574 ]
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50626 diff changeset	575 return self._files
50469 9fdc28e21b68 store: introduce a EntryFile object to actually access file info Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50468 diff changeset	576
9fdc28e21b68 store: introduce a EntryFile object to actually access file info Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50468 diff changeset	577
50471 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50470 diff changeset	578 @attr.s(slots=True, init=False)
50470 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50469 diff changeset	579 class RevlogStoreEntry(BaseStoreEntry):
5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50469 diff changeset	580 """A revlog entry in the store"""
5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50469 diff changeset	581
5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50469 diff changeset	582 is_revlog = True
50493 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	583
50470 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50469 diff changeset	584 revlog_type = attr.ib(default=None)
50480 60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50479 diff changeset	585 target_id = attr.ib(default=None)
51549 49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51544 diff changeset	586 maybe_volatile = attr.ib(default=True)
50493 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	587 _path_prefix = attr.ib(default=None)
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	588 _details = attr.ib(default=None)
50628 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50626 diff changeset	589 _files = attr.ib(default=None)
50470 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50469 diff changeset	590
50471 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50470 diff changeset	591 def __init__(
c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50470 diff changeset	592 self,
c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50470 diff changeset	593 revlog_type,
50493 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	594 path_prefix,
50480 60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50479 diff changeset	595 target_id,
50493 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	596 details,
50471 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50470 diff changeset	597 ):
50493 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	598 super().__init__()
50471 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50470 diff changeset	599 self.revlog_type = revlog_type
50480 60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50479 diff changeset	600 self.target_id = target_id
50493 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	601 self._path_prefix = path_prefix
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	602 assert b'.i' in details, (path_prefix, details)
51549 49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51544 diff changeset	603 for ext in details:
49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51544 diff changeset	604 if ext.endswith(REVLOG_FILES_VOLATILE_EXT):
49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51544 diff changeset	605 self.maybe_volatile = True
49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51544 diff changeset	606 break
49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51544 diff changeset	607 else:
49faa72b994e streamclone: stop listing files for entries that have no volatile files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51544 diff changeset	608 self.maybe_volatile = False
50493 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	609 self._details = details
50628 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50626 diff changeset	610 self._files = None
50471 c37450a5f1dc store: have custom init for entries class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50470 diff changeset	611
50494 66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50493 diff changeset	612 @property
66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50493 diff changeset	613 def is_changelog(self):
50677 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	614 return self.revlog_type == KIND_CHANGELOG
50494 66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50493 diff changeset	615
66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50493 diff changeset	616 @property
66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50493 diff changeset	617 def is_manifestlog(self):
50677 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	618 return self.revlog_type == KIND_MANIFESTLOG
50494 66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50493 diff changeset	619
66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50493 diff changeset	620 @property
66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50493 diff changeset	621 def is_filelog(self):
50677 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	622 return self.revlog_type == KIND_FILELOG
50494 66c556968222 store: introduce boolean property for revlog type Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50493 diff changeset	623
50489 3473d18c029a store: introduce a main_file_path method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50480 diff changeset	624 def main_file_path(self):
3473d18c029a store: introduce a main_file_path method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50480 diff changeset	625 """unencoded path of the main revlog file"""
50493 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	626 return self._path_prefix + b'.i'
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	627
50635 8fc10bfd9887 store: declare a `files` method on BaseStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50629 diff changeset	628 def files(self) -> List[StoreFile]:
50628 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50626 diff changeset	629 if self._files is None:
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50626 diff changeset	630 self._files = []
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50626 diff changeset	631 for ext in sorted(self._details, key=_ext_key):
b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50626 diff changeset	632 path = self._path_prefix + ext
50667 11562d72cb7b store: directly pass the filesize in the `details` of revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50666 diff changeset	633 file_size = self._details[ext]
50665 6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50638 diff changeset	634 # files that are "volatile" and might change between
6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50638 diff changeset	635 # listing and streaming
6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50638 diff changeset	636 #
6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50638 diff changeset	637 # note: the ".nd" file are nodemap data and won't "change"
6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50638 diff changeset	638 # but they might be deleted.
6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50638 diff changeset	639 volatile = ext.endswith(REVLOG_FILES_VOLATILE_EXT)
50666 cdb471c8ebcf store: explicitly pass file_size when creating StoreFile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50665 diff changeset	640 f = StoreFile(path, file_size, volatile)
50665 6b522a9e7451 store: have the revlog determine which files are volatile itself Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50638 diff changeset	641 self._files.append(f)
50628 b59e0a4f692f store: cache the `files()` return for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50626 diff changeset	642 return self._files
50489 3473d18c029a store: introduce a main_file_path method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50480 diff changeset	643
50637 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	644 def get_streams(
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	645 self,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	646 repo=None,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	647 vfs=None,
52012 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51878 diff changeset	648 volatiles=None,
50637 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	649 max_changeset=None,
50703 0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50677 diff changeset	650 preserve_file_count=False,
50637 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	651 ):
51544 8e2a68e10f5c stream: in v3, skip the "size" fast path if the entries as some unknown size Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	652 pre_sized = all(f.has_size for f in self.files())
8e2a68e10f5c stream: in v3, skip the "size" fast path if the entries as some unknown size Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50926 diff changeset	653 if pre_sized and (
50669 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	654 repo is None
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	655 or max_changeset is None
50637 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	656 # This use revlog-v2, ignore for now
50669 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	657 or any(k.endswith(b'.idx') for k in self._details.keys())
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	658 # This is not inline, no race expected
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	659 or b'.d' in self._details
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	660 ):
50637 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	661 return super().get_streams(
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	662 repo=repo,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	663 vfs=vfs,
52012 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51878 diff changeset	664 volatiles=volatiles,
50637 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	665 max_changeset=max_changeset,
50703 0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50677 diff changeset	666 preserve_file_count=preserve_file_count,
50637 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	667 )
50703 0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50677 diff changeset	668 elif not preserve_file_count:
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50677 diff changeset	669 stream = [
52012 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51878 diff changeset	670 f.get_stream(vfs, volatiles)
50703 0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50677 diff changeset	671 for f in self.files()
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50677 diff changeset	672 if not f.unencoded_path.endswith((b'.i', b'.d'))
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50677 diff changeset	673 ]
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50677 diff changeset	674 rl = self.get_revlog_instance(repo).get_revlog()
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50677 diff changeset	675 rl_stream = rl.get_streams(max_changeset)
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50677 diff changeset	676 stream.extend(rl_stream)
0452af304808 stream-clone: add a v3 version of the protocol Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50677 diff changeset	677 return stream
50669 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	678
50637 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	679 name_to_size = {}
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	680 for f in self.files():
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	681 name_to_size[f.unencoded_path] = f.file_size(None)
50669 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	682
50637 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	683 stream = [
52012 e308439339e2 stream: rename TempCopyManager to VolatileManager Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51878 diff changeset	684 f.get_stream(vfs, volatiles)
50637 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	685 for f in self.files()
50669 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	686 if not f.unencoded_path.endswith(b'.i')
50637 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	687 ]
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	688
50669 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	689 index_path = self._path_prefix + b'.i'
50638 5460424092e2 stream-clone: smoothly detect and handle a case were a revlog is split Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50637 diff changeset	690
50669 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	691 index_file = None
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	692 try:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	693 index_file = vfs(index_path)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	694 header = index_file.read(INDEX_HEADER.size)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	695 if revlogmod.revlog.is_inline_index(header):
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	696 size = name_to_size[index_path]
50638 5460424092e2 stream-clone: smoothly detect and handle a case were a revlog is split Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50637 diff changeset	697
50669 3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	698 # no split underneath, just return the stream
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	699 def get_stream():
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	700 fp = index_file
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	701 try:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	702 fp.seek(0)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	703 yield None
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	704 if size <= 65536:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	705 yield fp.read(size)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	706 else:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	707 yield from util.filechunkiter(fp, limit=size)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	708 finally:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	709 fp.close()
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	710
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	711 s = get_stream()
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	712 next(s)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	713 index_file = None
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	714 stream.append((index_path, s, size))
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	715 else:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	716 rl = self.get_revlog_instance(repo).get_revlog()
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	717 rl_stream = rl.get_streams(max_changeset, force_inline=True)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	718 for name, s, size in rl_stream:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	719 if name_to_size.get(name, 0) != size:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	720 msg = _(b"expected %d bytes but %d provided for %s")
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	721 msg %= name_to_size.get(name, 0), size, name
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	722 raise error.Abort(msg)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	723 stream.extend(rl_stream)
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	724 finally:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	725 if index_file is not None:
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	726 index_file.close()
3b56395404a1 stream-clone: avoid opening a revlog in case we do not need it Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50668 diff changeset	727
50637 9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	728 files = self.files()
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	729 assert len(stream) == len(files), (
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	730 stream,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	731 files,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	732 self._path_prefix,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	733 self.target_id,
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	734 )
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	735 return stream
9caa860dcbec stream-clone: implement decidated `get_streams` method for revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50636 diff changeset	736
50624 e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50511 diff changeset	737 def get_revlog_instance(self, repo):
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50511 diff changeset	738 """Obtain a revlog instance from this store entry
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50511 diff changeset	739
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50511 diff changeset	740 An instance of the appropriate class is returned.
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50511 diff changeset	741 """
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50511 diff changeset	742 if self.is_changelog:
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50511 diff changeset	743 return changelog.changelog(repo.svfs)
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50511 diff changeset	744 elif self.is_manifestlog:
50625 3ea3767c23a4 store: do not drop the final `/` when creating manifestlog instance Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50624 diff changeset	745 mandir = self.target_id
50624 e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50511 diff changeset	746 return manifest.manifestrevlog(
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50511 diff changeset	747 repo.nodeconstants, repo.svfs, tree=mandir
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50511 diff changeset	748 )
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50511 diff changeset	749 else:
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50511 diff changeset	750 return filelog.filelog(repo.svfs, self.target_id)
e1ee6910f6bc store: add a `get_revlog_instance` method on revlog entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50511 diff changeset	751
50470 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50469 diff changeset	752
50476 5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	753 def _gather_revlog(files_data):
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	754 """group files per revlog prefix
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	755
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	756 The returns a two level nested dict. The top level key is the revlog prefix
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	757 without extension, the second level is all the file "suffix" that were
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	758 seen for this revlog and arbitrary file data as value.
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	759 """
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	760 revlogs = collections.defaultdict(dict)
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	761 for u, value in files_data:
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	762 name, ext = _split_revlog_ext(u)
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	763 revlogs[name][ext] = value
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	764 return sorted(revlogs.items())
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	765
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	766
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	767 def _split_revlog_ext(filename):
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	768 """split the revlog file prefix from the variable extension"""
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	769 if filename.endswith(REVLOG_FILES_LONG_EXT):
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	770 char = b'-'
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	771 else:
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	772 char = b'.'
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	773 idx = filename.rfind(char)
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	774 return filename[:idx], filename[idx:]
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	775
5217e36356bb store: add logic to group revlog file together Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50475 diff changeset	776
50478 b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	777 def _ext_key(ext):
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	778 """a key to order revlog suffix
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	779
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	780 important to issue .i after other entry."""
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	781 # the only important part of this order is to keep the `.i` last.
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	782 if ext.endswith(b'.n'):
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	783 return (0, ext)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	784 elif ext.endswith(b'.nd'):
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	785 return (10, ext)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	786 elif ext.endswith(b'.d'):
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	787 return (20, ext)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	788 elif ext.endswith(b'.i'):
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	789 return (50, ext)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	790 else:
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	791 return (40, ext)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	792
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	793
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48941 diff changeset	794 class basicstore:
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	795 '''base class for local repository stores'''
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	796
17651 3b49c28658f6 store: rename "openertype" argument to "vfstype" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17649 diff changeset	797 def __init__(self, path, vfstype):
17724 bf4b72d8dd4d store: initialize vfs field first to use it for initialization of others FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17722 diff changeset	798 vfs = vfstype(path)
bf4b72d8dd4d store: initialize vfs field first to use it for initialization of others FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17722 diff changeset	799 self.path = vfs.base
17726 7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17725 diff changeset	800 self.createmode = _calcmode(vfs)
17652 2c6f7231becc store: rename "op" variables to "vfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17651 diff changeset	801 vfs.createmode = self.createmode
17728 004bd533880d store: invoke "os.path.isdir()" via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17727 diff changeset	802 self.rawvfs = vfs
31234 9b7a2ef4f27c vfs: use 'vfs' module directly in 'mercurial.store' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31219 diff changeset	803 self.vfs = vfsmod.filtervfs(vfs, encodedir)
17653 dacb50696b75 store: initialize "vfs" fields by "vfs" constructors FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17652 diff changeset	804 self.opener = self.vfs
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	805
80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	806 def join(self, f):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	807 return self.path + b'/' + encodedir(f)
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	808
50490 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50489 diff changeset	809 def _walk(self, relpath, recurse, undecodable=None):
47877 2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47785 diff changeset	810 '''yields (revlog_type, unencoded, size)'''
13426 643b8212813e store: remove pointless pathjoiner parameter Adrian Buehlmann <adrian@cadifra.com> parents: 13391 diff changeset	811 path = self.path
643b8212813e store: remove pointless pathjoiner parameter Adrian Buehlmann <adrian@cadifra.com> parents: 13391 diff changeset	812 if relpath:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	813 path += b'/' + relpath
13426 643b8212813e store: remove pointless pathjoiner parameter Adrian Buehlmann <adrian@cadifra.com> parents: 13391 diff changeset	814 striplen = len(self.path) + 1
6899 56a7a54e074f store: simplify walking Matt Mackall <mpm@selenic.com> parents: 6898 diff changeset	815 l = []
17728 004bd533880d store: invoke "os.path.isdir()" via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17727 diff changeset	816 if self.rawvfs.isdir(path):
6899 56a7a54e074f store: simplify walking Matt Mackall <mpm@selenic.com> parents: 6898 diff changeset	817 visit = [path]
17747 aad3bce98f76 store: invoke "osutil.listdir()" via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17745 diff changeset	818 readdir = self.rawvfs.readdir
6899 56a7a54e074f store: simplify walking Matt Mackall <mpm@selenic.com> parents: 6898 diff changeset	819 while visit:
56a7a54e074f store: simplify walking Matt Mackall <mpm@selenic.com> parents: 6898 diff changeset	820 p = visit.pop()
17747 aad3bce98f76 store: invoke "osutil.listdir()" via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17745 diff changeset	821 for f, kind, st in readdir(p, stat=True):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	822 fp = p + b'/' + f
50668 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	823 if is_revlog(f, kind, st):
6900 def492d1b592 store: change handling of decoding errors Matt Mackall <mpm@selenic.com> parents: 6899 diff changeset	824 n = util.pconvert(fp[striplen:])
50668 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	825 l.append((decodedir(n), st.st_size))
6899 56a7a54e074f store: simplify walking Matt Mackall <mpm@selenic.com> parents: 6898 diff changeset	826 elif kind == stat.S_IFDIR and recurse:
56a7a54e074f store: simplify walking Matt Mackall <mpm@selenic.com> parents: 6898 diff changeset	827 visit.append(fp)
50475 1c0244a8cdaf store: change `_walk` return to `(filename, (type, size))` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50474 diff changeset	828
17054 125ff5654b72 store: sort filenames in place Bryan O'Sullivan <bryano@fb.com> parents: 16404 diff changeset	829 l.sort()
125ff5654b72 store: sort filenames in place Bryan O'Sullivan <bryano@fb.com> parents: 16404 diff changeset	830 return l
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	831
46607 e9901d01d135 revlog: add a mechanism to verify expected file position before appending Kyle Lippincott <spectral@google.com> parents: 46383 diff changeset	832 def changelog(self, trypending, concurrencychecker=None):
e9901d01d135 revlog: add a mechanism to verify expected file position before appending Kyle Lippincott <spectral@google.com> parents: 46383 diff changeset	833 return changelog.changelog(
e9901d01d135 revlog: add a mechanism to verify expected file position before appending Kyle Lippincott <spectral@google.com> parents: 46383 diff changeset	834 self.vfs,
e9901d01d135 revlog: add a mechanism to verify expected file position before appending Kyle Lippincott <spectral@google.com> parents: 46383 diff changeset	835 trypending=trypending,
e9901d01d135 revlog: add a mechanism to verify expected file position before appending Kyle Lippincott <spectral@google.com> parents: 46383 diff changeset	836 concurrencychecker=concurrencychecker,
e9901d01d135 revlog: add a mechanism to verify expected file position before appending Kyle Lippincott <spectral@google.com> parents: 46383 diff changeset	837 )
42905 3df3b139a43d localrepo: push manifestlog and changelog construction code into store Augie Fackler <augie@google.com> parents: 42715 diff changeset	838
52066 6641a3ae8313 manifest: drop the CamelCase name for `manifest.manifestlog` Matt Harbison <matt_harbison@yahoo.com> parents: 52013 diff changeset	839 def manifestlog(self, repo, storenarrowmatch) -> manifest.manifestlog:
46780 6266d19556ad node: introduce nodeconstants class Joerg Sonnenberger <joerg@bec.de> parents: 46607 diff changeset	840 rootstore = manifest.manifestrevlog(repo.nodeconstants, self.vfs)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	841 return manifest.manifestlog(self.vfs, repo, rootstore, storenarrowmatch)
42905 3df3b139a43d localrepo: push manifestlog and changelog construction code into store Augie Fackler <augie@google.com> parents: 42715 diff changeset	842
50501 862e3a13da44 store: rename `datafiles` to `data_entries` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50495 diff changeset	843 def data_entries(
50468 521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50466 diff changeset	844 self, matcher=None, undecodable=None
50470 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50469 diff changeset	845 ) -> Generator[BaseStoreEntry, None, None]:
47877 2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47785 diff changeset	846 """Like walk, but excluding the changelog and root manifest.
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47785 diff changeset	847
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47785 diff changeset	848 When [undecodable] is None, revlogs names that can't be
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47785 diff changeset	849 decoded cause an exception. When it is provided, it should
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47785 diff changeset	850 be a list and the filenames that can't be decoded are added
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47785 diff changeset	851 to it instead. This is very rarely needed."""
50479 1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50478 diff changeset	852 dirs = [
50677 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	853 (b'data', KIND_FILELOG, False),
e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	854 (b'meta', KIND_MANIFESTLOG, True),
50479 1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50478 diff changeset	855 ]
50626 309cbd8400ae store: properly compute the targer_id of manifestlog in no-fncache walk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50625 diff changeset	856 for base_dir, rl_type, strip_filename in dirs:
50490 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50489 diff changeset	857 files = self._walk(base_dir, True, undecodable=undecodable)
50479 1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50478 diff changeset	858 for revlog, details in _gather_revlog(files):
50493 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	859 revlog_target_id = revlog.split(b'/', 1)[1]
50626 309cbd8400ae store: properly compute the targer_id of manifestlog in no-fncache walk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50625 diff changeset	860 if strip_filename and b'/' in revlog:
309cbd8400ae store: properly compute the targer_id of manifestlog in no-fncache walk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50625 diff changeset	861 revlog_target_id = revlog_target_id.rsplit(b'/', 1)[0]
309cbd8400ae store: properly compute the targer_id of manifestlog in no-fncache walk Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50625 diff changeset	862 revlog_target_id += b'/'
50493 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	863 yield RevlogStoreEntry(
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	864 path_prefix=revlog,
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	865 revlog_type=rl_type,
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	866 target_id=revlog_target_id,
50668 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	867 details=details,
50493 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	868 )
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	869
50511 0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50510 diff changeset	870 def top_entries(
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50510 diff changeset	871 self, phase=False, obsolescence=False
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50510 diff changeset	872 ) -> Generator[BaseStoreEntry, None, None]:
50510 5a62d56e3955 store: yield phases before changelog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50509 diff changeset	873 if phase and self.vfs.exists(b'phaseroots'):
5a62d56e3955 store: yield phases before changelog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50509 diff changeset	874 yield SimpleStoreEntry(
5a62d56e3955 store: yield phases before changelog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50509 diff changeset	875 entry_path=b'phaseroots',
5a62d56e3955 store: yield phases before changelog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50509 diff changeset	876 is_volatile=True,
5a62d56e3955 store: yield phases before changelog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50509 diff changeset	877 )
5a62d56e3955 store: yield phases before changelog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50509 diff changeset	878
50511 0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50510 diff changeset	879 if obsolescence and self.vfs.exists(b'obsstore'):
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50510 diff changeset	880 # XXX if we had the file size it could be non-volatile
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50510 diff changeset	881 yield SimpleStoreEntry(
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50510 diff changeset	882 entry_path=b'obsstore',
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50510 diff changeset	883 is_volatile=True,
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50510 diff changeset	884 )
0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50510 diff changeset	885
50478 b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	886 files = reversed(self._walk(b'', False))
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	887
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	888 changelogs = collections.defaultdict(dict)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	889 manifestlogs = collections.defaultdict(dict)
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	890
50668 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	891 for u, s in files:
46895 6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	892 if u.startswith(b'00changelog'):
50478 b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	893 name, ext = _split_revlog_ext(u)
50668 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	894 changelogs[name][ext] = s
46895 6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	895 elif u.startswith(b'00manifest'):
50478 b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	896 name, ext = _split_revlog_ext(u)
50668 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	897 manifestlogs[name][ext] = s
46895 6085b7f1536d store: also return some information about the type of file `walk` found Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46854 diff changeset	898 else:
50470 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50469 diff changeset	899 yield SimpleStoreEntry(
50492 ed8cda1c18e1 store: rename `unencoded_path` to `entry_path` for StoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50491 diff changeset	900 entry_path=u,
50668 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	901 is_volatile=False,
50470 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50469 diff changeset	902 file_size=s,
5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50469 diff changeset	903 )
50478 b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	904 # yield manifest before changelog
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	905 top_rl = [
50677 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	906 (manifestlogs, KIND_MANIFESTLOG),
e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	907 (changelogs, KIND_CHANGELOG),
50478 b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	908 ]
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	909 assert len(manifestlogs) <= 1
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	910 assert len(changelogs) <= 1
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	911 for data, revlog_type in top_rl:
b08243dbc2e3 store: also gather files per revlog in `topfiles` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50477 diff changeset	912 for revlog, details in sorted(data.items()):
50493 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	913 yield RevlogStoreEntry(
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	914 path_prefix=revlog,
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	915 revlog_type=revlog_type,
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	916 target_id=b'',
50668 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	917 details=details,
50493 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	918 )
19177 1e104aaa4c44 store: move top file walk to a separate function Durham Goode <durham@fb.com> parents: 18435 diff changeset	919
50509 a32d739b0ffb store: make `walk` return an entry for phase if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50502 diff changeset	920 def walk(
50511 0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50510 diff changeset	921 self, matcher=None, phase=False, obsolescence=False
50509 a32d739b0ffb store: make `walk` return an entry for phase if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50502 diff changeset	922 ) -> Generator[BaseStoreEntry, None, None]:
50448 584ff1f97201 comments: fix spelling Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50319 diff changeset	923 """return files related to data storage (ie: revlogs)
46853 eed3e2b79b48 store: document the `walk` method Pierre-Yves David <pierre-yves.david@octobus.net> parents: 46819 diff changeset	924
50509 a32d739b0ffb store: make `walk` return an entry for phase if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50502 diff changeset	925 yields instance from BaseStoreEntry subclasses
40340 2d45b549392f store: pass matcher to store.datafiles() Pulkit Goyal <pulkit@yandex-team.ru> parents: 39698 diff changeset	926
2d45b549392f store: pass matcher to store.datafiles() Pulkit Goyal <pulkit@yandex-team.ru> parents: 39698 diff changeset	927 if a matcher is passed, storage files of only those tracked paths
2d45b549392f store: pass matcher to store.datafiles() Pulkit Goyal <pulkit@yandex-team.ru> parents: 39698 diff changeset	928 are passed with matches the matcher
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	929 """
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	930 # yield data files first
50501 862e3a13da44 store: rename `datafiles` to `data_entries` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50495 diff changeset	931 for x in self.data_entries(matcher):
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	932 yield x
50511 0925eaf09c8b store: make `walk` return an entry for obsolescence if requested so Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50510 diff changeset	933 for x in self.top_entries(phase=phase, obsolescence=obsolescence):
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	934 yield x
80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	935
6903 0642d9d7ec80 clone: get a list of files to clone from store Matt Mackall <mpm@selenic.com> parents: 6902 diff changeset	936 def copylist(self):
45483 d252f51ab032 share: introduce config option to store requires in .hg/store Pulkit Goyal <7895pulkit@gmail.com> parents: 45351 diff changeset	937 return _data
6903 0642d9d7ec80 clone: get a list of files to clone from store Matt Mackall <mpm@selenic.com> parents: 6902 diff changeset	938
20883 cd443c7589cc fncache: move fncache writing to be in a transaction Durham Goode <durham@fb.com> parents: 20879 diff changeset	939 def write(self, tr):
13391 d00bbff8600e fncachestore: defer updating the fncache file to a single file open Adrian Buehlmann <adrian@cadifra.com> parents: 13169 diff changeset	940 pass
d00bbff8600e fncachestore: defer updating the fncache file to a single file open Adrian Buehlmann <adrian@cadifra.com> parents: 13169 diff changeset	941
20884 2efdd186925d caches: invalidate store caches when lock is taken Durham Goode <durham@fb.com> parents: 20883 diff changeset	942 def invalidatecaches(self):
2efdd186925d caches: invalidate store caches when lock is taken Durham Goode <durham@fb.com> parents: 20883 diff changeset	943 pass
2efdd186925d caches: invalidate store caches when lock is taken Durham Goode <durham@fb.com> parents: 20883 diff changeset	944
20885 f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	945 def markremoved(self, fn):
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	946 pass
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	947
17744 09d5b2055295 store: add a contains method to basicstore smuralid parents: 17738 diff changeset	948 def __contains__(self, path):
09d5b2055295 store: add a contains method to basicstore smuralid parents: 17738 diff changeset	949 '''Checks if the store contains path'''
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	950 path = b"/".join((b"data", path))
17744 09d5b2055295 store: add a contains method to basicstore smuralid parents: 17738 diff changeset	951 # file?
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	952 if self.vfs.exists(path + b".i"):
17744 09d5b2055295 store: add a contains method to basicstore smuralid parents: 17738 diff changeset	953 return True
09d5b2055295 store: add a contains method to basicstore smuralid parents: 17738 diff changeset	954 # dir?
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	955 if not path.endswith(b"/"):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	956 path = path + b"/"
19903 ca875b271ac3 store: use "vfs.exists()" instead of "os.path.exists()" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 19177 diff changeset	957 return self.vfs.exists(path)
17744 09d5b2055295 store: add a contains method to basicstore smuralid parents: 17738 diff changeset	958
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	959
6898 69aeaaaf6e07 store: simplify class hierarchy Matt Mackall <mpm@selenic.com> parents: 6897 diff changeset	960 class encodedstore(basicstore):
17651 3b49c28658f6 store: rename "openertype" argument to "vfstype" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17649 diff changeset	961 def __init__(self, path, vfstype):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	962 vfs = vfstype(path + b'/store')
17724 bf4b72d8dd4d store: initialize vfs field first to use it for initialization of others FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17722 diff changeset	963 self.path = vfs.base
17726 7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17725 diff changeset	964 self.createmode = _calcmode(vfs)
17652 2c6f7231becc store: rename "op" variables to "vfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17651 diff changeset	965 vfs.createmode = self.createmode
17728 004bd533880d store: invoke "os.path.isdir()" via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17727 diff changeset	966 self.rawvfs = vfs
31234 9b7a2ef4f27c vfs: use 'vfs' module directly in 'mercurial.store' Pierre-Yves David <pierre-yves.david@ens-lyon.org> parents: 31219 diff changeset	967 self.vfs = vfsmod.filtervfs(vfs, encodefilename)
17653 dacb50696b75 store: initialize "vfs" fields by "vfs" constructors FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17652 diff changeset	968 self.opener = self.vfs
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	969
50490 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50489 diff changeset	970 def _walk(self, relpath, recurse, undecodable=None):
816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50489 diff changeset	971 old = super()._walk(relpath, recurse)
816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50489 diff changeset	972 new = []
816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50489 diff changeset	973 for f1, value in old:
6892 dab95717058d verify: check repo.store Adrian Buehlmann <adrian@cadifra.com> parents: 6890 diff changeset	974 try:
47877 2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47785 diff changeset	975 f2 = decodefilename(f1)
6892 dab95717058d verify: check repo.store Adrian Buehlmann <adrian@cadifra.com> parents: 6890 diff changeset	976 except KeyError:
47877 2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47785 diff changeset	977 if undecodable is None:
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47785 diff changeset	978 msg = _(b'undecodable revlog name %s') % f1
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47785 diff changeset	979 raise error.StorageError(msg)
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47785 diff changeset	980 else:
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47785 diff changeset	981 undecodable.append(f1)
2174f54aab18 store: return just one filename in walk functions Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> parents: 47785 diff changeset	982 continue
50490 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50489 diff changeset	983 new.append((f2, value))
816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50489 diff changeset	984 return new
816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50489 diff changeset	985
50501 862e3a13da44 store: rename `datafiles` to `data_entries` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50495 diff changeset	986 def data_entries(
50490 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50489 diff changeset	987 self, matcher=None, undecodable=None
816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50489 diff changeset	988 ) -> Generator[BaseStoreEntry, None, None]:
50501 862e3a13da44 store: rename `datafiles` to `data_entries` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50495 diff changeset	989 entries = super(encodedstore, self).data_entries(
862e3a13da44 store: rename `datafiles` to `data_entries` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50495 diff changeset	990 undecodable=undecodable
862e3a13da44 store: rename `datafiles` to `data_entries` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50495 diff changeset	991 )
50490 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50489 diff changeset	992 for entry in entries:
50491 b4953fad744e store: do the revlog matching on entry directly Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50490 diff changeset	993 if _match_tracked_entry(entry, matcher):
50490 816e8bc6e066 store: split the wrapping of encodedstore between _wrap and datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50489 diff changeset	994 yield entry
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	995
80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	996 def join(self, f):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	997 return self.path + b'/' + encodefilename(f)
6840 80e51429cb9a introduce store classes Adrian Buehlmann <adrian@cadifra.com> parents: 6839 diff changeset	998
6903 0642d9d7ec80 clone: get a list of files to clone from store Matt Mackall <mpm@selenic.com> parents: 6902 diff changeset	999 def copylist(self):
45351 909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	1000 return [b'requires', b'00changelog.i'] + [b'store/' + f for f in _data]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	1001
6903 0642d9d7ec80 clone: get a list of files to clone from store Matt Mackall <mpm@selenic.com> parents: 6902 diff changeset	1002
48946 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 48941 diff changeset	1003 class fncache:
8531 810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	1004 # the filename used to be partially encoded
810387f59696 filelog encoding: move the encoding/decoding into store Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8530 diff changeset	1005 # hence the encodedir/decodedir dance
17722 3b976051034d store: rename field name from "opener" to "vfs" in internal classes for fncache FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17721 diff changeset	1006 def __init__(self, vfs):
3b976051034d store: rename field name from "opener" to "vfs" in internal classes for fncache FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17721 diff changeset	1007 self.vfs = vfs
50315 cf6e1d535602 fncache: make it possible to ignore some file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49306 diff changeset	1008 self._ignores = set()
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1009 self.entries = None
13391 d00bbff8600e fncachestore: defer updating the fncache file to a single file open Adrian Buehlmann <adrian@cadifra.com> parents: 13169 diff changeset	1010 self._dirty = False
40736 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40624 diff changeset	1011 # set of new additions to fncache
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40624 diff changeset	1012 self.addls = set()
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1013
52169 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52066 diff changeset	1014 @property
ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52066 diff changeset	1015 def is_loaded(self):
ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52066 diff changeset	1016 return self.entries is not None
ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52066 diff changeset	1017
42715 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42341 diff changeset	1018 def ensureloaded(self, warn=None):
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	1019 """read the fncache file if not already read.
42715 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42341 diff changeset	1020
f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42341 diff changeset	1021 If the file on disk is corrupted, raise. If warn is provided,
45942 89a2afe31e82 formating: upgrade to black 20.8b1 Augie Fackler <raf@durin42.com> parents: 45483 diff changeset	1022 warn and keep going instead."""
52169 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52066 diff changeset	1023 if not self.is_loaded:
42715 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42341 diff changeset	1024 self._load(warn)
f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42341 diff changeset	1025
f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42341 diff changeset	1026 def _load(self, warn=None):
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1027 '''fill the entries from the fncache file'''
13391 d00bbff8600e fncachestore: defer updating the fncache file to a single file open Adrian Buehlmann <adrian@cadifra.com> parents: 13169 diff changeset	1028 self._dirty = False
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1029 try:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1030 fp = self.vfs(b'fncache', mode=b'rb')
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1031 except IOError:
03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1032 # skip nonexistent file
16404 9fca5b056c0a store: speed up read and write of large fncache files Bryan O'Sullivan <bryano@fb.com> parents: 15742 diff changeset	1033 self.entries = set()
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1034 return
41978 a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41973 diff changeset	1035
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41973 diff changeset	1036 self.entries = set()
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41973 diff changeset	1037 chunk = b''
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41973 diff changeset	1038 for c in iter(functools.partial(fp.read, fncache_chunksize), b''):
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41973 diff changeset	1039 chunk += c
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41973 diff changeset	1040 try:
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41973 diff changeset	1041 p = chunk.rindex(b'\n')
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	1042 self.entries.update(decodedir(chunk[: p + 1]).splitlines())
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	1043 chunk = chunk[p + 1 :]
41978 a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41973 diff changeset	1044 except ValueError:
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41973 diff changeset	1045 # substring '\n' not found, maybe the entry is bigger than the
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41973 diff changeset	1046 # chunksize, so let's keep iterating
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41973 diff changeset	1047 pass
a56487081109 store: don't read the whole fncache in memory Pulkit Goyal <pulkit@yandex-team.ru> parents: 41973 diff changeset	1048
41981 a920a9e1795a store: error out if fncache does not ends with a newline Pulkit Goyal <pulkit@yandex-team.ru> parents: 41978 diff changeset	1049 if chunk:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1050 msg = _(b"fncache does not ends with a newline")
42715 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42341 diff changeset	1051 if warn:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1052 warn(msg + b'\n')
42715 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42341 diff changeset	1053 else:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	1054 raise error.Abort(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	1055 msg,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	1056 hint=_(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1057 b"use 'hg debugrebuildfncache' to "
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1058 b"rebuild the fncache"
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	1059 ),
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	1060 )
42715 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42341 diff changeset	1061 self._checkentries(fp, warn)
41973 d7ef84e595f8 store: move logic to check for invalid entry in fncache to own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41093 diff changeset	1062 fp.close()
d7ef84e595f8 store: move logic to check for invalid entry in fncache to own function Pulkit Goyal <pulkit@yandex-team.ru> parents: 41093 diff changeset	1063
42715 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42341 diff changeset	1064 def _checkentries(self, fp, warn):
47062 f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1 Kyle Lippincott <spectral@google.com> parents: 46990 diff changeset	1065 """make sure there is no empty string in entries"""
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1066 if b'' in self.entries:
16404 9fca5b056c0a store: speed up read and write of large fncache files Bryan O'Sullivan <bryano@fb.com> parents: 15742 diff changeset	1067 fp.seek(0)
48941 fd5b8e696b75 py3: stop using util.iterfile() Gregory Szorc <gregory.szorc@gmail.com> parents: 48913 diff changeset	1068 for n, line in enumerate(fp):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1069 if not line.rstrip(b'\n'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1070 t = _(b'invalid entry in fncache, line %d') % (n + 1)
42715 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42341 diff changeset	1071 if warn:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1072 warn(t + b'\n')
42715 f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42341 diff changeset	1073 else:
f59f8a5e9096 fncache: make debugrebuildfncache not fail on broken fncache Valentin Gatien-Baron <vgatien-baron@janestreet.com> parents: 42341 diff changeset	1074 raise error.Abort(t)
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1075
20883 cd443c7589cc fncache: move fncache writing to be in a transaction Durham Goode <durham@fb.com> parents: 20879 diff changeset	1076 def write(self, tr):
16404 9fca5b056c0a store: speed up read and write of large fncache files Bryan O'Sullivan <bryano@fb.com> parents: 15742 diff changeset	1077 if self._dirty:
52169 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52066 diff changeset	1078 assert self.is_loaded
40748 df8ed31a8ad8 store: write fncache only once if there are both adds and removes Pulkit Goyal <pulkit@yandex-team.ru> parents: 40736 diff changeset	1079 self.entries = self.entries \| self.addls
df8ed31a8ad8 store: write fncache only once if there are both adds and removes Pulkit Goyal <pulkit@yandex-team.ru> parents: 40736 diff changeset	1080 self.addls = set()
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1081 tr.addbackup(b'fncache')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1082 fp = self.vfs(b'fncache', mode=b'wb', atomictemp=True)
20879 cd03854a2e06 fncache: remove the rewriting logic Durham Goode <durham@fb.com> parents: 19903 diff changeset	1083 if self.entries:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1084 fp.write(encodedir(b'\n'.join(self.entries) + b'\n'))
20879 cd03854a2e06 fncache: remove the rewriting logic Durham Goode <durham@fb.com> parents: 19903 diff changeset	1085 fp.close()
cd03854a2e06 fncache: remove the rewriting logic Durham Goode <durham@fb.com> parents: 19903 diff changeset	1086 self._dirty = False
40736 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40624 diff changeset	1087 if self.addls:
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40624 diff changeset	1088 # if we have just new entries, let's append them to the fncache
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1089 tr.addbackup(b'fncache')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1090 fp = self.vfs(b'fncache', mode=b'ab', atomictemp=True)
40736 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40624 diff changeset	1091 if self.addls:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1092 fp.write(encodedir(b'\n'.join(self.addls) + b'\n'))
40736 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40624 diff changeset	1093 fp.close()
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40624 diff changeset	1094 self.entries = None
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40624 diff changeset	1095 self.addls = set()
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1096
50315 cf6e1d535602 fncache: make it possible to ignore some file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49306 diff changeset	1097 def addignore(self, fn):
cf6e1d535602 fncache: make it possible to ignore some file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49306 diff changeset	1098 self._ignores.add(fn)
cf6e1d535602 fncache: make it possible to ignore some file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49306 diff changeset	1099
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1100 def add(self, fn):
50315 cf6e1d535602 fncache: make it possible to ignore some file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49306 diff changeset	1101 if fn in self._ignores:
cf6e1d535602 fncache: make it possible to ignore some file Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49306 diff changeset	1102 return
52169 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52066 diff changeset	1103 if not self.is_loaded:
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1104 self._load()
10577 d5bd1beff794 store: only add new entries to the fncache file Adrian Buehlmann <adrian@cadifra.com> parents: 10339 diff changeset	1105 if fn not in self.entries:
40736 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40624 diff changeset	1106 self.addls.add(fn)
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1107
20885 f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1108 def remove(self, fn):
52169 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52066 diff changeset	1109 if not self.is_loaded:
20885 f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1110 self._load()
40736 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40624 diff changeset	1111 if fn in self.addls:
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40624 diff changeset	1112 self.addls.remove(fn)
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40624 diff changeset	1113 return
20885 f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1114 try:
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1115 self.entries.remove(fn)
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1116 self._dirty = True
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1117 except KeyError:
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1118 pass
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1119
17782 8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1120 def __contains__(self, fn):
40736 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40624 diff changeset	1121 if fn in self.addls:
0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40624 diff changeset	1122 return True
52169 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52066 diff changeset	1123 if not self.is_loaded:
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1124 self._load()
17782 8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1125 return fn in self.entries
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1126
03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1127 def __iter__(self):
52169 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52066 diff changeset	1128 if not self.is_loaded:
8530 03196ac9a8b9 store: refactor the fncache handling Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 8480 diff changeset	1129 self._load()
40736 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40624 diff changeset	1130 return iter(self.entries \| self.addls)
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1131
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	1132
41093 6498f0e03526 vfs: fix proxyvfs inheritance Boris Feld <boris.feld@octobus.net> parents: 40748 diff changeset	1133 class _fncachevfs(vfsmod.proxyvfs):
17721 cf236e3501c3 store: rename argument name from "op"(ener) to "vfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17693 diff changeset	1134 def __init__(self, vfs, fnc, encode):
33412 a42369e04aee vfs: rename auditvfs to proxyvfs Yuya Nishihara <yuya@tcha.org> parents: 32372 diff changeset	1135 vfsmod.proxyvfs.__init__(self, vfs)
52170 9fbdf355c916 fncachestore: add typing information Rapha?l Gom?s <rgomes@octobus.net> parents: 52169 diff changeset	1136 self.fncache: fncache = fnc
14194 3a90fb3addc1 store: break up reference cycle introduced in 9cbff8a39a2a Adrian Buehlmann <adrian@cadifra.com> parents: 14166 diff changeset	1137 self.encode = encode
52171 bd43465af568 fncache: add attribute to check whether we're using dotencode Rapha?l Gom?s <rgomes@octobus.net> parents: 52170 diff changeset	1138 self.uses_dotencode = encode is _pathencode
14194 3a90fb3addc1 store: break up reference cycle introduced in 9cbff8a39a2a Adrian Buehlmann <adrian@cadifra.com> parents: 14166 diff changeset	1139
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1140 def __call__(self, path, mode=b'r', args, *kw):
38661 8ac0c9cd4c48 fncache: avoid loading the filename cache when not actually modifying it Martijn Pieters <mj@zopatista.com> parents: 37409 diff changeset	1141 encoded = self.encode(path)
50464 0dbab42adca5 store: do not record file that are not part of a revlog in fncache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50448 diff changeset	1142 if (
0dbab42adca5 store: do not record file that are not part of a revlog in fncache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50448 diff changeset	1143 mode not in (b'r', b'rb')
0dbab42adca5 store: do not record file that are not part of a revlog in fncache Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50448 diff changeset	1144 and (path.startswith(b'data/') or path.startswith(b'meta/'))
50668 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	1145 and is_revlog_file(path)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	1146 ):
38661 8ac0c9cd4c48 fncache: avoid loading the filename cache when not actually modifying it Martijn Pieters <mj@zopatista.com> parents: 37409 diff changeset	1147 # do not trigger a fncache load when adding a file that already is
8ac0c9cd4c48 fncache: avoid loading the filename cache when not actually modifying it Martijn Pieters <mj@zopatista.com> parents: 37409 diff changeset	1148 # known to exist.
52169 ba9df4e6abb8 fncache: refactor load check into a property Rapha?l Gom?s <rgomes@octobus.net> parents: 52066 diff changeset	1149 notload = not self.fncache.is_loaded and (
50809 30381812708d fncache: fix a bug that corrupts the fncache after transaction rollback Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50703 diff changeset	1150 # if the file has size zero, it should be considered as missing.
30381812708d fncache: fix a bug that corrupts the fncache after transaction rollback Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50703 diff changeset	1151 # Such zero-size files are the result of truncation when a
30381812708d fncache: fix a bug that corrupts the fncache after transaction rollback Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50703 diff changeset	1152 # transaction is aborted.
30381812708d fncache: fix a bug that corrupts the fncache after transaction rollback Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50703 diff changeset	1153 self.vfs.exists(encoded)
30381812708d fncache: fix a bug that corrupts the fncache after transaction rollback Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50703 diff changeset	1154 and self.vfs.stat(encoded).st_size
30381812708d fncache: fix a bug that corrupts the fncache after transaction rollback Arseniy Alekseyev <aalekseyev@janestreet.com> parents: 50703 diff changeset	1155 )
38661 8ac0c9cd4c48 fncache: avoid loading the filename cache when not actually modifying it Martijn Pieters <mj@zopatista.com> parents: 37409 diff changeset	1156 if not notload:
8ac0c9cd4c48 fncache: avoid loading the filename cache when not actually modifying it Martijn Pieters <mj@zopatista.com> parents: 37409 diff changeset	1157 self.fncache.add(path)
8ac0c9cd4c48 fncache: avoid loading the filename cache when not actually modifying it Martijn Pieters <mj@zopatista.com> parents: 37409 diff changeset	1158 return self.vfs(encoded, mode, args, *kw)
14194 3a90fb3addc1 store: break up reference cycle introduced in 9cbff8a39a2a Adrian Buehlmann <adrian@cadifra.com> parents: 14166 diff changeset	1159
51878 2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51859 diff changeset	1160 def join(self, path: Optional[bytes], *insidef: bytes) -> bytes:
2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51859 diff changeset	1161 insidef = (self.encode(f) for f in insidef)
2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51859 diff changeset	1162
17725 ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17724 diff changeset	1163 if path:
51878 2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51859 diff changeset	1164 return self.vfs.join(self.encode(path), *insidef)
17725 ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17724 diff changeset	1165 else:
51878 2391a5fa111e store: fix a signature mismatch for a vfs subclass Matt Harbison <matt_harbison@yahoo.com> parents: 51859 diff changeset	1166 return self.vfs.join(path, *insidef)
17725 ffd589d4b785 vfs: define "join()" in each classes derived from "abstractvfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17724 diff changeset	1167
47443 9ab54aa56982 vfs: add a `register_file` method on the vfs class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47389 diff changeset	1168 def register_file(self, path):
9ab54aa56982 vfs: add a `register_file` method on the vfs class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47389 diff changeset	1169 """generic hook point to lets fncache steer its stew"""
9ab54aa56982 vfs: add a `register_file` method on the vfs class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47389 diff changeset	1170 if path.startswith(b'data/') or path.startswith(b'meta/'):
9ab54aa56982 vfs: add a `register_file` method on the vfs class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47389 diff changeset	1171 self.fncache.add(path)
9ab54aa56982 vfs: add a `register_file` method on the vfs class Pierre-Yves David <pierre-yves.david@octobus.net> parents: 47389 diff changeset	1172
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	1173
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1174 class fncachestore(basicstore):
17651 3b49c28658f6 store: rename "openertype" argument to "vfstype" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17649 diff changeset	1175 def __init__(self, path, vfstype, dotencode):
17591 9a5c2ecd1158 store: move encode lambda logic into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17590 diff changeset	1176 if dotencode:
18435 8c019d2fd7c0 store: switch to C-based hashed path encoding Bryan O'Sullivan <bryano@fb.com> parents: 18430 diff changeset	1177 encode = _pathencode
17591 9a5c2ecd1158 store: move encode lambda logic into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17590 diff changeset	1178 else:
9a5c2ecd1158 store: move encode lambda logic into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17590 diff changeset	1179 encode = _plainhybridencode
12687 34d8247a4595 store: encode first period or space in filenames (issue1713) Adrian Buehlmann <adrian@cadifra.com> parents: 12171 diff changeset	1180 self.encode = encode
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1181 vfs = vfstype(path + b'/store')
17724 bf4b72d8dd4d store: initialize vfs field first to use it for initialization of others FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17722 diff changeset	1182 self.path = vfs.base
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1183 self.pathsep = self.path + b'/'
17726 7cb7e17c23b2 store: invoke "os.stat()" for "createmode" initialization via vfs FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17725 diff changeset	1184 self.createmode = _calcmode(vfs)
17652 2c6f7231becc store: rename "op" variables to "vfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17651 diff changeset	1185 vfs.createmode = self.createmode
17727 6492b39a44d5 store: replace invocation of "getsize()" by "vfs.stat()" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17726 diff changeset	1186 self.rawvfs = vfs
17652 2c6f7231becc store: rename "op" variables to "vfs" FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17651 diff changeset	1187 fnc = fncache(vfs)
9133 996c1cd8f530 store: eliminate reference cycle in fncachestore Simon Heimberg <simohe@besonet.ch> parents: 8778 diff changeset	1188 self.fncache = fnc
17653 dacb50696b75 store: initialize "vfs" fields by "vfs" constructors FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17652 diff changeset	1189 self.vfs = _fncachevfs(vfs, fnc, encode)
dacb50696b75 store: initialize "vfs" fields by "vfs" constructors FUJIWARA Katsunori <foozy@lares.dti.ne.jp> parents: 17652 diff changeset	1190 self.opener = self.vfs
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1191
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1192 def join(self, f):
17562 b42b0729744d store: reduce string concatenation when joining Bryan O'Sullivan <bryano@fb.com> parents: 17555 diff changeset	1193 return self.pathsep + self.encode(f)
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1194
17731 c85dbae29684 store: restore getsize method Matt Mackall <mpm@selenic.com> parents: 17728 diff changeset	1195 def getsize(self, path):
c85dbae29684 store: restore getsize method Matt Mackall <mpm@selenic.com> parents: 17728 diff changeset	1196 return self.rawvfs.stat(path).st_size
c85dbae29684 store: restore getsize method Matt Mackall <mpm@selenic.com> parents: 17728 diff changeset	1197
50501 862e3a13da44 store: rename `datafiles` to `data_entries` Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50495 diff changeset	1198 def data_entries(
50468 521fec115dad store: use a StoreEntry object instead of tuple for store files Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50466 diff changeset	1199 self, matcher=None, undecodable=None
50470 5a2fb64d38b2 store: use specialized class for store entries Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50469 diff changeset	1200 ) -> Generator[BaseStoreEntry, None, None]:
50477 d4f54aded22e store: also group files by revlog in fncache version of datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50476 diff changeset	1201 # Note: all files in fncache should be revlog related, However the
d4f54aded22e store: also group files by revlog in fncache version of datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50476 diff changeset	1202 # fncache might contains such file added by previous version of
d4f54aded22e store: also group files by revlog in fncache version of datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50476 diff changeset	1203 # Mercurial.
50668 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	1204 files = ((f, None) for f in self.fncache if is_revlog_file(f))
50477 d4f54aded22e store: also group files by revlog in fncache version of datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50476 diff changeset	1205 by_revlog = _gather_revlog(files)
d4f54aded22e store: also group files by revlog in fncache version of datafiles Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50476 diff changeset	1206 for revlog, details in by_revlog:
50479 1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50478 diff changeset	1207 if revlog.startswith(b'data/'):
50677 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	1208 rl_type = KIND_FILELOG
50480 60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50479 diff changeset	1209 revlog_target_id = revlog.split(b'/', 1)[1]
50479 1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50478 diff changeset	1210 elif revlog.startswith(b'meta/'):
50677 e324329fc97e store: use the revlog type from revlog's constant instead of custom one Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50669 diff changeset	1211 rl_type = KIND_MANIFESTLOG
50480 60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50479 diff changeset	1212 # drop the initial directory and the `00manifest` file part
60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50479 diff changeset	1213 tmp = revlog.split(b'/', 1)[1]
60e613f6a229 store: add a `target_id` attribute on RevlogStoreEntry Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50479 diff changeset	1214 revlog_target_id = tmp.rsplit(b'/', 1)[0] + b'/'
50479 1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50478 diff changeset	1215 else:
1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50478 diff changeset	1216 # unreachable
1fc25227b068 store: actually tag tree manifest revlogs as manifest revlogs Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50478 diff changeset	1217 assert False, revlog
50493 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	1218 entry = RevlogStoreEntry(
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	1219 path_prefix=revlog,
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	1220 revlog_type=rl_type,
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	1221 target_id=revlog_target_id,
50668 e06d1a779eb6 store: stop relying on a `revlog_type` property Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50667 diff changeset	1222 details=details,
50493 e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	1223 )
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	1224 if _match_tracked_entry(entry, matcher):
e50d1fe7ebb4 store: issue a single entry for each revlog Pierre-Yves David <pierre-yves.david@octobus.net> parents: 50492 diff changeset	1225 yield entry
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1226
7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1227 def copylist(self):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	1228 d = (
45351 909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	1229 b'bookmarks',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	1230 b'narrowspec',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	1231 b'data',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	1232 b'meta',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	1233 b'dh',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	1234 b'fncache',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	1235 b'phaseroots',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	1236 b'obsstore',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	1237 b'00manifest.d',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	1238 b'00manifest.i',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	1239 b'00changelog.d',
909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	1240 b'00changelog.i',
45483 d252f51ab032 share: introduce config option to store requires in .hg/store Pulkit Goyal <7895pulkit@gmail.com> parents: 45351 diff changeset	1241 b'requires',
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 42905 diff changeset	1242 )
45351 909dafff6a78 store: refactor space delimited list to proper data structure Pulkit Goyal <7895pulkit@gmail.com> parents: 44452 diff changeset	1243 return [b'requires', b'00changelog.i'] + [b'store/' + f for f in d]
7229 7946503ec76e introduce fncache repository layout Adrian Buehlmann <adrian@cadifra.com> parents: 6989 diff changeset	1244
20883 cd443c7589cc fncache: move fncache writing to be in a transaction Durham Goode <durham@fb.com> parents: 20879 diff changeset	1245 def write(self, tr):
cd443c7589cc fncache: move fncache writing to be in a transaction Durham Goode <durham@fb.com> parents: 20879 diff changeset	1246 self.fncache.write(tr)
13391 d00bbff8600e fncachestore: defer updating the fncache file to a single file open Adrian Buehlmann <adrian@cadifra.com> parents: 13169 diff changeset	1247
20884 2efdd186925d caches: invalidate store caches when lock is taken Durham Goode <durham@fb.com> parents: 20883 diff changeset	1248 def invalidatecaches(self):
2efdd186925d caches: invalidate store caches when lock is taken Durham Goode <durham@fb.com> parents: 20883 diff changeset	1249 self.fncache.entries = None
40736 0728d87a8631 store: append to fncache if there are only new files to write Pulkit Goyal <pulkit@yandex-team.ru> parents: 40624 diff changeset	1250 self.fncache.addls = set()
20884 2efdd186925d caches: invalidate store caches when lock is taken Durham Goode <durham@fb.com> parents: 20883 diff changeset	1251
20885 f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1252 def markremoved(self, fn):
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1253 self.fncache.remove(fn)
f49d60fa40a5 fncache: clean up fncache during strips Durham Goode <durham@fb.com> parents: 20884 diff changeset	1254
17783 df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1255 def _exists(self, f):
df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1256 ef = self.encode(f)
df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1257 try:
df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1258 self.getsize(ef)
df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1259 return True
49306 2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT Manuel Jacob <me@manueljacob.de> parents: 49284 diff changeset	1260 except FileNotFoundError:
17783 df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1261 return False
df55ce6854c3 store: add new _exists helper function on fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17782 diff changeset	1262
17745 b9a56b816ff2 store: add a contains method to fncachestore smuralid parents: 17744 diff changeset	1263 def __contains__(self, path):
b9a56b816ff2 store: add a contains method to fncachestore smuralid parents: 17744 diff changeset	1264 '''Checks if the store contains path'''
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1265 path = b"/".join((b"data", path))
17782 8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1266 # check for files (exact match)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1267 e = path + b'.i'
17784 73e1ab39792c store: fncache may contain non-existent entries (fixes b9a56b816ff2) Adrian Buehlmann <adrian@cadifra.com> parents: 17783 diff changeset	1268 if e in self.fncache and self._exists(e):
17782 8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1269 return True
8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1270 # now check for directories (prefix match)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1271 if not path.endswith(b'/'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	1272 path += b'/'
17782 8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1273 for e in self.fncache:
17784 73e1ab39792c store: fncache may contain non-existent entries (fixes b9a56b816ff2) Adrian Buehlmann <adrian@cadifra.com> parents: 17783 diff changeset	1274 if e.startswith(path) and self._exists(e):
17782 8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1275 return True
8095306c1fb2 store: move __contains__() implementation from class fncache into fncachestore Adrian Buehlmann <adrian@cadifra.com> parents: 17747 diff changeset	1276 return False

Mercurial > public > mercurial-scm > hg

annotate mercurial/store.py @ 52639:9db77d46de79