mercurial-scm/hg-stable: hgext/remotefilelog/datapack.py annotate

annotate hgext/remotefilelog/datapack.py @ 53040:cdd7bf612c7b stable tip

bundle-spec: properly format boolean parameter (issue6960) This was breaking automatic clone bundle generation. This changeset fixes it and add a test to catch it in the future.

author	Pierre-Yves David <pierre-yves.david@octobus.net>
date	Tue, 11 Mar 2025 02:29:42 +0100
parents	5cc8deb96b48
children

rev	line source
51901 f4733654f144 typing: add `from __future__ import annotations` to most files Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	1 from __future__ import annotations
f4733654f144 typing: add `from __future__ import annotations` to most files Matt Harbison <matt_harbison@yahoo.com> parents: 51762 diff changeset	2
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	3 import struct
40557 e2a1584e9e3f remotefilelog: rip out lz4 support Augie Fackler <augie@google.com> parents: 40556 diff changeset	4 import zlib
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	5
47055 d55b71393907 node: replace nullid and friends with nodeconstants class Joerg Sonnenberger <joerg@bec.de> parents: 44470 diff changeset	6 from mercurial.node import (
d55b71393907 node: replace nullid and friends with nodeconstants class Joerg Sonnenberger <joerg@bec.de> parents: 44470 diff changeset	7 hex,
d55b71393907 node: replace nullid and friends with nodeconstants class Joerg Sonnenberger <joerg@bec.de> parents: 44470 diff changeset	8 sha1nodeconstants,
d55b71393907 node: replace nullid and friends with nodeconstants class Joerg Sonnenberger <joerg@bec.de> parents: 44470 diff changeset	9 )
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	10 from mercurial.i18n import _
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	11 from mercurial import (
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	12 util,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	13 )
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	14 from . import (
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	15 basepack,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	16 constants,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	17 shallowutil,
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	18 )
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	19
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	20 NODELENGTH = 20
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	21
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	22 # The indicator value in the index for a fulltext entry.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	23 FULLTEXTINDEXMARK = -1
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	24 NOBASEINDEXMARK = -2
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	25
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	26 INDEXSUFFIX = b'.dataidx'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	27 PACKSUFFIX = b'.datapack'
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	28
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	29
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	30 class datapackstore(basepack.basepackstore):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	31 INDEXSUFFIX = INDEXSUFFIX
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	32 PACKSUFFIX = PACKSUFFIX
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	33
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	34 def __init__(self, ui, path):
52668 5cc8deb96b48 pyupgrade: modernize calls to superclass methods Matt Harbison <matt_harbison@yahoo.com> parents: 51901 diff changeset	35 super().__init__(ui, path)
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	36
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	37 def getpack(self, path):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	38 return datapack(path)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	39
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	40 def get(self, name, node):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	41 raise RuntimeError(b"must use getdeltachain with datapackstore")
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	42
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	43 def getmeta(self, name, node):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	44 for pack in self.packs:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	45 try:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	46 return pack.getmeta(name, node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	47 except KeyError:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	48 pass
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	49
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	50 for pack in self.refresh():
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	51 try:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	52 return pack.getmeta(name, node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	53 except KeyError:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	54 pass
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	55
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	56 raise KeyError((name, hex(node)))
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	57
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	58 def getdelta(self, name, node):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	59 for pack in self.packs:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	60 try:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	61 return pack.getdelta(name, node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	62 except KeyError:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	63 pass
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	64
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	65 for pack in self.refresh():
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	66 try:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	67 return pack.getdelta(name, node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	68 except KeyError:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	69 pass
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	70
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	71 raise KeyError((name, hex(node)))
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	72
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	73 def getdeltachain(self, name, node):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	74 for pack in self.packs:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	75 try:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	76 return pack.getdeltachain(name, node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	77 except KeyError:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	78 pass
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	79
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	80 for pack in self.refresh():
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	81 try:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	82 return pack.getdeltachain(name, node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	83 except KeyError:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	84 pass
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	85
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	86 raise KeyError((name, hex(node)))
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	87
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	88 def add(self, name, node, data):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	89 raise RuntimeError(b"cannot add to datapackstore")
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	90
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	91
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	92 class datapack(basepack.basepack):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	93 INDEXSUFFIX = INDEXSUFFIX
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	94 PACKSUFFIX = PACKSUFFIX
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	95
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	96 # Format is <node><delta offset><pack data offset><pack data size>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	97 # See the mutabledatapack doccomment for more details.
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	98 INDEXFORMAT = b'!20siQQ'
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	99 INDEXENTRYLENGTH = 40
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	100
40556 10c10da14c5d remotefilelog: bump pack file formats to version 2 Augie Fackler <augie@google.com> parents: 40545 diff changeset	101 SUPPORTED_VERSIONS = [2]
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	102
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	103 def getmissing(self, keys):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	104 missing = []
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	105 for name, node in keys:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	106 value = self._find(node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	107 if not value:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	108 missing.append((name, node))
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	109
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	110 return missing
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	111
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	112 def get(self, name, node):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	113 raise RuntimeError(
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	114 b"must use getdeltachain with datapack (%s:%s)" % (name, hex(node))
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	115 )
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	116
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	117 def getmeta(self, name, node):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	118 value = self._find(node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	119 if value is None:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	120 raise KeyError((name, hex(node)))
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	121
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	122 node, deltabaseoffset, offset, size = value
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	123 rawentry = self._data[offset : offset + size]
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	124
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	125 # see docstring of mutabledatapack for the format
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	126 offset = 0
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	127 offset += struct.unpack_from(b'!H', rawentry, offset)[0] + 2 # filename
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	128 offset += 40 # node, deltabase node
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	129 offset += struct.unpack_from(b'!Q', rawentry, offset)[0] + 8 # delta
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	130
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	131 metalen = struct.unpack_from(b'!I', rawentry, offset)[0]
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	132 offset += 4
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	133
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	134 meta = shallowutil.parsepackmeta(rawentry[offset : offset + metalen])
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	135
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	136 return meta
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	137
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	138 def getdelta(self, name, node):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	139 value = self._find(node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	140 if value is None:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	141 raise KeyError((name, hex(node)))
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	142
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	143 node, deltabaseoffset, offset, size = value
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	144 entry = self._readentry(offset, size, getmeta=True)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	145 filename, node, deltabasenode, delta, meta = entry
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	146
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	147 # If we've read a lot of data from the mmap, free some memory.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	148 self.freememory()
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	149
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	150 return delta, filename, deltabasenode, meta
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	151
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	152 def getdeltachain(self, name, node):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	153 value = self._find(node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	154 if value is None:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	155 raise KeyError((name, hex(node)))
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	156
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	157 params = self.params
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	158
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	159 # Precompute chains
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	160 chain = [value]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	161 deltabaseoffset = value[1]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	162 entrylen = self.INDEXENTRYLENGTH
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	163 while (
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	164 deltabaseoffset != FULLTEXTINDEXMARK
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	165 and deltabaseoffset != NOBASEINDEXMARK
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	166 ):
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	167 loc = params.indexstart + deltabaseoffset
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	168 value = struct.unpack(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	169 self.INDEXFORMAT, self._index[loc : loc + entrylen]
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	170 )
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	171 deltabaseoffset = value[1]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	172 chain.append(value)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	173
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	174 # Read chain data
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	175 deltachain = []
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	176 for node, deltabaseoffset, offset, size in chain:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	177 filename, node, deltabasenode, delta = self._readentry(offset, size)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	178 deltachain.append((filename, node, filename, deltabasenode, delta))
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	179
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	180 # If we've read a lot of data from the mmap, free some memory.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	181 self.freememory()
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	182
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	183 return deltachain
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	184
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	185 def _readentry(self, offset, size, getmeta=False):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	186 rawentry = self._data[offset : offset + size]
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	187 self._pagedin += len(rawentry)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	188
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	189 # <2 byte len> + <filename>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	190 lengthsize = 2
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	191 filenamelen = struct.unpack(b'!H', rawentry[:2])[0]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	192 filename = rawentry[lengthsize : lengthsize + filenamelen]
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	193
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	194 # <20 byte node> + <20 byte deltabase>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	195 nodestart = lengthsize + filenamelen
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	196 deltabasestart = nodestart + NODELENGTH
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	197 node = rawentry[nodestart:deltabasestart]
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	198 deltabasenode = rawentry[deltabasestart : deltabasestart + NODELENGTH]
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	199
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	200 # <8 byte len> + <delta>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	201 deltastart = deltabasestart + NODELENGTH
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	202 rawdeltalen = rawentry[deltastart : deltastart + 8]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	203 deltalen = struct.unpack(b'!Q', rawdeltalen)[0]
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	204
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	205 delta = rawentry[deltastart + 8 : deltastart + 8 + deltalen]
40557 e2a1584e9e3f remotefilelog: rip out lz4 support Augie Fackler <augie@google.com> parents: 40556 diff changeset	206 delta = self._decompress(delta)
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	207
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	208 if getmeta:
40556 10c10da14c5d remotefilelog: bump pack file formats to version 2 Augie Fackler <augie@google.com> parents: 40545 diff changeset	209 metastart = deltastart + 8 + deltalen
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	210 metalen = struct.unpack_from(b'!I', rawentry, metastart)[0]
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	211
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	212 rawmeta = rawentry[metastart + 4 : metastart + 4 + metalen]
40556 10c10da14c5d remotefilelog: bump pack file formats to version 2 Augie Fackler <augie@google.com> parents: 40545 diff changeset	213 meta = shallowutil.parsepackmeta(rawmeta)
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	214 return filename, node, deltabasenode, delta, meta
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	215 else:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	216 return filename, node, deltabasenode, delta
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	217
40557 e2a1584e9e3f remotefilelog: rip out lz4 support Augie Fackler <augie@google.com> parents: 40556 diff changeset	218 def _decompress(self, data):
e2a1584e9e3f remotefilelog: rip out lz4 support Augie Fackler <augie@google.com> parents: 40556 diff changeset	219 return zlib.decompress(data)
e2a1584e9e3f remotefilelog: rip out lz4 support Augie Fackler <augie@google.com> parents: 40556 diff changeset	220
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	221 def add(self, name, node, data):
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	222 raise RuntimeError(b"cannot add to datapack (%s:%s)" % (name, node))
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	223
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	224 def _find(self, node):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	225 params = self.params
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	226 fanoutkey = struct.unpack(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	227 params.fanoutstruct, node[: params.fanoutprefix]
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	228 )[0]
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	229 fanout = self._fanouttable
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	230
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	231 start = fanout[fanoutkey] + params.indexstart
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	232 indexend = self._indexend
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	233
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	234 # Scan forward to find the first non-same entry, which is the upper
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	235 # bound.
49292 d44e3c45f0e4 py3: replace `pycompat.xrange` by `range` Manuel Jacob <me@manueljacob.de> parents: 49004 diff changeset	236 for i in range(fanoutkey + 1, params.fanoutcount):
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	237 end = fanout[i] + params.indexstart
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	238 if end != start:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	239 break
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	240 else:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	241 end = indexend
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	242
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	243 # Bisect between start and end to find node
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	244 index = self._index
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	245 startnode = index[start : start + NODELENGTH]
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	246 endnode = index[end : end + NODELENGTH]
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	247 entrylen = self.INDEXENTRYLENGTH
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	248 if startnode == node:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	249 entry = index[start : start + entrylen]
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	250 elif endnode == node:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	251 entry = index[end : end + entrylen]
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	252 else:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	253 while start < end - entrylen:
41541 eb37d95cc486 py3: use integer division instead of `int(...)` call Boris Feld <boris.feld@octobus.net> parents: 41489 diff changeset	254 mid = start + (end - start) // 2
eb37d95cc486 py3: use integer division instead of `int(...)` call Boris Feld <boris.feld@octobus.net> parents: 41489 diff changeset	255 mid = mid - ((mid - params.indexstart) % entrylen)
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	256 midnode = index[mid : mid + NODELENGTH]
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	257 if midnode == node:
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	258 entry = index[mid : mid + entrylen]
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	259 break
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	260 if node > midnode:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	261 start = mid
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	262 elif node < midnode:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	263 end = mid
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	264 else:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	265 return None
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	266
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	267 return struct.unpack(self.INDEXFORMAT, entry)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	268
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	269 def markledger(self, ledger, options=None):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	270 for filename, node in self:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	271 ledger.markdataentry(self, filename, node)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	272
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	273 def cleanup(self, ledger):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	274 entries = ledger.sources.get(self, [])
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	275 allkeys = set(self)
44470 9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things Augie Fackler <augie@google.com> parents: 43105 diff changeset	276 repackedkeys = {
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	277 (e.filename, e.node) for e in entries if e.datarepacked or e.gced
44470 9d2b2df2c2ba cleanup: run pyupgrade on our source tree to clean up varying things Augie Fackler <augie@google.com> parents: 43105 diff changeset	278 }
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	279
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	280 if len(allkeys - repackedkeys) == 0:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	281 if self.path not in ledger.created:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	282 util.unlinkpath(self.indexpath, ignoremissing=True)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	283 util.unlinkpath(self.packpath, ignoremissing=True)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	284
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	285 def __iter__(self):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	286 for f, n, deltabase, deltalen in self.iterentries():
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	287 yield f, n
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	288
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	289 def iterentries(self):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	290 # Start at 1 to skip the header
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	291 offset = 1
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	292 data = self._data
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	293 while offset < self.datasize:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	294 oldoffset = offset
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	295
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	296 # <2 byte len> + <filename>
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	297 filenamelen = struct.unpack(b'!H', data[offset : offset + 2])[0]
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	298 offset += 2
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	299 filename = data[offset : offset + filenamelen]
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	300 offset += filenamelen
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	301
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	302 # <20 byte node>
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	303 node = data[offset : offset + constants.NODESIZE]
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	304 offset += constants.NODESIZE
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	305 # <20 byte deltabase>
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	306 deltabase = data[offset : offset + constants.NODESIZE]
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	307 offset += constants.NODESIZE
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	308
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	309 # <8 byte len> + <delta>
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	310 rawdeltalen = data[offset : offset + 8]
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	311 deltalen = struct.unpack(b'!Q', rawdeltalen)[0]
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	312 offset += 8
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	313
40557 e2a1584e9e3f remotefilelog: rip out lz4 support Augie Fackler <augie@google.com> parents: 40556 diff changeset	314 # TODO(augie): we should store a header that is the
e2a1584e9e3f remotefilelog: rip out lz4 support Augie Fackler <augie@google.com> parents: 40556 diff changeset	315 # uncompressed size.
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	316 uncompressedlen = len(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	317 self._decompress(data[offset : offset + deltalen])
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	318 )
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	319 offset += deltalen
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	320
40556 10c10da14c5d remotefilelog: bump pack file formats to version 2 Augie Fackler <augie@google.com> parents: 40545 diff changeset	321 # <4 byte len> + <metadata-list>
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	322 metalen = struct.unpack_from(b'!I', data, offset)[0]
40556 10c10da14c5d remotefilelog: bump pack file formats to version 2 Augie Fackler <augie@google.com> parents: 40545 diff changeset	323 offset += 4 + metalen
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	324
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	325 yield (filename, node, deltabase, uncompressedlen)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	326
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	327 # If we've read a lot of data from the mmap, free some memory.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	328 self._pagedin += offset - oldoffset
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	329 if self.freememory():
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	330 data = self._data
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	331
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	332
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	333 class mutabledatapack(basepack.mutablebasepack):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	334 """A class for constructing and serializing a datapack file and index.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	335
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	336 A datapack is a pair of files that contain the revision contents for various
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	337 file revisions in Mercurial. It contains only revision contents (like file
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	338 contents), not any history information.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	339
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	340 It consists of two files, with the following format. All bytes are in
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	341 network byte order (big endian).
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	342
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	343 .datapack
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	344 The pack itself is a series of revision deltas with some basic header
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	345 information on each. A revision delta may be a fulltext, represented by
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	346 a deltabasenode equal to the nullid.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	347
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	348 datapack = <version: 1 byte>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	349 [<revision>,...]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	350 revision = <filename len: 2 byte unsigned int>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	351 <filename>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	352 <node: 20 byte>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	353 <deltabasenode: 20 byte>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	354 <delta len: 8 byte unsigned int>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	355 <delta>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	356 <metadata-list len: 4 byte unsigned int> [1]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	357 <metadata-list> [1]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	358 metadata-list = [<metadata-item>, ...]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	359 metadata-item = <metadata-key: 1 byte>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	360 <metadata-value len: 2 byte unsigned>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	361 <metadata-value>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	362
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	363 metadata-key could be METAKEYFLAG or METAKEYSIZE or other single byte
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	364 value in the future.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	365
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	366 .dataidx
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	367 The index file consists of two parts, the fanout and the index.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	368
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	369 The index is a list of index entries, sorted by node (one per revision
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	370 in the pack). Each entry has:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	371
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	372 - node (The 20 byte node of the entry; i.e. the commit hash, file node
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	373 hash, etc)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	374 - deltabase index offset (The location in the index of the deltabase for
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	375 this entry. The deltabase is the next delta in
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	376 the chain, with the chain eventually
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	377 terminating in a full-text, represented by a
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	378 deltabase offset of -1. This lets us compute
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	379 delta chains from the index, then do
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	380 sequential reads from the pack if the revision
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	381 are nearby on disk.)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	382 - pack entry offset (The location of this entry in the datapack)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	383 - pack content size (The on-disk length of this entry's pack data)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	384
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	385 The fanout is a quick lookup table to reduce the number of steps for
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	386 bisecting the index. It is a series of 4 byte pointers to positions
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	387 within the index. It has 2^16 entries, which corresponds to hash
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	388 prefixes [0000, 0001,..., FFFE, FFFF]. Example: the pointer in slot
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	389 4F0A points to the index position of the first revision whose node
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	390 starts with 4F0A. This saves log(2^16)=16 bisect steps.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	391
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	392 dataidx = <fanouttable>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	393 <index>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	394 fanouttable = [<index offset: 4 byte unsigned int>,...] (2^16 entries)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	395 index = [<index entry>,...]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	396 indexentry = <node: 20 byte>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	397 <deltabase location: 4 byte signed int>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	398 <pack entry offset: 8 byte unsigned int>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	399 <pack entry size: 8 byte unsigned int>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	400
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	401 [1]: new in version 1.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	402 """
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	403
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	404 INDEXSUFFIX = INDEXSUFFIX
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	405 PACKSUFFIX = PACKSUFFIX
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	406
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	407 # v[01] index format: <node><delta offset><pack data offset><pack data size>
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	408 INDEXFORMAT = datapack.INDEXFORMAT
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	409 INDEXENTRYLENGTH = datapack.INDEXENTRYLENGTH
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	410
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	411 # v1 has metadata support
40556 10c10da14c5d remotefilelog: bump pack file formats to version 2 Augie Fackler <augie@google.com> parents: 40545 diff changeset	412 SUPPORTED_VERSIONS = [2]
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	413
40557 e2a1584e9e3f remotefilelog: rip out lz4 support Augie Fackler <augie@google.com> parents: 40556 diff changeset	414 def _compress(self, data):
e2a1584e9e3f remotefilelog: rip out lz4 support Augie Fackler <augie@google.com> parents: 40556 diff changeset	415 return zlib.compress(data)
e2a1584e9e3f remotefilelog: rip out lz4 support Augie Fackler <augie@google.com> parents: 40556 diff changeset	416
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	417 def add(self, name, node, deltabasenode, delta, metadata=None):
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	418 # metadata is a dict, ex. {METAKEYFLAG: flag}
51762 ca7bde5dbafb black: format the codebase with 23.3.0 Rapha?l Gom?s <rgomes@octobus.net> parents: 51759 diff changeset	419 if len(name) > 2**16:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	420 raise RuntimeError(_(b"name too long %s") % name)
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	421 if len(node) != 20:
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	422 raise RuntimeError(_(b"node should be 20 bytes %s") % node)
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	423
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	424 if node in self.entries:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	425 # The revision has already been added
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	426 return
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	427
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	428 # TODO: allow configurable compression
40557 e2a1584e9e3f remotefilelog: rip out lz4 support Augie Fackler <augie@google.com> parents: 40556 diff changeset	429 delta = self._compress(delta)
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	430
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	431 rawdata = b''.join(
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	432 (
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	433 struct.pack(b'!H', len(name)), # unsigned 2 byte int
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	434 name,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	435 node,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	436 deltabasenode,
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	437 struct.pack(b'!Q', len(delta)), # unsigned 8 byte int
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	438 delta,
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	439 )
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	440 )
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	441
40556 10c10da14c5d remotefilelog: bump pack file formats to version 2 Augie Fackler <augie@google.com> parents: 40545 diff changeset	442 # v1 support metadata
10c10da14c5d remotefilelog: bump pack file formats to version 2 Augie Fackler <augie@google.com> parents: 40545 diff changeset	443 rawmeta = shallowutil.buildpackmeta(metadata)
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	444 rawdata += struct.pack(b'!I', len(rawmeta)) # unsigned 4 byte
40556 10c10da14c5d remotefilelog: bump pack file formats to version 2 Augie Fackler <augie@google.com> parents: 40545 diff changeset	445 rawdata += rawmeta
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	446
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	447 offset = self.packfp.tell()
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	448
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	449 size = len(rawdata)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	450
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	451 self.entries[node] = (deltabasenode, offset, size)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	452
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	453 self.writeraw(rawdata)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	454
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	455 def createindex(self, nodelocations, indexoffset):
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	456 entries = sorted(
49004 f254fc73d956 global: bulk replace simple pycompat.iteritems(x) with x.items() Gregory Szorc <gregory.szorc@gmail.com> parents: 48966 diff changeset	457 (n, db, o, s) for n, (db, o, s) in self.entries.items()
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	458 )
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	459
43077 687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals Augie Fackler <augie@google.com> parents: 43076 diff changeset	460 rawindex = b''
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	461 fmt = self.INDEXFORMAT
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	462 for node, deltabase, offset, size in entries:
47055 d55b71393907 node: replace nullid and friends with nodeconstants class Joerg Sonnenberger <joerg@bec.de> parents: 44470 diff changeset	463 if deltabase == sha1nodeconstants.nullid:
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	464 deltabaselocation = FULLTEXTINDEXMARK
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	465 else:
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	466 # Instead of storing the deltabase node in the index, let's
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	467 # store a pointer directly to the index entry for the deltabase.
43076 2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	468 deltabaselocation = nodelocations.get(
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	469 deltabase, NOBASEINDEXMARK
2372284d9457 formatting: blacken the codebase Augie Fackler <augie@google.com> parents: 41541 diff changeset	470 )
40545 3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	471
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	472 entry = struct.pack(fmt, node, deltabaselocation, offset, size)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	473 rawindex += entry
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	474
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental Augie Fackler <augie@google.com> parents: diff changeset	475 return rawindex

Mercurial > public > mercurial-scm > hg-stable

annotate hgext/remotefilelog/datapack.py @ 53040:cdd7bf612c7b stable tip