mercurial-scm/hg-stable: mercurial/revlogutils/randomaccessfile.py annotate

annotate mercurial/revlogutils/randomaccessfile.py @ 51902:1c5810ce737e

typing: add `from __future__ import annotations` to remaining source files Most of these look newer than when the original imports referenced in the previous commit were dropped, so these weren't covered by the backout. These were found with: hg files mercurial hgext hgext3rd -I '**.py' -X '**/thirdparty' \ | xargs grep -L 'from __future__ import annotations' All of the `__init__.py` files that finds are empty, so those were ignored and the rest manually edited.

author	Matt Harbison <matt_harbison@yahoo.com>
date	Thu, 12 Sep 2024 16:27:58 -0400
parents	eb9dea148233
children

rev	line source
47431 e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	1 # Copyright Mercurial Contributors
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	2 #
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	3 # This software may be used and distributed according to the terms of the
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	4 # GNU General Public License version 2 or any later version.
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	5
51902 1c5810ce737e typing: add `from __future__ import annotations` to remaining source files Matt Harbison <matt_harbison@yahoo.com> parents: 51893 diff changeset	6 from __future__ import annotations
1c5810ce737e typing: add `from __future__ import annotations` to remaining source files Matt Harbison <matt_harbison@yahoo.com> parents: 51893 diff changeset	7
47431 e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	8 import contextlib
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	9
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	10 from ..i18n import _
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	11 from .. import (
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	12 error,
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	13 util,
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	14 )
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	15
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	16
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	17 _MAX_CACHED_CHUNK_SIZE = 1048576 # 1 MiB
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	18
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	19 PARTIAL_READ_MSG = _(
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	20 b'partial read of revlog %s; expected %d bytes from offset %d, got %d'
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	21 )
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	22
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	23
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	24 def _is_power_of_two(n):
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	25 return (n & (n - 1) == 0) and n != 0
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	26
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	27
51107 222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	28 class appender:
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	29 """the changelog index must be updated last on disk, so we use this class
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	30 to delay writes to it"""
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	31
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	32 def __init__(self, vfs, name, mode, buf):
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	33 self.data = buf
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	34 fp = vfs(name, mode)
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	35 self.fp = fp
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	36 self.offset = fp.tell()
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	37 self.size = vfs.fstat(fp).st_size
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	38 self._end = self.size
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	39
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	40 def end(self):
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	41 return self._end
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	42
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	43 def tell(self):
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	44 return self.offset
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	45
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	46 def flush(self):
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	47 pass
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	48
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	49 @property
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	50 def closed(self):
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	51 return self.fp.closed
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	52
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	53 def close(self):
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	54 self.fp.close()
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	55
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	56 def seek(self, offset, whence=0):
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	57 '''virtual file offset spans real file and data'''
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	58 if whence == 0:
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	59 self.offset = offset
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	60 elif whence == 1:
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	61 self.offset += offset
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	62 elif whence == 2:
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	63 self.offset = self.end() + offset
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	64 if self.offset < self.size:
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	65 self.fp.seek(self.offset)
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	66
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	67 def read(self, count=-1):
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	68 '''only trick here is reads that span real file and data'''
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	69 ret = b""
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	70 if self.offset < self.size:
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	71 s = self.fp.read(count)
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	72 ret = s
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	73 self.offset += len(s)
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	74 if count > 0:
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	75 count -= len(s)
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	76 if count != 0:
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	77 doff = self.offset - self.size
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	78 self.data.insert(0, b"".join(self.data))
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	79 del self.data[1:]
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	80 s = self.data[0][doff : doff + count]
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	81 self.offset += len(s)
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	82 ret += s
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	83 return ret
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	84
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	85 def write(self, s):
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	86 self.data.append(bytes(s))
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	87 self.offset += len(s)
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	88 self._end += len(s)
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	89
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	90 def __enter__(self):
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	91 self.fp.__enter__()
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	92 return self
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	93
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	94 def __exit__(self, *args):
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	95 return self.fp.__exit__(*args)
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	96
222b89224397 changelog-delay: move the appender class next to randomaccessfile Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51106 diff changeset	97
49037 642e31cb55f0 py3: use class X: instead of class X(object): Gregory Szorc <gregory.szorc@gmail.com> parents: 47469 diff changeset	98 class randomaccessfile:
47431 e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	99 """Accessing arbitrary chuncks of data within a file, with some caching"""
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	100
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	101 def __init__(
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	102 self,
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	103 opener,
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	104 filename,
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	105 default_cached_chunk_size,
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	106 initial_cache=None,
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	107 ):
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	108 # Required by bitwise manipulation below
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	109 assert _is_power_of_two(default_cached_chunk_size)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	110
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	111 self.opener = opener
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	112 self.filename = filename
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	113 self.default_cached_chunk_size = default_cached_chunk_size
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	114 self.writing_handle = None # This is set from revlog.py
47469 5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	115 self.reading_handle = None
47431 e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	116 self._cached_chunk = b''
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	117 self._cached_chunk_position = 0 # Offset from the start of the file
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	118 if initial_cache:
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	119 self._cached_chunk_position, self._cached_chunk = initial_cache
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	120
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	121 def clear_cache(self):
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	122 self._cached_chunk = b''
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	123 self._cached_chunk_position = 0
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	124
51106 594f912818ab changelog-delay: adds some check around delaying and diverting write Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51032 diff changeset	125 @property
594f912818ab changelog-delay: adds some check around delaying and diverting write Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51032 diff changeset	126 def is_open(self):
594f912818ab changelog-delay: adds some check around delaying and diverting write Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51032 diff changeset	127 """True if any file handle is being held
594f912818ab changelog-delay: adds some check around delaying and diverting write Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51032 diff changeset	128
594f912818ab changelog-delay: adds some check around delaying and diverting write Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51032 diff changeset	129 Used for assert and debug in the python code"""
594f912818ab changelog-delay: adds some check around delaying and diverting write Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51032 diff changeset	130 return (
594f912818ab changelog-delay: adds some check around delaying and diverting write Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51032 diff changeset	131 self.reading_handle is not None or self.writing_handle is not None
594f912818ab changelog-delay: adds some check around delaying and diverting write Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51032 diff changeset	132 )
594f912818ab changelog-delay: adds some check around delaying and diverting write Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51032 diff changeset	133
47431 e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	134 def _open(self, mode=b'r'):
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	135 """Return a file object"""
51203 a93e52f0b6ff changelog: disallow delayed write on inline changesets Pierre-Yves David <pierre-yves.david@octobus.net> parents: 51110 diff changeset	136 return self.opener(self.filename, mode=mode)
47431 e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	137
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	138 @contextlib.contextmanager
51032 3314c41c3759 randomaccessfile: drop explicit passing of file description Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49037 diff changeset	139 def _read_handle(self):
47431 e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	140 """File object suitable for reading data"""
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	141 # Use a file handle being actively used for writes, if available.
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	142 # There is some danger to doing this because reads will seek the
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	143 # file. However, revlog._writeentry performs a SEEK_END before all
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	144 # writes, so we should be safe.
51032 3314c41c3759 randomaccessfile: drop explicit passing of file description Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49037 diff changeset	145 if self.writing_handle:
47431 e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	146 yield self.writing_handle
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	147
47469 5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	148 elif self.reading_handle:
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	149 yield self.reading_handle
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	150
47431 e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	151 # Otherwise open a new file handle.
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	152 else:
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	153 with self._open() as fp:
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	154 yield fp
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	155
47469 5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	156 @contextlib.contextmanager
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	157 def reading(self):
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	158 """Context manager that keeps the file open for reading"""
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	159 if (
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	160 self.reading_handle is None
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	161 and self.writing_handle is None
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	162 and self.filename is not None
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	163 ):
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	164 with self._open() as fp:
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	165 self.reading_handle = fp
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	166 try:
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	167 yield
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	168 finally:
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	169 self.reading_handle = None
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	170 else:
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	171 yield
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing Simon Sapin <simon.sapin@octobus.net> parents: 47431 diff changeset	172
51032 3314c41c3759 randomaccessfile: drop explicit passing of file description Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49037 diff changeset	173 def read_chunk(self, offset, length):
47431 e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	174 """Read a chunk of bytes from the file.
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	175
51893 eb9dea148233 revlog: cleanup some outdated docstrings Rapha?l Gom?s <rgomes@octobus.net> parents: 51203 diff changeset	176 Accepts an absolute offset, length to read.
47431 e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	177
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	178 Returns a str or buffer of raw byte data.
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	179
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	180 Raises if the requested number of bytes could not be read.
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	181 """
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	182 end = offset + length
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	183 cache_start = self._cached_chunk_position
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	184 cache_end = cache_start + len(self._cached_chunk)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	185 # Is the requested chunk within the cache?
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	186 if cache_start <= offset and end <= cache_end:
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	187 if cache_start == offset and end == cache_end:
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	188 return self._cached_chunk # avoid a copy
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	189 relative_start = offset - cache_start
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	190 return util.buffer(self._cached_chunk, relative_start, length)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	191
51032 3314c41c3759 randomaccessfile: drop explicit passing of file description Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49037 diff changeset	192 return self._read_and_update_cache(offset, length)
47431 e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	193
51032 3314c41c3759 randomaccessfile: drop explicit passing of file description Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49037 diff changeset	194 def _read_and_update_cache(self, offset, length):
47431 e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	195 # Cache data both forward and backward around the requested
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	196 # data, in a fixed size window. This helps speed up operations
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	197 # involving reading the revlog backwards.
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	198 real_offset = offset & ~(self.default_cached_chunk_size - 1)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	199 real_length = (
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	200 (offset + length + self.default_cached_chunk_size)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	201 & ~(self.default_cached_chunk_size - 1)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	202 ) - real_offset
51032 3314c41c3759 randomaccessfile: drop explicit passing of file description Pierre-Yves David <pierre-yves.david@octobus.net> parents: 49037 diff changeset	203 with self._read_handle() as file_obj:
47431 e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	204 file_obj.seek(real_offset)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	205 data = file_obj.read(real_length)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	206
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	207 self._add_cached_chunk(real_offset, data)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	208
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	209 relative_offset = offset - real_offset
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	210 got = len(data) - relative_offset
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	211 if got < length:
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	212 message = PARTIAL_READ_MSG % (self.filename, length, offset, got)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	213 raise error.RevlogError(message)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	214
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	215 if offset != real_offset or real_length != length:
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	216 return util.buffer(data, relative_offset, length)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	217 return data
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	218
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	219 def _add_cached_chunk(self, offset, data):
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	220 """Add to or replace the cached data chunk.
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	221
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	222 Accepts an absolute offset and the data that is at that location.
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	223 """
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	224 if (
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	225 self._cached_chunk_position + len(self._cached_chunk) == offset
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	226 and len(self._cached_chunk) + len(data) < _MAX_CACHED_CHUNK_SIZE
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	227 ):
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	228 # add to existing cache
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	229 self._cached_chunk += data
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	230 else:
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	231 self._cached_chunk = data
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic Simon Sapin <simon.sapin@octobus.net> parents: diff changeset	232 self._cached_chunk_position = offset

Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/revlogutils/randomaccessfile.py @ 51902:1c5810ce737e