Mercurial > public > mercurial-scm > hg-stable

annotate mercurial/revlogutils/randomaccessfile.py @ 51032:3314c41c3759

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
randomaccessfile: drop explicit passing of file description The goal of this object is to manage IO, we still have to open the file if necessary, but this is all internal now.
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Sat, 30 Sep 2023 02:02:36 +0200
parents 642e31cb55f0
children 594f912818ab
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
47431
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
1 # Copyright Mercurial Contributors
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
2 #
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
3 # This software may be used and distributed according to the terms of the
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
4 # GNU General Public License version 2 or any later version.
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
5
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
6 import contextlib
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
7
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
8 from ..i18n import _
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
9 from .. import (
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
10 error,
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
11 util,
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
12 )
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
13
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
14
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
15 _MAX_CACHED_CHUNK_SIZE = 1048576 # 1 MiB
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
16
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
17 PARTIAL_READ_MSG = _(
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
18 b'partial read of revlog %s; expected %d bytes from offset %d, got %d'
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
19 )
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
20
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
21
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
22 def _is_power_of_two(n):
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
23 return (n & (n - 1) == 0) and n != 0
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
24
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
25
49037
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 47469
diff changeset
26 class randomaccessfile:
47431
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
27 """Accessing arbitrary chuncks of data within a file, with some caching"""
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
28
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
29 def __init__(
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
30 self,
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
31 opener,
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
32 filename,
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
33 default_cached_chunk_size,
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
34 initial_cache=None,
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
35 ):
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
36 # Required by bitwise manipulation below
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
37 assert _is_power_of_two(default_cached_chunk_size)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
38
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
39 self.opener = opener
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
40 self.filename = filename
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
41 self.default_cached_chunk_size = default_cached_chunk_size
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
42 self.writing_handle = None # This is set from revlog.py
47469
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
43 self.reading_handle = None
47431
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
44 self._cached_chunk = b''
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
45 self._cached_chunk_position = 0 # Offset from the start of the file
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
46 if initial_cache:
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
47 self._cached_chunk_position, self._cached_chunk = initial_cache
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
48
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
49 def clear_cache(self):
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
50 self._cached_chunk = b''
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
51 self._cached_chunk_position = 0
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
52
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
53 def _open(self, mode=b'r'):
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
54 """Return a file object"""
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
55 return self.opener(self.filename, mode=mode)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
56
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
57 @contextlib.contextmanager
51032
3314c41c3759 randomaccessfile: drop explicit passing of file description
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49037
diff changeset
58 def _read_handle(self):
47431
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
59 """File object suitable for reading data"""
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
60 # Use a file handle being actively used for writes, if available.
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
61 # There is some danger to doing this because reads will seek the
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
62 # file. However, revlog._writeentry performs a SEEK_END before all
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
63 # writes, so we should be safe.
51032
3314c41c3759 randomaccessfile: drop explicit passing of file description
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49037
diff changeset
64 if self.writing_handle:
47431
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
65 yield self.writing_handle
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
66
47469
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
67 elif self.reading_handle:
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
68 yield self.reading_handle
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
69
47431
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
70 # Otherwise open a new file handle.
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
71 else:
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
72 with self._open() as fp:
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
73 yield fp
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
74
47469
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
75 @contextlib.contextmanager
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
76 def reading(self):
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
77 """Context manager that keeps the file open for reading"""
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
78 if (
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
79 self.reading_handle is None
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
80 and self.writing_handle is None
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
81 and self.filename is not None
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
82 ):
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
83 with self._open() as fp:
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
84 self.reading_handle = fp
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
85 try:
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
86 yield
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
87 finally:
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
88 self.reading_handle = None
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
89 else:
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
90 yield
5fa083a5ff04 copies: Keep changelog sidedata file open during copy tracing
Simon Sapin <simon.sapin@octobus.net>
parents: 47431
diff changeset
91
51032
3314c41c3759 randomaccessfile: drop explicit passing of file description
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49037
diff changeset
92 def read_chunk(self, offset, length):
47431
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
93 """Read a chunk of bytes from the file.
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
94
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
95 Accepts an absolute offset, length to read, and an optional existing
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
96 file handle to read from.
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
97
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
98 If an existing file handle is passed, it will be seeked and the
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
99 original seek position will NOT be restored.
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
100
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
101 Returns a str or buffer of raw byte data.
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
102
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
103 Raises if the requested number of bytes could not be read.
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
104 """
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
105 end = offset + length
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
106 cache_start = self._cached_chunk_position
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
107 cache_end = cache_start + len(self._cached_chunk)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
108 # Is the requested chunk within the cache?
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
109 if cache_start <= offset and end <= cache_end:
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
110 if cache_start == offset and end == cache_end:
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
111 return self._cached_chunk # avoid a copy
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
112 relative_start = offset - cache_start
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
113 return util.buffer(self._cached_chunk, relative_start, length)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
114
51032
3314c41c3759 randomaccessfile: drop explicit passing of file description
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49037
diff changeset
115 return self._read_and_update_cache(offset, length)
47431
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
116
51032
3314c41c3759 randomaccessfile: drop explicit passing of file description
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49037
diff changeset
117 def _read_and_update_cache(self, offset, length):
47431
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
118 # Cache data both forward and backward around the requested
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
119 # data, in a fixed size window. This helps speed up operations
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
120 # involving reading the revlog backwards.
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
121 real_offset = offset & ~(self.default_cached_chunk_size - 1)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
122 real_length = (
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
123 (offset + length + self.default_cached_chunk_size)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
124 & ~(self.default_cached_chunk_size - 1)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
125 ) - real_offset
51032
3314c41c3759 randomaccessfile: drop explicit passing of file description
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 49037
diff changeset
126 with self._read_handle() as file_obj:
47431
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
127 file_obj.seek(real_offset)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
128 data = file_obj.read(real_length)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
129
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
130 self._add_cached_chunk(real_offset, data)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
131
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
132 relative_offset = offset - real_offset
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
133 got = len(data) - relative_offset
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
134 if got < length:
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
135 message = PARTIAL_READ_MSG % (self.filename, length, offset, got)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
136 raise error.RevlogError(message)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
137
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
138 if offset != real_offset or real_length != length:
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
139 return util.buffer(data, relative_offset, length)
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
140 return data
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
141
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
142 def _add_cached_chunk(self, offset, data):
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
143 """Add to or replace the cached data chunk.
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
144
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
145 Accepts an absolute offset and the data that is at that location.
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
146 """
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
147 if (
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
148 self._cached_chunk_position + len(self._cached_chunk) == offset
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
149 and len(self._cached_chunk) + len(data) < _MAX_CACHED_CHUNK_SIZE
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
150 ):
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
151 # add to existing cache
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
152 self._cached_chunk += data
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
153 else:
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
154 self._cached_chunk = data
e0a314bcbc9d revlog: Extract low-level random-access file read caching logic
Simon Sapin <simon.sapin@octobus.net>
parents:
diff changeset
155 self._cached_chunk_position = offset