comparison mercurial/revlog.py @ 50638:5460424092e2

stream-clone: smoothly detect and handle a case were a revlog is split This detect and handle the most common case for a race condition around stream and revlog splitting. The one were the revlog is split between the initial collection of data and the time were we start considering stream that data. In such case, we repatch an inlined version of that revlog together when this happens. This is necessary as stream-v2 promised a specific number of bytes and a specific number of files to the client. In stream-v3, we will have the opportunity to just send a split revlog instead. Getting a better version of the protocol for stream-v3 is still useful, but it is no longer a blocket to fix that race condition. Note that another, rarer race condition exist, were the revlog is split while we creating the revlog and extracing content from it. This can be dealt with later.
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Mon, 29 May 2023 18:41:58 +0200
parents 9caa860dcbec
children 3b56395404a1
comparison
equal deleted inserted replaced
50637:9caa860dcbec 50638:5460424092e2
504 else: 504 else:
505 return fp.read(size) 505 return fp.read(size)
506 except FileNotFoundError: 506 except FileNotFoundError:
507 return b'' 507 return b''
508 508
509 def get_streams(self, max_linkrev): 509 def get_streams(self, max_linkrev, force_inline=False):
510 n = len(self) 510 n = len(self)
511 index = self.index 511 index = self.index
512 while n > 0: 512 while n > 0:
513 linkrev = index[n - 1][4] 513 linkrev = index[n - 1][4]
514 if linkrev < max_linkrev: 514 if linkrev < max_linkrev:
539 size = index_size + data_size 539 size = index_size + data_size
540 if size <= 65536: 540 if size <= 65536:
541 yield fp.read(size) 541 yield fp.read(size)
542 else: 542 else:
543 yield from util.filechunkiter(fp, limit=size) 543 yield from util.filechunkiter(fp, limit=size)
544
545 inline_stream = get_stream()
546 next(inline_stream)
547 return [
548 (self._indexfile, inline_stream, index_size + data_size),
549 ]
550 elif force_inline:
551
552 def get_stream():
553 with self._datafp() as fp_d:
554 yield None
555
556 for rev in range(n):
557 idx = self.index.entry_binary(rev)
558 if rev == 0 and self._docket is None:
559 # re-inject the inline flag
560 header = self._format_flags
561 header |= self._format_version
562 header |= FLAG_INLINE_DATA
563 header = self.index.pack_header(header)
564 idx = header + idx
565 yield idx
566 yield self._getsegmentforrevs(rev, rev, df=fp_d)[1]
544 567
545 inline_stream = get_stream() 568 inline_stream = get_stream()
546 next(inline_stream) 569 next(inline_stream)
547 return [ 570 return [
548 (self._indexfile, inline_stream, index_size + data_size), 571 (self._indexfile, inline_stream, index_size + data_size),