comparison mercurial/revlog.py @ 32717:19b9fc40cc51

revlog: skeleton support for version 2 revlogs There are a number of improvements we want to make to revlogs that will require a new version - version 2. It is unclear what the full set of improvements will be or when we'll be done with them. What I do know is that the process will likely take longer than a single release, will require input from various stakeholders to evaluate changes, and will have many contentious debates and bikeshedding. It is unrealistic to develop revlog version 2 up front: there are just too many uncertainties that we won't know until things are implemented and experiments are run. Some changes will also be invasive and prone to bit rot, so sitting on dozens of patches is not practical. This commit introduces skeleton support for version 2 revlogs in a way that is flexible and not bound by backwards compatibility concerns. An experimental repo requirement for denoting revlog v2 has been added. The requirement string has a sub-version component to it. This will allow us to declare multiple requirements in the course of developing revlog v2. Whenever we change the in-development revlog v2 format, we can tweak the string, creating a new requirement and locking out old clients. This will allow us to make as many backwards incompatible changes and experiments to revlog v2 as we want. In other words, we can land code and make meaningful progress towards revlog v2 while still maintaining extreme format flexibility up until the point we freeze the format and remove the experimental labels. To enable the new repo requirement, you must supply an experimental and undocumented config option. But not just any boolean flag will do: you need to explicitly use a value that no sane person should ever type. This is an additional guard against enabling revlog v2 on an installation it shouldn't be enabled on. The specific scenario I'm trying to prevent is say a user with a 4.4 client with a frozen format enabling the option but then downgrading to 4.3 and accidentally creating repos with an outdated and unsupported repo format. Requiring a "challenge" string should prevent this. Because the format is not yet finalized and I don't want to take any chances, revlog v2's version is currently 0xDEAD. I figure squatting on a value we're likely never to use as an actual revlog version to mean "internal testing only" is acceptable. And "dead" is easily recognized as something meaningful. There is a bunch of cleanup that is needed before work on revlog v2 begins in earnest. I plan on doing that work once this patch is accepted and we're comfortable with the idea of starting down this path.
author Gregory Szorc <gregory.szorc@gmail.com>
date Fri, 19 May 2017 20:29:11 -0700
parents af854b1b36f8
children fa247f53f647
comparison
equal deleted inserted replaced
32716:0c09afdf5704 32717:19b9fc40cc51
49 _zlibdecompress = zlib.decompress 49 _zlibdecompress = zlib.decompress
50 50
51 # revlog header flags 51 # revlog header flags
52 REVLOGV0 = 0 52 REVLOGV0 = 0
53 REVLOGV1 = 1 53 REVLOGV1 = 1
54 # Dummy value until file format is finalized.
55 # Reminder: change the bounds check in revlog.__init__ when this is changed.
56 REVLOGV2 = 0xDEAD
54 FLAG_INLINE_DATA = (1 << 16) 57 FLAG_INLINE_DATA = (1 << 16)
55 FLAG_GENERALDELTA = (1 << 17) 58 FLAG_GENERALDELTA = (1 << 17)
56 REVLOG_DEFAULT_FLAGS = FLAG_INLINE_DATA 59 REVLOG_DEFAULT_FLAGS = FLAG_INLINE_DATA
57 REVLOG_DEFAULT_FORMAT = REVLOGV1 60 REVLOG_DEFAULT_FORMAT = REVLOGV1
58 REVLOG_DEFAULT_VERSION = REVLOG_DEFAULT_FORMAT | REVLOG_DEFAULT_FLAGS 61 REVLOG_DEFAULT_VERSION = REVLOG_DEFAULT_FORMAT | REVLOG_DEFAULT_FLAGS
59 REVLOGV1_FLAGS = FLAG_INLINE_DATA | FLAG_GENERALDELTA 62 REVLOGV1_FLAGS = FLAG_INLINE_DATA | FLAG_GENERALDELTA
63 REVLOGV2_FLAGS = REVLOGV1_FLAGS
60 64
61 # revlog index flags 65 # revlog index flags
62 REVIDX_ISCENSORED = (1 << 15) # revision has censor metadata, must be verified 66 REVIDX_ISCENSORED = (1 << 15) # revision has censor metadata, must be verified
63 REVIDX_ELLIPSIS = (1 << 14) # revision hash does not match data (narrowhg) 67 REVIDX_ELLIPSIS = (1 << 14) # revision hash does not match data (narrowhg)
64 REVIDX_EXTSTORED = (1 << 13) # revision data is stored externally 68 REVIDX_EXTSTORED = (1 << 13) # revision data is stored externally
289 self._compengine = 'zlib' 293 self._compengine = 'zlib'
290 294
291 v = REVLOG_DEFAULT_VERSION 295 v = REVLOG_DEFAULT_VERSION
292 opts = getattr(opener, 'options', None) 296 opts = getattr(opener, 'options', None)
293 if opts is not None: 297 if opts is not None:
294 if 'revlogv1' in opts: 298 if 'revlogv2' in opts:
299 # version 2 revlogs always use generaldelta.
300 v = REVLOGV2 | FLAG_GENERALDELTA | FLAG_INLINE_DATA
301 elif 'revlogv1' in opts:
295 if 'generaldelta' in opts: 302 if 'generaldelta' in opts:
296 v |= FLAG_GENERALDELTA 303 v |= FLAG_GENERALDELTA
297 else: 304 else:
298 v = 0 305 v = 0
299 if 'chunkcachesize' in opts: 306 if 'chunkcachesize' in opts:
336 raise RevlogError(_('unknown flags (%#04x) in version %d ' 343 raise RevlogError(_('unknown flags (%#04x) in version %d '
337 'revlog %s') % 344 'revlog %s') %
338 (flags >> 16, fmt, self.indexfile)) 345 (flags >> 16, fmt, self.indexfile))
339 elif fmt == REVLOGV1: 346 elif fmt == REVLOGV1:
340 if flags & ~REVLOGV1_FLAGS: 347 if flags & ~REVLOGV1_FLAGS:
348 raise RevlogError(_('unknown flags (%#04x) in version %d '
349 'revlog %s') %
350 (flags >> 16, fmt, self.indexfile))
351 elif fmt == REVLOGV2:
352 if flags & ~REVLOGV2_FLAGS:
341 raise RevlogError(_('unknown flags (%#04x) in version %d ' 353 raise RevlogError(_('unknown flags (%#04x) in version %d '
342 'revlog %s') % 354 'revlog %s') %
343 (flags >> 16, fmt, self.indexfile)) 355 (flags >> 16, fmt, self.indexfile))
344 else: 356 else:
345 raise RevlogError(_('unknown version (%d) in revlog %s') % 357 raise RevlogError(_('unknown version (%d) in revlog %s') %