comparison mercurial/branchmap.py @ 48701:f8f2ecdde4b5

branchmap: skip obsolete revisions while computing heads It's time to make this part of core Mercurial obsolescence-aware. Not considering obsolete revisions when computing heads is clearly what Mercurial should do. But there are a couple of small issues: - Let's say tip of the repo is obsolete. There are two ways of finding tiprev for branchcache (both are in use): looking at input data for update() and looking at computed heads after update(). Previously, repo tip would be tiprev of the branchcache. With this patch, an obsolete revision can no longer be tiprev. And depending on what way we use for finding tiprev (input data vs computed heads) we'll get a different result. This is relevant when recomputing cache key from cache contents, and may lead to updating cache for obsolete revisions multiple times (not from scratch, because it still would be considered valid for a subset of revisions in the repo). - If all commits on a branch are obsolete, the branchcache will include that branch, but the list of heads will be empty (that's why there's now `if not heads` when recomputing tiprev/tipnode from cache contents). Having an entry for every branch is currently required for notify extension (and test-notify.t to pass), because notify doesn't handle revsets in its subscription config very well and will throw an error if e.g. a branch doesn't exist. - Cloning static HTTP repos may try to stat() a non-existent obsstore file. The issue is that we now care about obsolescence during clone, but statichttpvfs doesn't implement a stat method, so a regular vfs.stat() is used, and it assumes that file is local and calls os.stat(). During a clone, we're trying to stat() .hg/store/obsstore, but in static HTTP case we provide a literal URL to the obsstore file on the remote as if it were a local file path. On windows it actually results in a failure in test-static-http.t. The first issue is going to be addressed in a series dedicated to making sure branchcache is properly and timely written on disk (it wasn't perfect even before this patch, but there aren't enough tests to demonstrate that). The second issue will be addressed in a future patch for notify extension that will make it not raise an exception if a branch doesn't exist. And the third one was partially addressed in the previous patch in this series and will be properly fixed in a future patch when this series is accepted. filteredhash() grows a keyword argument to make sure that branchcache is also invalidated when there are new obsolete revisions in its repo view. This way the on-disk cache format is unchanged and compatible between versions (although it will obviously be recomputed when switching versions before/after this patch and the repo has obsolete revisions). There's one test that uses plain `hg up` without arguments while updated to a pruned commit. To make this test pass, simply return current working directory parent. Later in this series this code will be replaced by what prune command does: updating to the closest non-obsolete ancestor. Test changes: test-branch-change.t: update branch head and cache update message. The head of default listed in hg heads is changed because revision 2 was rewritten as 7, and 1 is the closest ancestor on the same branch, so it's the head of default now. The cache invalidation message appears now because of the cache hash change, since we're now accounting for obsolete revisions. Here's some context: "served.hidden" repo filter means everything is visible (no filtered revisions), so before this series branch2-served.hidden file would not contain any cache hash, only revnum and node. Now it also has a hash when there are obsolete changesets in the repo. The command that the message appears for is changing branch of 5 and 6, which are now obsolete, so the cache hash changes. In general, when cache is simply out-of-date, it can be updated using the old version as a base. But if cache hash differs, then the cache for that particular repo filter is recomputed (at least with the current implementation). This is what happens here. test-obsmarker-template.t: the pull reports 2 heads changed, but after that the repo correctly sees only 1. The new message could be better, but it's still an improvement over the previous one where hg pull suggested merging with an obsolete revision. test-obsolete.t: we can see these revisions in hg log --hidden, but they shouldn't be considered heads even with --hidden. test-rebase-obsolete{,2}.t: there were new heads created previously after making new orphan changesets, but they weren't detected. Now we are properly detecting and reporting them. test-rebase-obsolete4.t: there's only one head now because the other head is pruned and was falsely reported before. test-static-http.t: add obsstore to the list of requested files. This file doesn't exist on the remotes, but clients want it anyway (they get 404). This is fine, because there are other nonexistent files that clients request, like .hg/bookmarks or .hg/cache/tags2-served. Differential Revision: https://phab.mercurial-scm.org/D12097
author Anton Shestakov <av6@dwimlabs.net>
date Fri, 07 Jan 2022 11:53:23 +0300
parents 8e5effbf52d0
children 8b393f40a5e6
comparison
equal deleted inserted replaced
48700:4507bc001365 48701:f8f2ecdde4b5
15 nullrev, 15 nullrev,
16 ) 16 )
17 from . import ( 17 from . import (
18 encoding, 18 encoding,
19 error, 19 error,
20 obsolete,
20 pycompat, 21 pycompat,
21 scmutil, 22 scmutil,
22 util, 23 util,
23 ) 24 )
24 from .utils import ( 25 from .utils import (
182 <branch head hex node> <open/closed state> <branch name> 183 <branch head hex node> <open/closed state> <branch name>
183 ... 184 ...
184 185
185 The first line is used to check if the cache is still valid. If the 186 The first line is used to check if the cache is still valid. If the
186 branch cache is for a filtered repo view, an optional third hash is 187 branch cache is for a filtered repo view, an optional third hash is
187 included that hashes the hashes of all filtered revisions. 188 included that hashes the hashes of all filtered and obsolete revisions.
188 189
189 The open/closed state is represented by a single letter 'o' or 'c'. 190 The open/closed state is represented by a single letter 'o' or 'c'.
190 This field can be used to avoid changelog reads when determining if a 191 This field can be used to avoid changelog reads when determining if a
191 branch head closes a branch or not. 192 branch head closes a branch or not.
192 """ 193 """
355 356
356 - False when cached tipnode is unknown or if we detect a strip. 357 - False when cached tipnode is unknown or if we detect a strip.
357 - True when cache is up to date or a subset of current repo.""" 358 - True when cache is up to date or a subset of current repo."""
358 try: 359 try:
359 return (self.tipnode == repo.changelog.node(self.tiprev)) and ( 360 return (self.tipnode == repo.changelog.node(self.tiprev)) and (
360 self.filteredhash == scmutil.filteredhash(repo, self.tiprev) 361 self.filteredhash
362 == scmutil.filteredhash(repo, self.tiprev, needobsolete=True)
361 ) 363 )
362 except IndexError: 364 except IndexError:
363 return False 365 return False
364 366
365 def _branchtip(self, heads): 367 def _branchtip(self, heads):
475 topoheads = None 477 topoheads = None
476 478
477 # If a changeset is visible, its parents must be visible too, so 479 # If a changeset is visible, its parents must be visible too, so
478 # use the faster unfiltered parent accessor. 480 # use the faster unfiltered parent accessor.
479 parentrevs = repo.unfiltered().changelog.parentrevs 481 parentrevs = repo.unfiltered().changelog.parentrevs
482
483 # Faster than using ctx.obsolete()
484 obsrevs = obsolete.getrevs(repo, b'obsolete')
480 485
481 for branch, newheadrevs in pycompat.iteritems(newbranches): 486 for branch, newheadrevs in pycompat.iteritems(newbranches):
482 # For every branch, compute the new branchheads. 487 # For every branch, compute the new branchheads.
483 # A branchhead is a revision such that no descendant is on 488 # A branchhead is a revision such that no descendant is on
484 # the same branch. 489 # the same branch.
516 # This computation is heavy and avoided if at all possible. 521 # This computation is heavy and avoided if at all possible.
517 bheads = self._entries.setdefault(branch, []) 522 bheads = self._entries.setdefault(branch, [])
518 bheadset = {cl.rev(node) for node in bheads} 523 bheadset = {cl.rev(node) for node in bheads}
519 uncertain = set() 524 uncertain = set()
520 for newrev in sorted(newheadrevs): 525 for newrev in sorted(newheadrevs):
526 if newrev in obsrevs:
527 # We ignore obsolete changesets as they shouldn't be
528 # considered heads.
529 continue
530
521 if not bheadset: 531 if not bheadset:
522 bheadset.add(newrev) 532 bheadset.add(newrev)
523 continue 533 continue
524 534
525 parents = [p for p in parentrevs(newrev) if p != nullrev] 535 parents = [p for p in parentrevs(newrev) if p != nullrev]
526 samebranch = set() 536 samebranch = set()
527 otherbranch = set() 537 otherbranch = set()
538 obsparents = set()
528 for p in parents: 539 for p in parents:
529 if p in bheadset or getbranchinfo(p)[0] == branch: 540 if p in obsrevs:
541 # We ignored this obsolete changeset earlier, but now
542 # that it has non-ignored children, we need to make
543 # sure their ancestors are not considered heads. To
544 # achieve that, we will simply treat this obsolete
545 # changeset as a parent from other branch.
546 obsparents.add(p)
547 elif p in bheadset or getbranchinfo(p)[0] == branch:
530 samebranch.add(p) 548 samebranch.add(p)
531 else: 549 else:
532 otherbranch.add(p) 550 otherbranch.add(p)
533 if otherbranch and not (len(bheadset) == len(samebranch) == 1): 551 if not (len(bheadset) == len(samebranch) == 1):
534 uncertain.update(otherbranch) 552 uncertain.update(otherbranch)
553 uncertain.update(obsparents)
535 bheadset.difference_update(samebranch) 554 bheadset.difference_update(samebranch)
536 bheadset.add(newrev) 555 bheadset.add(newrev)
537 556
538 if uncertain: 557 if uncertain:
539 if topoheads is None: 558 if topoheads is None:
540 topoheads = set(cl.headrevs()) 559 topoheads = set(cl.headrevs())
541 if bheadset - topoheads: 560 if bheadset - topoheads:
542 floorrev = min(bheadset) 561 floorrev = min(bheadset)
543 ancestors = set(cl.ancestors(newheadrevs, floorrev)) 562 if floorrev <= max(uncertain):
544 bheadset -= ancestors 563 ancestors = set(cl.ancestors(uncertain, floorrev))
564 bheadset -= ancestors
545 bheadrevs = sorted(bheadset) 565 bheadrevs = sorted(bheadset)
546 self[branch] = [cl.node(rev) for rev in bheadrevs] 566 self[branch] = [cl.node(rev) for rev in bheadrevs]
547 tiprev = bheadrevs[-1] 567 tiprev = max(newheadrevs)
548 if tiprev > ntiprev: 568 if tiprev > ntiprev:
549 ntiprev = tiprev 569 ntiprev = tiprev
550 570
551 if ntiprev > self.tiprev: 571 if ntiprev > self.tiprev:
552 self.tiprev = ntiprev 572 self.tiprev = ntiprev
553 self.tipnode = cl.node(ntiprev) 573 self.tipnode = cl.node(ntiprev)
554 574
555 if not self.validfor(repo): 575 if not self.validfor(repo):
556 # cache key are not valid anymore 576 # old cache key is now invalid for the repo, but we've just updated
577 # the cache and we assume it's valid, so let's make the cache key
578 # valid as well by recomputing it from the cached data
557 self.tipnode = repo.nullid 579 self.tipnode = repo.nullid
558 self.tiprev = nullrev 580 self.tiprev = nullrev
559 for heads in self.iterheads(): 581 for heads in self.iterheads():
582 if not heads:
583 # all revisions on a branch are obsolete
584 continue
585 # note: tiprev is not necessarily the tip revision of repo,
586 # because the tip could be obsolete (i.e. not a head)
560 tiprev = max(cl.rev(node) for node in heads) 587 tiprev = max(cl.rev(node) for node in heads)
561 if tiprev > self.tiprev: 588 if tiprev > self.tiprev:
562 self.tipnode = cl.node(tiprev) 589 self.tipnode = cl.node(tiprev)
563 self.tiprev = tiprev 590 self.tiprev = tiprev
564 self.filteredhash = scmutil.filteredhash(repo, self.tiprev) 591 self.filteredhash = scmutil.filteredhash(
592 repo, self.tiprev, needobsolete=True
593 )
565 594
566 duration = util.timer() - starttime 595 duration = util.timer() - starttime
567 repo.ui.log( 596 repo.ui.log(
568 b'branchcache', 597 b'branchcache',
569 b'updated %s in %.4f seconds\n', 598 b'updated %s in %.4f seconds\n',