comparison mercurial/transaction.py @ 32558:aa91085cadf3

transaction: delete callbacks after use Before this change, localrepository instances that performed multiple transactions would leak transaction objects. This could occur when running `hg convert`. When running `hg convert`, the leak would be ~90 MB per 10,000 changesets as measured with the Mercurial repo itself. The leak I tracked down involved the "validate" closure from localrepository.transaction(). It appeared to be keeping a reference to the original transaction via __closure__. __del__ semantics and a circular reference involving the repo object may have also come into play. Attempting to refactor the "validate" closure proved to be difficult because the "tr" reference in that closure may reference an object that isn't created until transaction.__init__ is called. And the "validate" closure is passed as an argument to transaction.__init__. Plus there is a giant warning comment in "validate" about how hacky it is. I did not want to venture into the dragon den. Anyway, we've had problems with transactions causing leaks before. The solution then (14e683d6b273) is the same as the solution in this patch: drop references to callbacks after they are called. This not only breaks cycles in core Mercurial but can help break cycles in extensions that accidentally introduce them. While I only tracked down a leak due to self.validator, since this is the 2nd time I've tracked down leaks due to transaction callbacks I figure enough is enough and we should prevent the class of leak from occurring regardless of the variable. That's why all callback variables are now nuked.
author Gregory Szorc <gregory.szorc@gmail.com>
date Fri, 26 May 2017 13:27:21 -0700
parents bd872f64a8ba
children 2312e70cf78b
comparison
equal deleted inserted replaced
32557:3fdcc34c0aba 32558:aa91085cadf3
429 @active 429 @active
430 def close(self): 430 def close(self):
431 '''commit the transaction''' 431 '''commit the transaction'''
432 if self.count == 1: 432 if self.count == 1:
433 self.validator(self) # will raise exception if needed 433 self.validator(self) # will raise exception if needed
434 self.validator = None # Help prevent cycles.
434 self._generatefiles(group=gengroupprefinalize) 435 self._generatefiles(group=gengroupprefinalize)
435 categories = sorted(self._finalizecallback) 436 categories = sorted(self._finalizecallback)
436 for cat in categories: 437 for cat in categories:
437 self._finalizecallback[cat](self) 438 self._finalizecallback[cat](self)
438 # Prevent double usage and help clear cycles. 439 # Prevent double usage and help clear cycles.
462 % (vfs.join(b), inst)) 463 % (vfs.join(b), inst))
463 self.entries = [] 464 self.entries = []
464 self._writeundo() 465 self._writeundo()
465 if self.after: 466 if self.after:
466 self.after() 467 self.after()
468 self.after = None # Help prevent cycles.
467 if self.opener.isfile(self._backupjournal): 469 if self.opener.isfile(self._backupjournal):
468 self.opener.unlink(self._backupjournal) 470 self.opener.unlink(self._backupjournal)
469 if self.opener.isfile(self.journal): 471 if self.opener.isfile(self.journal):
470 self.opener.unlink(self.journal) 472 self.opener.unlink(self.journal)
471 for l, _f, b, c in self._backupentries: 473 for l, _f, b, c in self._backupentries:
485 % (vfs.join(b), inst)) 487 % (vfs.join(b), inst))
486 self._backupentries = [] 488 self._backupentries = []
487 self.journal = None 489 self.journal = None
488 490
489 self.releasefn(self, True) # notify success of closing transaction 491 self.releasefn(self, True) # notify success of closing transaction
492 self.releasefn = None # Help prevent cycles.
490 493
491 # run post close action 494 # run post close action
492 categories = sorted(self._postclosecallback) 495 categories = sorted(self._postclosecallback)
493 for cat in categories: 496 for cat in categories:
494 self._postclosecallback[cat](self) 497 self._postclosecallback[cat](self)
555 except BaseException: 558 except BaseException:
556 self.report(_("rollback failed - please run hg recover\n")) 559 self.report(_("rollback failed - please run hg recover\n"))
557 finally: 560 finally:
558 self.journal = None 561 self.journal = None
559 self.releasefn(self, False) # notify failure of transaction 562 self.releasefn(self, False) # notify failure of transaction
563 self.releasefn = None # Help prevent cycles.
560 564
561 def rollback(opener, vfsmap, file, report): 565 def rollback(opener, vfsmap, file, report):
562 """Rolls back the transaction contained in the given file 566 """Rolls back the transaction contained in the given file
563 567
564 Reads the entries in the specified file, and the corresponding 568 Reads the entries in the specified file, and the corresponding