comparison mercurial/revlog.py @ 27430:e240e914d226 stable

revlog: seek to end of file before writing (issue4943) Revlogs were recently refactored to open file handles in "a+" and use a persistent file handle for reading and writing. This drastically reduced the number of file handles being opened. Unfortunately, it appears that some versions of Solaris lose the file offset when performing a write after the handle has been seeked. The simplest workaround is to seek to EOF on files opened in a+ mode before writing to them, which is what this patch does. Ideally, this code would exist in the vfs layer. However, this would require creating a proxy class for file objects in order to provide a custom implementation of write(). This would add overhead. Since revlogs are the only files we open in a+ mode, the one-off workaround in revlog.py should be sufficient. This patch appears to have little to no impact on performance on my Linux machine.
author Gregory Szorc <gregory.szorc@gmail.com>
date Thu, 17 Dec 2015 17:16:02 -0800
parents 2f5c45fe3a3b
children 8f016345e6b0 e47841c8343d
comparison
equal deleted inserted replaced
27392:00aa37c65e0a 27430:e240e914d226
11 and O(changes) merge between branches. 11 and O(changes) merge between branches.
12 """ 12 """
13 13
14 # import stuff from node for others to import from revlog 14 # import stuff from node for others to import from revlog
15 import collections 15 import collections
16 import os
16 from node import bin, hex, nullid, nullrev 17 from node import bin, hex, nullid, nullrev
17 from i18n import _ 18 from i18n import _
18 import ancestor, mdiff, parsers, error, util, templatefilters 19 import ancestor, mdiff, parsers, error, util, templatefilters
19 import struct, zlib, errno 20 import struct, zlib, errno
20 21
1424 self._cache = (node, curr, text) 1425 self._cache = (node, curr, text)
1425 self._basecache = (curr, chainbase) 1426 self._basecache = (curr, chainbase)
1426 return node 1427 return node
1427 1428
1428 def _writeentry(self, transaction, ifh, dfh, entry, data, link, offset): 1429 def _writeentry(self, transaction, ifh, dfh, entry, data, link, offset):
1430 # Files opened in a+ mode have inconsistent behavior on various
1431 # platforms. Windows requires that a file positioning call be made
1432 # when the file handle transitions between reads and writes. See
1433 # 3686fa2b8eee and the mixedfilemodewrapper in windows.py. On other
1434 # platforms, Python or the platform itself can be buggy. Some versions
1435 # of Solaris have been observed to not append at the end of the file
1436 # if the file was seeked to before the end. See issue4943 for more.
1437 #
1438 # We work around this issue by inserting a seek() before writing.
1439 # Note: This is likely not necessary on Python 3.
1440 ifh.seek(0, os.SEEK_END)
1441 if dfh:
1442 dfh.seek(0, os.SEEK_END)
1443
1429 curr = len(self) - 1 1444 curr = len(self) - 1
1430 if not self._inline: 1445 if not self._inline:
1431 transaction.add(self.datafile, offset) 1446 transaction.add(self.datafile, offset)
1432 transaction.add(self.indexfile, curr * len(entry)) 1447 transaction.add(self.indexfile, curr * len(entry))
1433 if data[0]: 1448 if data[0]: