Mercurial > public > mercurial-scm > hg
comparison mercurial/revlog.py @ 27430:e240e914d226 stable
revlog: seek to end of file before writing (issue4943)
Revlogs were recently refactored to open file handles in "a+" and use a
persistent file handle for reading and writing. This drastically
reduced the number of file handles being opened.
Unfortunately, it appears that some versions of Solaris lose the file
offset when performing a write after the handle has been seeked.
The simplest workaround is to seek to EOF on files opened in a+ mode
before writing to them, which is what this patch does.
Ideally, this code would exist in the vfs layer. However, this would
require creating a proxy class for file objects in order to provide a
custom implementation of write(). This would add overhead. Since
revlogs are the only files we open in a+ mode, the one-off workaround
in revlog.py should be sufficient.
This patch appears to have little to no impact on performance on my
Linux machine.
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Thu, 17 Dec 2015 17:16:02 -0800 |
parents | 2f5c45fe3a3b |
children | 8f016345e6b0 e47841c8343d |
comparison
equal
deleted
inserted
replaced
27392:00aa37c65e0a | 27430:e240e914d226 |
---|---|
11 and O(changes) merge between branches. | 11 and O(changes) merge between branches. |
12 """ | 12 """ |
13 | 13 |
14 # import stuff from node for others to import from revlog | 14 # import stuff from node for others to import from revlog |
15 import collections | 15 import collections |
16 import os | |
16 from node import bin, hex, nullid, nullrev | 17 from node import bin, hex, nullid, nullrev |
17 from i18n import _ | 18 from i18n import _ |
18 import ancestor, mdiff, parsers, error, util, templatefilters | 19 import ancestor, mdiff, parsers, error, util, templatefilters |
19 import struct, zlib, errno | 20 import struct, zlib, errno |
20 | 21 |
1424 self._cache = (node, curr, text) | 1425 self._cache = (node, curr, text) |
1425 self._basecache = (curr, chainbase) | 1426 self._basecache = (curr, chainbase) |
1426 return node | 1427 return node |
1427 | 1428 |
1428 def _writeentry(self, transaction, ifh, dfh, entry, data, link, offset): | 1429 def _writeentry(self, transaction, ifh, dfh, entry, data, link, offset): |
1430 # Files opened in a+ mode have inconsistent behavior on various | |
1431 # platforms. Windows requires that a file positioning call be made | |
1432 # when the file handle transitions between reads and writes. See | |
1433 # 3686fa2b8eee and the mixedfilemodewrapper in windows.py. On other | |
1434 # platforms, Python or the platform itself can be buggy. Some versions | |
1435 # of Solaris have been observed to not append at the end of the file | |
1436 # if the file was seeked to before the end. See issue4943 for more. | |
1437 # | |
1438 # We work around this issue by inserting a seek() before writing. | |
1439 # Note: This is likely not necessary on Python 3. | |
1440 ifh.seek(0, os.SEEK_END) | |
1441 if dfh: | |
1442 dfh.seek(0, os.SEEK_END) | |
1443 | |
1429 curr = len(self) - 1 | 1444 curr = len(self) - 1 |
1430 if not self._inline: | 1445 if not self._inline: |
1431 transaction.add(self.datafile, offset) | 1446 transaction.add(self.datafile, offset) |
1432 transaction.add(self.indexfile, curr * len(entry)) | 1447 transaction.add(self.indexfile, curr * len(entry)) |
1433 if data[0]: | 1448 if data[0]: |