Mercurial > public > mercurial-scm > hg-stable

diff mercurial/util.py @ 17962:4c29668ca316 stable

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
util: make chunkbuffer non-quadratic on Windows The old str-based += collector performed very nicely on Linux, but turns out to be quadratically expensive on Windows, causing chunkbuffer to dominate in profiles. This list-based version has been measured to significantly improve performance with large chunks on Windows, with negligible overall overhead on Linux (though microbenchmarks show it to be about 50% slower). This may increase memory overhead where += didn't behave quadratically. If we want to gather up 1G of data to join, we temporarily have 1G in our list and 1G in our string.
author Matt Mackall <mpm@selenic.com>
date Mon, 26 Nov 2012 15:42:52 -0600
parents 9ee25d7b1aed
children 98c867ac1330
line wrap: on
line diff
--- a/mercurial/util.py	Mon Nov 26 13:44:11 2012 -0600
+++ b/mercurial/util.py	Mon Nov 26 15:42:52 2012 -0600
@@ -899,7 +899,7 @@
         """Read L bytes of data from the iterator of chunks of data.
         Returns less than L bytes if the iterator runs dry."""
         left = l
-        buf = ''
+        buf = []
         queue = self._queue
         while left > 0:
             # refill the queue
@@ -917,11 +917,11 @@
             left -= len(chunk)
             if left < 0:
                 queue.appendleft(chunk[left:])
-                buf += chunk[:left]
+                buf.append(chunk[:left])
             else:
-                buf += chunk
+                buf.append(chunk)
 
-        return buf
+        return ''.join(buf)
 
 def filechunkiter(f, size=65536, limit=None):
     """Create a generator that produces the data in the file size