comparison mercurial/revlog.py @ 30792:4215dc1b708b

revlog: make compressed size comparisons consistent revlog.compress() compares the compressed size to the input size and throws away the compressed data if it is larger than the input. This is the correct thing to do, as storing compressed data that is larger than the input takes up more storage space and makes reading slower. However, the comparison was implemented inconsistently. For the streaming compression mode, we threw away the result if it was greater than or equal to the input size. But for the one-shot compression, we threw away the compression only if it was greater than the input size! This patch changes the comparison for the simple case so it is consistent with the streaming case. As a few tests demonstrate, this adds 1 byte to some revlog entries. This is because of an added 'u' header on the chunk. It seems somewhat wrong to increase the revlog size here. However, IMO the cost of 1 byte in storage is insignificant compared to the performance gains of avoiding decompression. This patch should invite questions around the heuristic for throwing away compressed data. For example, I'd argue we should be more liberal about rejecting compressed data, additionally doing so where the number of bytes saved fails to reach a threshold. But we can have this discussion another time.
author Gregory Szorc <gregory.szorc@gmail.com>
date Mon, 02 Jan 2017 11:50:17 -0800
parents 1c7368d1a25f
children b6f455a6e4d6
comparison
equal deleted inserted replaced
30791:ada160a8cfd8 30792:4215dc1b708b
1501 p.append(z.flush()) 1501 p.append(z.flush())
1502 if sum(map(len, p)) < l: 1502 if sum(map(len, p)) < l:
1503 bin = "".join(p) 1503 bin = "".join(p)
1504 else: 1504 else:
1505 bin = _compress(text) 1505 bin = _compress(text)
1506 if bin is None or len(bin) > l: 1506 if bin is None or len(bin) >= l:
1507 if text[0] == '\0': 1507 if text[0] == '\0':
1508 return ("", text) 1508 return ("", text)
1509 return ('u', text) 1509 return ('u', text)
1510 return ("", bin) 1510 return ("", bin)
1511 1511