Commit Graph

5 Commits

Author SHA1 Message Date
Mike Hommey
50e9c3bb84 bundle2: properly request phases during getbundle
getbundle was requesting the "phase" namespace instead of the "phases"
namespace, which led to the client still requesting the phases
separately after getbundle finished.
2016-05-05 20:57:38 +09:00
Mads Kiilerich
39acd325e0 streamclone: fix error when store files grow while stream cloning
Effectively a backout of d573a437d564, but updated to using 'with'.
2016-03-13 02:29:11 +01:00
Mads Kiilerich
6372bd0ae3 tests: add test of stream clone of repo that is changing
This reveals an error introduced by d573a437d564.
2016-03-13 02:28:46 +01:00
timeless
156f8d111d bdiff: (pure) support array.array arrays (issue5130) 2016-03-08 17:26:12 +00:00
Gregory Szorc
a05892eae0 streamclone: use backgroundfilecloser (issue4889)
Closing files that have been appended to is slow on Windows/NTFS.
CloseHandle() calls on this platform often take 1-10ms - and that's
on my i7-6700K Skylake processor with a modern and fast SSD. Contrast
with other I/O operations, such as writing data, which take <100us.

This means that creating/appending thousands of files can add
significant overhead. For example, cloning mozilla-central creates
~232,000 revlog files. Assuming 1ms per CloseHandle(), that yields
232s (3:52) of wall time waiting for file closes!

The impact of this overhead can be measured most directly when applying
stream clone bundles. Applying these files is effectively uncompressing
a tar archive (read: it's very fast).

Using a RAM disk (read: no I/O wait), the difference in wall time for a
`hg debugapplystreamclonebundle` for a ~1731 MB mozilla-central bundle
between Windows and Linux from the same machine is drastic:

Linux:    ~12.8s (128MB/s)
Windows: ~352.0s (4.7MB/s)

Windows is ~27.5x slower. Yikes!

After this patch:

Linux:    ~12.8s (128MB/s)
Windows: ~102.1s (16.1MB/s)

Windows is now ~3.4x faster. Unfortunately, it is still ~8x slower than
Linux. Profiling reveals a few hot code paths that could likely be
improved. But those are for other patches.

This patch introduces test-clone-uncompressed.t because existing tests
of `clone --uncompressed` are scattered about and adding a variation for
background thread closing to e.g. test-http.t doesn't feel correct.
2016-01-14 13:44:01 -08:00