facebook/sapling
1
1
Fork 0
mirror of https://github.com/facebook/sapling.git synced 2024-10-09 16:31:02 +03:00
Code Issues Projects Releases Wiki Activity
49c61f06be
sapling/mercurial
History
Gregory Szorc 49c61f06be scmutil: support background file closing 
Closing files that have been appended to is relatively slow on
Windows/NTFS. This makes several Mercurial operations slower on
Windows.

The workaround to this issue is conceptually simple: use multiple
threads for I/O. Unfortunately, Python doesn't scale well to multiple
threads because of the GIL. And, refactoring our code to use threads
everywhere would be a huge undertaking. So, we decide to tackle this
problem by starting small: establishing a thread pool for closing
files.

This patch establishes a mechanism for closing file handles on separate
threads. The coordinator object is basically a queue of file handles to
operate on and a thread pool consuming from the queue.

When files are opened through the VFS layer, the caller can specify
that delay closing is allowed.

A proxy class for file handles has been added. We must use a proxy
because it isn't possible to modify __class__ on built-in types. This
adds some overhead. But as future patches will show, this overhead
is cancelled out by the benefit of closing file handles on background
threads.
2016-01-14 13:34:59 -08:00
..
default.d mergetools.rc: find OSX FileMerge in the new location inside Xcode 4.3 2015-10-16 11:37:34 +02:00
help scmutil: support background file closing 2016-01-14 13:34:59 -08:00
hgweb hgweb: support rendering a sub-topic 2015-12-30 17:15:10 -07:00
httpclient httpclient: update to 938f2107d6e2 of httpplus 2015-12-31 13:19:20 -05:00
pure osutil: implement __enter__ and __exit__ on posixfile 2016-01-02 16:41:57 -08:00
templates hgweb: escape class keyword when used as a js object property (issue4913) 2015-10-27 12:34:05 +08:00
__init__.py mercurial: support loading modules from zipimporter 2015-12-03 21:25:05 -08:00
ancestor.py ancestor: use absolute_import 2015-08-07 19:45:48 -07:00
archival.py error: get Abort from 'error' instead of 'util' 2015-10-08 12:55:45 -07:00
base85.c base85: fix comparison of an int and a long 2015-08-21 14:29:42 -04:00
bdiff.c bdiff: avoid a memory error on malloc failure 2013-10-30 16:03:42 -05:00
bookmarks.py with: use context manager for wlock in _writeactive 2016-01-15 13:14:45 -08:00
branchmap.py error: get Abort from 'error' instead of 'util' 2015-10-08 12:55:45 -07:00
bundle2.py cleanup: use modern @property/@foo.setter property specification 2016-01-16 10:50:28 -05:00
bundlerepo.py bundlerepo: use context manager for file I/O in _writetempbundle 2016-01-12 14:48:27 -08:00
byterange.py byterange: use absolute_import 2015-12-21 21:42:14 -08:00
changegroup.py with: use context manager for transaction in changegroup apply 2016-01-15 13:14:50 -08:00
changelog.py reachableroots: construct and sort baseset in revset module 2015-08-28 11:14:24 +09:00
cmdutil.py with: use context manager in amend 2016-01-15 13:14:50 -08:00
commands.py backout: commit changeset by default (BC) 2016-01-15 13:46:33 -08:00
commandserver.py commandserver: reset state of progress bar per command 2015-12-14 23:50:02 +09:00
config.py config: add hasconfig method and supporting plumbing 2016-01-07 19:45:03 -08:00
context.py with: use context manager for wlock in checklookup 2016-01-15 13:14:46 -08:00
copies.py copies: fix detection of divergent directory renames 2016-01-13 10:10:05 -06:00
crecord.py crecord: stop raising error.Abort if curses is not found (issue5008) 2015-12-16 10:39:00 -08:00
dagparser.py error: get Abort from 'error' instead of 'util' 2015-10-08 12:55:45 -07:00
dagutil.py dagutil: use absolute_import 2015-08-08 19:04:09 -07:00
demandimport.py demandimport: fix level passed to loader of sub-modules 2015-11-01 21:19:09 +09:00
destutil.py destmerge: extract logic based on branch heads in its own function 2015-10-15 03:15:54 +01:00
diffhelpers.c diffhelpers: fix botched return statement from e48d70075d8a 2015-01-28 13:34:20 -05:00
dirs.c dirs.c: pass C string, not Python string, to _finddir() 2015-05-08 14:13:12 -07:00
dirstate.py dirstate: add a way to get the ignore file/line matching an ignored file 2016-01-05 07:52:04 -08:00
discovery.py discovery: pass pushop to _nowarnheads 2015-11-10 11:16:25 -08:00
dispatch.py dispatch: copy inferrepo attribute to alias commands 2016-01-04 21:40:05 +09:00
dummycert.pem ssl: on OS X, use a dummy cert to trick Python/OpenSSL to use system CA certs 2014-09-26 02:19:48 +02:00
encoding.py encoding: escape U+007F (DEL) character in JSON 2016-01-16 18:30:01 +09:00
error.py error: make InterventionRequired take a hint 2015-12-23 08:52:52 +00:00
exchange.py exchange: implement function for inferring bundle specification 2016-01-14 22:49:03 -08:00
exewrapper.c exewrapper: add comments about PYTHONHOME 2015-10-14 12:23:49 +02:00
extensions.py cleanup: remove superfluous space after space after equals (python) 2015-12-31 08:16:59 +00:00
fancyopts.py error: get Abort from 'error' instead of 'util' 2015-10-08 12:55:45 -07:00
filelog.py filelog: use absolute_import 2015-08-08 19:11:42 -07:00
filemerge.py origpath: move from cmdutil to scmutil 2016-01-02 03:02:57 -08:00
fileset.py fileset: add hint for list error to use or 2015-12-23 17:54:13 +00:00
formatter.py error: get Abort from 'error' instead of 'util' 2015-10-08 12:55:45 -07:00
graphmod.py graphmod: compute slow revset query once prior to reachableroots (issue4782) 2015-09-08 23:00:44 +09:00
hbisect.py with: use context manager in bisect save_state 2016-01-15 13:14:49 -08:00
help.py mercurial: pass ui to extensions.load (issue5007) 2015-12-17 10:30:17 +00:00
hg.py cleanup: remove superfluous space after space after equals (python) 2015-12-31 08:16:59 +00:00
hook.py dirstate: don't write repo.currenttransaction to repo.dirstate if repo 2015-12-03 01:38:21 +01:00
httpconnection.py httpconnection: use absolute_import 2015-12-21 21:52:58 -08:00
httppeer.py cleanup: remove superfluous space after space after equals (python) 2015-12-31 08:16:59 +00:00
i18n.py i18n: use absolute_import 2015-08-08 19:28:49 -07:00
keepalive.py keepalive: use print function 2015-06-21 23:14:54 -07:00
localrepo.py with: use context manager in localrepo recover 2016-01-15 13:14:50 -08:00
lock.py lock: turn a lock into a Python context manager 2016-01-15 13:14:45 -08:00
lsprof.py lsprof: use print function 2016-01-02 11:40:53 -08:00
lsprofcalltree.py lsprofcalltree: use print function 2016-01-02 11:45:29 -08:00
mail.py mail: use print function 2016-01-02 11:47:07 -08:00
manifest.c lazymanifest: check more return values in filtercopy 2015-12-31 13:31:42 -05:00
manifest.py manifest: use absolute_import 2015-12-21 21:35:46 -08:00
match.py spelling: trivial spell checking 2015-10-17 00:58:46 +02:00
mdiff.py mdiff: don't emit a diff header for empty trivial deltas 2016-01-11 22:00:07 -05:00
merge.py with: use context manager in merge update 2016-01-15 13:14:49 -08:00
minirst.py minirst: change hgrole to use single quotes 2016-01-12 06:03:36 +00:00
mpatch.c mpatch: rewrite pointer overflow checks 2013-12-11 18:33:42 -06:00
namespaces.py namespaces: use absolute_import 2015-08-08 19:42:58 -07:00
node.py node: add 'nullhex', hex-encoded nullid 2015-11-16 11:23:32 -08:00
obsolete.py obsolete: make _computeobsoleteset much faster 2016-01-13 21:52:26 -08:00
osutil.c mac: ignore resource fork when checking file sizes 2016-01-14 12:37:15 -06:00
parser.py parser: move unescape helper from templater 2015-09-10 23:25:10 +09:00
parsers.c cleanup: remove superfluous space after space after equals (C) 2015-12-31 08:17:15 +00:00
patch.py with: use context manager for I/O in changedfiles in patch 2016-01-15 13:14:50 -08:00
pathencode.c pathencode: remove an unused assignment 2015-12-12 20:59:37 -08:00
pathutil.py pathutil: use temporary variables instead of complicated wrapping 2015-12-03 13:14:20 -08:00
peer.py peer: use absolute_import 2015-08-08 19:45:45 -07:00
phases.py with: use context manager for transaction in pushphase 2016-01-15 13:14:49 -08:00
posix.py posix: fix test-permissions regression 2015-11-09 15:53:11 -06:00
progress.py spelling: trivial spell checking 2015-10-17 00:58:46 +02:00
pushkey.py pushkey: use absolute_import 2015-08-08 19:57:27 -07:00
pvec.py pvec: use absolute_import 2015-12-21 21:32:58 -08:00
py3kcompat.py py3compat: use absolute_import 2015-12-21 21:31:57 -08:00
registrar.py registrar: add delayregistrar class to register function in extensions 2015-12-29 23:58:30 +09:00
repair.py with: use context manager for transaction in strip 2016-01-15 13:14:49 -08:00
repoview.py repoview: bypass changelog method to computed cache key 2015-12-04 14:22:15 -08:00
revlog.py revlog: fix bad indentation (replace tab by space) 2015-12-18 20:54:41 -08:00
revset.py cleanup: remove superfluous space after space after equals (python) 2015-12-31 08:16:59 +00:00
scmposix.py scmposix: use absolute_import 2015-12-21 21:24:49 -08:00
scmutil.py scmutil: support background file closing 2016-01-14 13:34:59 -08:00
scmwindows.py windows: read all global config files, not just the first (issue4491) (BC) 2015-10-12 20:13:12 +02:00
setdiscovery.py error: get Abort from 'error' instead of 'util' 2015-10-08 12:55:45 -07:00
similar.py similar: use absolute_import 2015-12-12 23:17:22 -08:00
simplemerge.py simplemerge: move conflict warning message to filemerge 2015-10-09 13:54:52 -07:00
sshpeer.py sshpeer: make remotelock a context manager 2016-01-15 13:14:50 -08:00
sshserver.py error: get Abort from 'error' instead of 'util' 2015-10-08 12:55:45 -07:00
sslutil.py sslutil: fix reversed logic (issue5034) 2016-01-08 16:27:25 +01:00
statichttprepo.py statichttprepo: implement __enter__ and __exit__ on httprangeheader 2016-01-02 15:33:01 -08:00
store.py store: use absolute_import 2015-12-21 21:19:57 -08:00
streamclone.py streamclone: extract code for reading header fields 2016-01-14 22:48:54 -08:00
strutil.py strutil: use absolute_import 2015-08-08 20:01:40 -07:00
subrepo.py with: use context manager in subrepo storeclean 2016-01-15 13:14:50 -08:00
tagmerge.py tagmerge: use absolute_import 2015-08-08 20:10:46 -07:00
tags.py tags: create new sortdict for performance reasons 2015-11-12 13:16:04 -08:00
templatefilters.py merge with stable 2015-11-04 15:17:52 -06:00
templatekw.py templatekw: add {namespaces} keyword 2016-01-12 23:56:48 +09:00
templater.py templater: make get(dict, key) return a single value 2016-01-16 13:53:32 +09:00
transaction.py transaction: turn a transaction into a Python context manager 2016-01-15 13:14:47 -08:00
treediscovery.py error: get Abort from 'error' instead of 'util' 2015-10-08 12:55:45 -07:00
ui.py config: add hasconfig method and supporting plumbing 2016-01-07 19:45:03 -08:00
unionrepo.py merge with stable 2016-01-13 15:47:37 -06:00
url.py url: drop support for python2.5 2015-10-15 17:21:08 -04:00
util.h util: fix the check for non-C99 compilers (issue4605) 2015-04-20 22:21:57 -05:00
util.py util: rename ctxmanager's __call__ method to enter 2016-01-14 09:31:01 -08:00
verify.py with: use context manager in verify 2016-01-15 13:14:49 -08:00
win32.py win32: use absolute_import 2015-08-08 18:52:59 -07:00
windows.py windows: correct the import of win32 2015-12-16 13:33:43 -05:00
wireproto.py wireproto: support disabling bundle1 only if repo is generaldelta 2015-12-20 11:56:24 -08:00
worker.py error: get Abort from 'error' instead of 'util' 2015-10-08 12:55:45 -07:00