Commit Graph

81 Commits

Author SHA1 Message Date
Durham Goode
85e48b58fd Move server and debug logic into their own files
__init__.py was getting quite large. This change moves the server and debug
logic into their own files.  Client-side logic remains in __init__.py
2013-11-25 16:36:44 -08:00
Durham Goode
d9d4477013 Remove global variable for tracking shallow remotes
Previously we used a global variable to track if the incoming connection was
from a shallow remote (based on if the network command was a *_shallow command).
This is hacky and overall a bad idea. The new implementation stores the shallow
flag as a bundlecapability passed to the getbundle command.

A side effect of this is remotefilelog won't work with versions of mercurial
that don't use the getbundle command.
2013-11-25 14:22:56 -08:00
Durham Goode
b88d1b44d4 Replace linknode fallback algorithm
The previous algorithm thought that if the system cache had the file rev, it was
guaranteed to be valid. This isn't true in the case of a machine in which
multiple people share the cache (one person may have pulled a rev but the other
hasn't).

The new algorithm is more explicit. It checks:

- system cache
- local cache
- local cache fallbacks
- remote cache
- master server
2013-11-22 13:41:54 -08:00
Durham Goode
cbddd1460d Switch to dummyssh for test runs 2013-11-22 13:39:35 -08:00
Durham Goode
e5f5e3244b Add more comments explaining various complexities 2013-11-05 17:19:59 -08:00
Durham Goode
24ce0242d7 Add example cache client implementation
Adds a cache client implementation using the opensource python-memcached
library. It's more of an educational example than a production ready one since
it doesn't perform the requests asynchronously.  It does however split up large
files into smaller chunks for you.
2013-10-17 14:18:23 -07:00
Durham Goode
18baf608df Remove unused time and traceback imports 2013-10-16 13:40:25 -07:00
Durham Goode
f5b8ea4e79 Tweak README 2013-10-16 13:41:34 -07:00
Durham Goode
d122f76e5b Add readme and GPL info 2013-10-15 17:20:12 -07:00
Durham Goode
1275d15990 Add include and exclude configuration settings
The remotefilelog extension currently doesn't work with tags. Adding include and
exclude patterns allows users to specify which files they want to treat as
shallow and which the want to download the entire history for. By excluding
.hgtags from being shallow, this enables tags to work in a mostly shallow repo.

This also enables largefile like scenarios where most files are full and only a
few large ones are kept remote.
2013-09-26 10:46:06 -07:00
Durham Goode
5a628dc440 Fix linknode test failure 2013-10-09 10:20:47 -07:00
Durham Goode
3c6137f555 Fix revert prefetch causing excess output 2013-10-07 17:13:00 -07:00
Durham Goode
b47e016320 Replace linknode recovery tests with a real world test 2013-10-04 14:40:47 -07:00
Durham Goode
7268e5b709 Refactor ancestormap linknode logic to handle a bug
A rare bug can occur where the local file blob might not exist, but a valid old
version of that blob does exist. This refactor the linknode logic in ancestormap
to check the old versions if the server fetch fails to find the blob.

It still prints an ugly warning message from the server, but this whole issue is
quite rare anyway.
2013-10-03 15:15:15 -07:00
Durham Goode
ab72a92e85 GC server cache and add GC tests 2013-10-02 16:21:48 -07:00
Durham Goode
be29ee042a Fix reverting from non-root directories 2013-10-02 09:45:52 -07:00
Durham Goode
335e1d1bfc Prefetch before revert 2013-09-17 11:24:31 -07:00
Durham Goode
2ccd88dfcd Support new mercurial _basesupported 2013-10-01 15:11:57 -07:00
Durham Goode
efdfcc1502 Send all available data during a pull 2013-09-19 16:22:14 -07:00
Durham Goode
3667c253fd Refresh changelog during getfiles loop 2013-09-19 15:56:26 -07:00
Durham Goode
cf9d751d8a Add remotefilelog debug commands 2013-09-17 20:15:08 -07:00
Durham Goode
6a8a2f0e58 Fix rare issue with broken linknodes in the ancestormap 2013-09-16 18:46:24 -07:00
Durham Goode
f480c7deef Remove remotefilectx.__str__
Recent changes to Mercurial mean this is implemented by a base class.
2013-09-11 12:29:01 -07:00
Durham Goode
4ce55b8a0f Add log file warning 2013-09-11 10:27:56 -07:00
Durham Goode
6781d80d25 Fix local pulls to send file data 2013-09-09 11:44:08 -07:00
Durham Goode
6acb5968a1 Clean up empty cache files if we encounter them 2013-09-09 11:23:03 -07:00
Durham Goode
3619a1911d Cut down number of sys calls during filelog reads
When the cache is stored on a filesystem, excessive stat calls can slow
mercurial updates down dramatically. This reduces it to a single open call for
the cache location and if that fails, a single open call for the local location.
2013-09-09 10:23:29 -07:00
Durham Goode
c17ec690c9 Change cache key to use a two character prefix for directories.
Some file systems can't handle having a ton of files/directories inside a
directory, so this splits up all our files amongst directories.
2013-09-06 13:28:15 -07:00
Durham Goode
4d70ed4fce Fix a bug with status prefetching in merge scenarios 2013-09-04 19:07:01 -07:00
Durham Goode
1391bcc1c6 Add a test suite 2013-09-03 20:03:43 -07:00
Durham Goode
29ba0e9bc1 If cacheprocess is not set, always use the fallback
This allows tests to run without a memcache process.
2013-09-03 20:03:24 -07:00
Durham Goode
4a5c8d437d Fix hg diff when fnode is None 2013-09-03 11:39:16 -07:00
Durham Goode
5ec22c7093 Prevent verify from checking filelogs 2013-08-30 15:43:22 -07:00
Durham Goode
b685d98f57 Prefetch revisions before a diff 2013-08-30 11:27:09 -07:00
Durham Goode
4edeed8417 Prefetch lookup set during hg status 2013-08-30 11:09:19 -07:00
Durham Goode
3c879ed1a8 Enable efficient pulling between shallow repos 2013-08-28 18:51:01 -07:00
Durham Goode
d0738cc010 Make cache files owned by uid/svnuser 2013-08-20 12:59:33 -07:00
Durham Goode
96bbab8f7a Fix shared cache permissions to be g+w 2013-08-15 10:59:11 -07:00
Durham Goode
f68d704603 Enable hg gc from outside a repo 2013-08-15 10:56:25 -07:00
Durham Goode
bf7491936d Fix hg diff with added or moved files.
A workingctx produces manifest entries with nullid+'a' or nullid+'m'
for any added or modified files. The extension was trying to prefetch
these but they didn't exist and caused an error. Luckily they are length
42 so we can check for them and not prefetch them.
2013-07-24 22:16:50 -07:00
Durham Goode
a5828ce7a3 Add newline to end of debug output 2013-07-24 18:49:14 -07:00
Durham Goode
9df6e83354 Prevent 'running ssh...' in stdout when run with -v 2013-07-24 13:20:13 -07:00
Durham Goode
3cbc732b42 Fix fallbackrepo not being present during the clone after update.
Make debug message get sent to stderr instead of stdout.
2013-07-23 19:06:40 -07:00
Durham Goode
77d31b12e4 Add hit/miss ratio to debug output 2013-07-01 17:37:55 -07:00
Durham Goode
9642a8a2d6 Add remotefilelog.fallbackrepo config 2013-07-01 16:28:34 -07:00
Durham Goode
58ff8f91f6 Prefetch before copy tracing 2013-07-01 15:35:08 -07:00
Durham Goode
027a1d4ab8 Set umask before writing files to shared cache 2013-06-28 17:12:20 -07:00
Durham Goode
8e037436cb Add gc command for cleaning up the cache 2013-06-28 15:57:15 -07:00
Durham Goode
6e3494bf98 Add incoming hook for producing file blobs 2013-06-27 15:14:22 -07:00
Durham Goode
1ac9b8cbc1 Move requirement string to a variable 2013-06-26 14:37:59 -07:00