Summary:
When updating to the null commit, the logic that computes the update
distance was broken. The null commit is pre-resolved to -1, which when passed to
a revset raw gets resolved as the tip commit. In large repositories this can
take a long time and use a lot of memory, since it's computing the difference
between tip and null.
Let's fix it to not pass the raw rev number, and also to handle the case of a 0
distance update.
Reviewed By: quark-zju
Differential Revision: D23358402
fbshipit-source-id: 3b0a1fe1bbcb07effba4d0ab2c092e66bdc02e67
Summary:
This fix was originally meant to prevent doing checkunknown files when
it was clear there was no conflict. Unfortunatley, the data doesn't appear to
show it helped, and in some cases it definitely hurt it. Let's back it out for
now until we can do more investigation.
Reviewed By: StanislavGlebik
Differential Revision: D22574000
fbshipit-source-id: aeb644ecd6da046df17e6d10418a72363c1ee532
Summary:
checkunknown is quite expensive since it has to read the contents of
every untracked file, which can be 10's of thousands of non-parallel stats and
reads. For files that don't exist in the working copy, it's just wasted work to
stat for the files at all. Status can efficiently tell us what files are
unknown, so let's use that to triage most "unknown" files to normal writes
before we even get to checkunknown.
The downside of this approach is that it makes an additional call to status,
which is not cached (only non-unknown+non-ignore+non-clean status calls are
cached). We could add more caching if this is a problem.
This doesn't help the case where a user might have 10k+ untracked files due to a
ctrl+c'd checkout, but we'll improve that in a future diff.
Reviewed By: quark-zju
Differential Revision: D22366758
fbshipit-source-id: b54fec113dc162f97a35e705ed083ddd14babe55
Summary:
With the Rust stores being enabled by default in the code, time to actually
remove the code that wasn't using it. A lot of code still remains as the
treemanifest code still uses it. Migrating the treemanifest code to use the
Rust store would allow for even more code to be removed!
Reviewed By: DurhamG
Differential Revision: D21987918
fbshipit-source-id: 8e0becd4a1fb2ac81e26b8517d13e6d06ab06f65
Summary:
Now that the workers are in Rust, we no longer need the forker version in
Python. For now, the Python LFS extension still uses the threaded worker so
keep this one for now, when that extension will be removed we can remove the
rest of the worker code.
In theory, not all repository would benefit from the Rust workers, but these
are not supported at FB due to not being remotefilelog based.
Reviewed By: DurhamG
Differential Revision: D21987295
fbshipit-source-id: d17b9730651671608cf13f7abe6a9bb32251e140
Summary: Replaces usages of whitelist/blacklist with include/exclude/allowed. These terms are more descriptive and less likely to contribute to racial stereotyping. More context: https://fb.workplace.com/groups/sourcecontrolteam/permalink/2926049127516414/
Reviewed By: kulshrax
Differential Revision: D22039297
fbshipit-source-id: ab6e55889e6cf42aed35c856475b2ff92f4cb802
Summary: Reformat using a newer version of Black.
Reviewed By: quark-zju
Differential Revision: D21426337
fbshipit-source-id: 1ac7f6e85a06feec0d41e9509eca09194f421a1d
Summary: This will help us debug slow commands
Reviewed By: xavierd
Differential Revision: D21075895
fbshipit-source-id: 3e7667bb0e4426d743841d8fda00fa4a315f0120
Summary:
When a file goes from being a symlink to a regular file, a regular update
action ("g") is used, and the Python code implicitely remove the symlink before
writing to it. In the Rust code, we don't and as a consequence write through
the symlink, not the intended behavior.
An alternative way of fixing this would be to perform an lstat(2) before
writing to a file, but the cost of doing that will be fairly high for a very
unlikely situation especially since the manifest diff can give us exactly this
information.
This whole merge code feels extremely fragile, so I'm definitively not sure if
I got all the places that needs updating :(.
Reviewed By: DurhamG
Differential Revision: D21082733
fbshipit-source-id: 4f36a67363915c9b67d5a0b290a226075a9f1d31
Summary:
The Rust worker code will only work with remotefilelog, and when the Rust
ContentStore is enabled, make sure to not enable it if these 2 conditions
aren't met.
Reviewed By: quark-zju
Differential Revision: D21033428
fbshipit-source-id: c34c1b39ddb81be399463712216fa2cd68771f41
Summary:
While failures in the Rust updater aren't expected, at least one valid case
requires requires retrying the operation in Python: old-style LFS pointers.
When these are stored in packfiles/indexedlog, only the Python code knows how
to deal with them, and thus the operation needs to be retried there.
Reviewed By: DurhamG
Differential Revision: D20603709
fbshipit-source-id: 7d24ba573f0ff540906d909f1b4440fd4d3469a6
Summary: We don't really need the Rust workers for this, as we do not expect thousands of files to be changed during an in-memory merge.
Reviewed By: DurhamG
Differential Revision: D20495141
fbshipit-source-id: e72f8c4b01deee46ee72364dcd6716692c4103ab
Summary:
Symlinks are treated a bit differently from plain files, what is stored in the
ContentStore is the destination of the symlink, not it's content (well, the
content of a symlink really is it's destination).
For now, only unix platforms support symlinks, in reality this should be a
filesystem property as writing to ntfs-3g should have the same behavior as on
Windows.
For executable, we just need to mark the file as executable after writing to
it.
Reviewed By: quark-zju
Differential Revision: D20250943
fbshipit-source-id: 022dabc750125df32953a151df7da60db69b2cec
Summary:
During `hg update`, Mercurial forks multiple processes to write files on disk
concurrently, this is done as fetching blobs from the content store, and
writing them to disk is CPU bound. Usually, threads would be the preferred way
of speeding up such process, but unfortunately, Python has GIL that severely
limit the available concurrency. So, multiple processes were chosen.
Unfortunately, the multi-process solution also brings a lot of other issues,
more recently, we've had cases where the connections to the server and memcache
had to be dropped after the fork. In some other cases, this caused deadlocks.
And the solution is not effective on Windows.
Now that Mercurial is getting more and more Rust, we could instead go back to
the threads solution by using them in Rust, and have Python just push work to
them, this is exactly what this change does.
Things that are left to be done, but I wanted to get a diff out first:
- no file path audit
- no file backup
- no symlink creation
- probably other things I'm missing
Reviewed By: quark-zju
Differential Revision: D20102888
fbshipit-source-id: d47829fd7818b97710586b9851880f178048e27b
Summary:
This is mostly result of:
```
find . -type f -exec sed -i "s/fbconduit/fbscmquery/g" {} \;
```
`fbconduit` extension doesn't use conduit anymore so the name is just misleading.
Reviewed By: ikostia
Differential Revision: D18748843
fbshipit-source-id: 0d59e61ba7a96d86d9d1333d81255108cc3141bc
Summary: Log files that required manual merge during a rebase to dev command timers so we have an idea of how often they happen and on which files
Reviewed By: simpkins
Differential Revision: D18648175
fbshipit-source-id: 83ffe6d9177ca00b8fd1095745c585186bc2c8e9
Summary:
In preparation for merging fb-mercurial sources to the Eden repository,
move everything from the top-level directory into an `eden/scm`
subdirectory.
