Summary:
Closing a repo is required to commit the pending transactions, or to run a
background repack when packfiles were downloaded. Usually, the close method is
called in dispatch after a command is run and a valid repository was passed via
-R. The debugedenimporthelper however may open the repository manually, and
since it doesn't close it, we may end up with a large number of packfiles.
Let's manually close it to avoid this issue.
Reviewed By: simpkins
Differential Revision: D14531514
fbshipit-source-id: 9b2d1648775e4a2c9eea4c232bf0ae4fee43297d
Summary:
Windows disallows rewriting or truncating mmaped files. Fix the tests by
either dropping the mmap, or skipping the test.
Reviewed By: sfilipco
Differential Revision: D14572119
fbshipit-source-id: dccafdc66db3830c2919232d899ba31365120066
Summary:
Windows has 2 kinds of paths - the UNC path (starting with `\\?\`), and paths
most people use (ex. `C:\foo\bar`). The former is more powerful (reserved names
like `nul` can be used), and is the "canonicalized" form as seen by Rust stdlib.
The UNC paths are stricter, though. `/` is not treated as `\` automatically,
`.` and `..` are considered illegal. That is, trying to canonicalize a UNC path
with `..` or `.` will result in an error.
It's possible to get `.`, or `..` into part of an UNC path, by using the
`PathBuf::join` API provided by the Rust stdlib. That is, a legal UNC path stored
in `PathBuf` can become illegal by `join`ing a non-UNC path.
I'm not sure what's the most "clean" fix. Perhaps using two different types to
represent UNC path and non-UNC path in stdlib? But that's definitely not a trivial
(or even practical) change.
For now, just teach the config parser to "friendly try again" by stripping the
UNC prefix and re-canonicalize paths. So it can handle `.` or `..` used by
`%include`.
Reviewed By: sfilipco
Differential Revision: D14568119
fbshipit-source-id: 2a55faa945c8d03574fd56e82d946c9ef7f0138f
Summary:
The `load_or_create_meta` function is subject to filesystem races. Solve it by
always taking a lock.
This hurts performance a little bit. But `open()` should not be in a hot loop.
So it should probably be fine.
Reviewed By: sfilipco
Differential Revision: D14568122
fbshipit-source-id: d9b28555ab94252da4717de709b780b361e1dda7
Summary:
On Windows it's impossible to open (2) a directory. Therefore add a utility
function that creates `lock` file automatically on Windows and open that file
instead.
Reviewed By: sfilipco
Differential Revision: D14568117
fbshipit-source-id: bc7ae7046be654560c38fbd98ec4dd58c071b1dc
Summary:
Previously, `load_or_create_meta` could return without actually creating the
meta file. That leads to problems when `load_or_create_meta` is called a
second time via `flush()`, it rewrites the primary file incorrectly. On Windows,
it will fail to rewrite the primary file.
Fix it by actually writing a meta file before returning.
Reviewed By: sfilipco
Differential Revision: D14568118
fbshipit-source-id: da3ad42bf48a923d732b1719839ca1953bd2b06c
Summary:
If the cloud sync or pushbackup commands start in background themselves, it will trigger a new background command to back up to the secondary afterwards.
Note that for cloud sync, for the secondary store we trigger pushbackup command, not cloud sync. If should be fine. We don't backup duplicates as we always check with the server first what is already backup. To call pushback looks simpler as it doesn't involve all the complexity of cloud sync, and basically we just need to backup to the secondary.
Reviewed By: markbt
Differential Revision: D14578770
fbshipit-source-id: f81a50fd76e64f2d77d8d7004b201fcfe8a090d2
Summary:
Change debugignore to test directories properly (which, only affects sparse
repos in practise) and stop printing out the source ignore rule location, as we
no longer have that information in the gitignore world.
Reviewed By: markbt
Differential Revision: D14547049
fbshipit-source-id: e7c307b1e1f4a8dcd30d00bec1f3e7a84a14b47b
Summary:
This particular situation happens in the wild when the amend doesn't rebase
because of conflicts and users work on their stack using `hg prev` and `hg next
--rebase`. In this case when there's non-obsolete child that's always the child
we want to choose.
We're verbose about what we're doing so it's not confusing to the users.
Reviewed By: quark-zju
Differential Revision: D14560584
fbshipit-source-id: a453c0301a5156eea0d19ceb40d9a64e80b7fca7
Summary: The next commit introduces a heurestic to resolve those.
Reviewed By: zertosh
Differential Revision: D14560585
fbshipit-source-id: b62b59e4b254e59e88cfc7859aa21feffc360b05
Summary: This diff adds serializable types representing batch requests for the Eden API. Just like the response types, these types must live the Mercurial's `types` crate so that they can be shared between the client and server.
Reviewed By: quark-zju
Differential Revision: D14573332
fbshipit-source-id: c31d718e6a97829ce1acfc25b8853dc3761323a7
Summary:
Move the logic of adding the common ancestor to make the DAG connected to the
smartlog revset. This makes it handy for power users to just use `log` and
`smartlog` revset to get interesting graphs. `sl` is now a very thin wrapper
around the `smartlog` revset function.
Reviewed By: DurhamG
Differential Revision: D14461520
fbshipit-source-id: 78e3991059c9da7ef4410e252a2b69b1e54918cb
Summary:
Wrap user-provided revs with `smartlog` revset function. This makes more sense
together with the next change.
The test change is because "parents" of drafts are selected.
Reviewed By: DurhamG
Differential Revision: D14461519
fbshipit-source-id: 2a48931680f0dc50b80b87cea827152c21cf4791
Summary:
With the last change, the benefit of ancestorcache is limited. Therefore just
remove it to reduce complexity. This also makes `smartlog` closer to `log`.
Reviewed By: DurhamG
Differential Revision: D14461523
fbshipit-source-id: eb108a09e12b07e5012f70aef0b2940b07d746fb
Summary:
Use the `ancestor` revset to replace the adhoc ancestor calcuation. This makes
the code much shorter.
It's in theory slightly different from the old logic. But there are no test changes.
The new code can no longer take advantage of ancestorcache. Fortunately, with
optimizations, it is pretty close to a fully warmed up ancestorcache. Of course,
it's much faster than a cold ancestorcache.
Before (ancestorcache disabled):
quark@devvm33994 ~/fbcode/scm/hg % ./hg sl -T '.' --time --pager=off --all --config smartlog.useancestorcache=0 >/dev/null
time: real 75.050 secs (user 52.540+0.000 sys 22.520+0.000)
Before (ancestorcache warmed up):
quark@devvm33994 ~/fbcode/scm/hg % ./hg sl -T '.' --time --pager=off --all --config smartlog.useancestorcache=1 >/dev/null
time: real 2.670 secs (user 2.550+0.000 sys 0.100+0.000)
After:
quark@devvm33994 ~/fbcode/scm/hg % ./hg sl -T '.' --time --pager=off --all --config smartlog.useancestorcache=0 >/dev/null
time: real 2.970 secs (user 2.760+0.000 sys 0.160+0.000)
There are 5110 commits in the above smartlog graph.
Reviewed By: DurhamG
Differential Revision: D14461524
fbshipit-source-id: 68bee3c4397be833e381c582c20a849b768b144d
Summary:
Previously, the default master is `.^` when `--rev` is passed. Change it to
null so we're not adding unexpected "master" if `--rev` is used.
Reviewed By: DurhamG, sfilipco
Differential Revision: D14516266
fbshipit-source-id: ce93f5e905d674c21cc07bb5a2957d0fad302722
Summary:
We encountered a bug where if a file descriptor was not flushed prior
to forking the process, then both processes would end up flushing the same data
to the file, resulting in duplicate data and a corrupt file.
The real fix is to stop using fork, but for now let's address the most visible
problem by flushing pack files before forking.
Reviewed By: kulshrax
Differential Revision: D14569005
fbshipit-source-id: e002abe72c8014cbe49ccffab6159f8372affdb0
Summary: The mock values for the `Node` type are intended to have hashes that consist of a repeated digit (e.g., `1111111111111111111111111111111111111111`). However, since the bytes were specified using a single hex digit instead of two, the hashes were actually like `0101010101010101010101010101010101010101`. This diff fixes the values so they look as expected.
Reviewed By: quark-zju
Differential Revision: D14557546
fbshipit-source-id: 23651d70b9715d2fb77db162f689b87d9d43e5a2
Summary:
Otherwise `hg.rust` might use a stale libchg.a and cannot pass the
client/server version check when installed.
Reviewed By: sfilipco
Differential Revision: D14531275
fbshipit-source-id: 6d8f04a75d91ad2c25ef2002ee3c3f9848094578
Summary: Deprecate due to complexity of the code.
Reviewed By: mitrandir77
Differential Revision: D14561405
fbshipit-source-id: 6184317f549c0ab84335b09c4b48efccdf31f7fc
Summary: Added new metrics to log loose files size and number during repack. We need it to understand how much better the pack files work in terms of disc and memory usage.
Reviewed By: markbt
Differential Revision: D14544811
fbshipit-source-id: 5a4d894bd5a3358c7e0f93ecc9db5e9f2c2f2372
Summary:
Basically we should check that the commits have been backed up.
If this is not true and the commits are local we can just back them up.
If they are not known by this repo, pull from the old one and then back them
up.
Reviewed By: markbt
Differential Revision: D14508239
fbshipit-source-id: 3fdd83335cb937b153510ec3c7510ecd1167d0ca
Summary:
BuildInfo symbols are generated for fbcode binaries, and scanned by the
internal procprint tool. The procprint tool also scans hg binaries, but cannot
extract any build information.
The procprint data is used by Push4Push to find out job owners running ancient
software.
This diff inserts BuildInfo symbols so Push4Push (and we) can use them to
figure out people running old versions of hg and send tasks to them.
Note rpmbuild strips symbols by default and is hard to customize. For now, I
just disable the debuginfo packages to preserve the symbols. If space usage or
runtime performance becomes an issue, we can try other workarounds (ex. provide
a shim of `%_find_debuginfo_dwz_opts` that skips `hg.rust`, or provide a shim
of `%__strip`, etc).
By default, the symbols only get added when `setup.py` is run by `rpmbuild`.
So `make local` won't rebuild `hg.rust` every time.
Reviewed By: singhsrb
Differential Revision: D14464056
fbshipit-source-id: ac6f9dedf28cee5fc8d65e144e3d7d1c544d1eda
Summary: Log data about round-trip count, and object count for files, trees, and SSH calls.
Differential Revision: D14515776
fbshipit-source-id: cce416fd7dccdad3c73a9f1751a04ddac0d2c507
Summary:
Make it possible to use `ui.metrics.gauge` to collect metrics for a single
command, via the sampling extension.
Differential Revision: D14515775
fbshipit-source-id: e8a53549b00c1bc7b6509a5990a51d955d767d7e
Summary: This makes sure `ui.log` always runs through the sampling code path.
Differential Revision: D14515774
fbshipit-source-id: 585cd14eaecda12a9c2dd6ed003f0a457d67daf1
Summary:
Before this patch, metrics was designed to send stats to a global counter. I'd
like to use it for stats local to the current command (ex. count of file
fetching, count of round-trips, etc).
Change the API so "entity=None" forbids stats from sending to a global counter.
Differential Revision: D14515779
fbshipit-source-id: b5b3b040d674c71f467153c308b56aa6f506eb0c
Summary:
Disable parsing `.hgignore` and related fileset `hgignore()` by default. They can
still be enabled via configuration. The plan is to completely remove them
later.
A replacement for `hgignore()` fileset was added as `gitignore()`.
The `hgignore()` fileset seems to be only used by zertosh in the past 3 months.
Reviewed By: singhsrb
Differential Revision: D14543232
fbshipit-source-id: f2385062a0e816331f693239f62448979876078a
Summary: This simplifies testing setup for all crates in CI.
Reviewed By: quark-zju
Differential Revision: D14543989
fbshipit-source-id: 83693fada6e64b7c21fd89a880d6452d811ea90d
Summary:
https://doc.rust-lang.org/rustdoc/documentation-tests.html#syntax-reference
Rust will treat an indentation of 4 or more spaces as a fenced code block and
attempt to run it as a docblock test
Reviewed By: singhsrb
Differential Revision: D14543987
fbshipit-source-id: 92f78e9e052befba0bd3eea80ac171f651f2fced
Summary:
The main issue is that cargo test fails preventing adding sandcastle
configuration that would run these tests on CI.
Reviewed By: singhsrb
Differential Revision: D14543988
fbshipit-source-id: c299148cce01316fad872b9cf8e15dea6633da48
Summary:
Part way through our hg-git repo's history we started adding the git
hash to the extras so it was easier to generate the map files. Let's make
git-updatemeta stop when it passes this boundary.
This is useful for speeding up internal infrastructure that needs to generate
the mapfile from scratch.
Reviewed By: kulshrax
Differential Revision: D14542354
fbshipit-source-id: 7c17fb1b1439f9b4c0c0acf8b5a85790d02a0861
Summary:
git-updatemeta performance starts to matter when we have to run
it from scratch in new containers. Let's optimize this a bit. In a large repo
this improves the speed by 30%.
Reviewed By: quark-zju
Differential Revision: D14542339
fbshipit-source-id: 34f06369543b8d4d22838fd4e3878c6bec9a597c
Summary:
Improve the performance of the revsets that calculate which commits to back up
by limiting them to consider only the non-obsolete commits that are also draft.
Reviewed By: quark-zju
Differential Revision: D14544883
fbshipit-source-id: db9ed4a7abd81956762f56140321242dbccf2df0
Summary:
Not every command requires a valid repo, but when one is used, it is always
properly closed. Hence, let's simply wrap the repo.close method instead of
wrapping around the runcommand function.
Reviewed By: kulshrax
Differential Revision: D14531515
fbshipit-source-id: bcdbe6530c94041c1185b18570846ba609b5f605
Summary:
Attempt to detect oscillation of commit cloud workspaces by comparing the new
state after the current sync with the state before the previous application of
cloud changes. If the version number is incremented by 1, the workspace is
brought back to the same state, and less than a minute has passed since the
last time that commit cloud sync ran, abort the current sync step.
This happens after the commits have been backed up, but before the new cloud
workspace is synced to the commit cloud service. This prevents further
oscillation whilst ensuring the user's commits are still backed up.
Reviewed By: quark-zju
Differential Revision: D14540355
fbshipit-source-id: 20e4b0333f5a7e34b512967a03099625f62ff9d5
Summary:
Change how infinitepush determines what to back up.
Commits to back up are all draft ancestors of non-obsolete commits, *plus* all
draft ancestors of bookmarked commits, which may be obsolete.
Previously, in a stack of the form:
```
| x B (obsolete, bookmarked)
| |
| o A
|/
o
```
neither A or B would be backed up, despite normally being visible to the user.
Reviewed By: liubov-dmitrieva
Differential Revision: D14540356
fbshipit-source-id: 0d6ad330c53c818b08f736a9af64704cf0be7cd5
Summary:
Change how commit cloud determines what to sync and what has been successfully
synced when some commits fail to push.
Commits to sync are all draft ancestors of non-obsolete commits, *plus* all
draft ancestors of bookmarked commits.
Commits to sync when only some commits have successfully been pushed are
ancestors of the newly pushed heads, *plus* ancestors of the commits to be
synced that were successfully synced last time.
Reviewed By: liubov-dmitrieva
Differential Revision: D14540357
fbshipit-source-id: c082a2f2822f8bce4cd2bbac93a70e27e2ffaa59
Summary:
If a bookmarked commit is shared between two machines, but was hidden in one
repo prior to commit cloud being enabled, then commit cloud sync will
repeatedly try to upload the commit on the machine where it is not hidden.
Reviewed By: liubov-dmitrieva
Differential Revision: D14520200
fbshipit-source-id: e2674387e34a5cbf87ef7286a5b8099020ea1f9f
Summary:
Now at the beginning of pushbackup we always check what the server already
have, this triggered changes in the output of the tests.
Before the limit of the check was 3 heads. I made it to 0 by default earlier and broke tests. Now we have fast way to check on the server side, so the limit 3 was not good. There is no reason to re-back up those.
Basically before we could backup the same thing again.
Reviewed By: markbt
Differential Revision: D14539582
fbshipit-source-id: 569b354580128c944a95fa64c0c964304a2cca8b
Summary:
Similarly to the previous change let's make the asyncmutabledatapack force the
Metadata to be present.
Reviewed By: sfilipco
Differential Revision: D14443510
fbshipit-source-id: 26f851e8d38297dcc37410f0df6a69083531d516
Summary:
Now that mutablepacks can only create v1 packfile, we can force the Metadata to
not be optional. The main reason for doing this is to avoid issues where LFS
data is stored without its corresponding LFS flag. This can cause issue down
the line as LFS data will be intepreted as is, instead of being interpreted as
a pointer to the LFS blob.
Reviewed By: sfilipco
Differential Revision: D14443509
fbshipit-source-id: 9e7812017fc1356072278496406648f935024f92
Summary:
The v0 doesn't support flags like whether the data is actually an LFS pointer. Let's simply
forbid creating them.
Reviewed By: quark-zju
Differential Revision: D14443512
fbshipit-source-id: 6ffa2e8fda2b2baba0aae53e749bc9248594a134
Summary:
These last 2 errors are still considered fatal, but shouldn't be and are most
likely transient. Failing to open a packfile that was successfully opened
before can for instance happen when the file is removed by another process, or
if it somehow become corrupted. Failing the removal of the pack-file should no
longer be an issue, but if it fails, we can also ignore it with the reasoning
that the next repack will take care of it.
Reviewed By: sfilipco
Differential Revision: D14441288
fbshipit-source-id: 6c2758c2a88fd5d2d83b55defe3d263ee9f974a1
