Summary:
We've seen corrupted repository in the wild on which we were unable to garbage
collect. Let's add a test to verify that the other repositories will be garbage
collected.
Reviewed By: DurhamG
Differential Revision: D13328487
fbshipit-source-id: 6f6a69d14455d00e468c5b9186d65cca8fd5c1c3
Summary:
Match patterns don't work the way I expected. They require the path to
not have '/' at the end, and they don't seem to match when a file or sub
directory is appended. Fix it by setting default to 'path'.
```
> matchmod.match("", "/", patterns=['foo/'])('foo/bar')
False
> matchmod.match("", "/", default='path', patterns=['foo/'])('foo/bar')
True
```
Reviewed By: quark-zju
Differential Revision: D13332653
fbshipit-source-id: e0f3fa9a51d36a40ac8a9c54f73296f431536d3c
Summary: Fix test failures (`test-check-code.t` and `test-check-config.t`) introduced by my Mononoke API diff stack. I thought Sandcastle would have surfaced these, but I guess not.
Reviewed By: quark-zju
Differential Revision: D13325133
fbshipit-source-id: fd14f4c7e7280155d6f677f18b20691ef54ceca3
Summary:
When garbage collecting lots of repository, a single corrupted
repository will stop the garbage collection for all the other repositories. In
the worst case, the first repository examined was bad, and thus no collection
will be done for the other repositories.
Let's log the faulty repository and continue with the other repositories.
Reviewed By: DurhamG
Differential Revision: D13316936
fbshipit-source-id: 9760710ff64d0173ae1c369991f438bbace11cf4
Summary: Allow MononokeClient to support both HTTP and HTTPS. The protocol use is determined by the scheme of the server base URI passed in. For example, specifying `https://mononoke-api.internal.tfbnw.net` would use HTTPS, whereas specifying `http://localhost:12345` would use HTTP. This is useful for local testing.
Reviewed By: DurhamG
Differential Revision: D13089197
fbshipit-source-id: 2da72ac98c60746200334e4bcc0e2568abe3073b
Summary: This diff adds the `hg debugmononokeapi` command to remotefilelog, demonstrating that the Mononoke API server can be accessed from within the remotefilelog extension. This paves the way for fetching data from the API server from remotefilelog.
Reviewed By: DurhamG
Differential Revision: D13055686
fbshipit-source-id: c826f4d79389f7364b45483b3d83afd0caf65063
Summary:
This diff adds Python FFI bindings for the `mononokeapi` crate introduced in the previous diff in this stack. It exposes a `PyMononokeClient` Python class which wraps the Rust `MononokeClient` class internally.
This class currently only has one method, `health_check()`, which hits the API server's health check endpoint, returning `None` on success or raising an exception otherwise.
Reviewed By: DurhamG
Differential Revision: D13055688
fbshipit-source-id: a3703617c919f5317c90029fb40d275f3d63d56f
Summary:
This diff adds a new `mononokeapi` crate, which is a Rust client library for the Mononoke API server. The crate is intended for use beyond Mercurial, and as such attempts to expose functionality in a reasonably generic way.
Right now, the only method supported by this crate is `/health_check`, which is the API server's health check endpoint that simply returns the string "I_AM_ALIVE" on success. Future diffs will expand this crate to include more of the API server's actual functionality. For now, this version serves as a proof of concept of how all the crate will be structured.
The crate currently uses the `hyper` crate for its HTTP client, with `native-tls` for TLS support. Given that the client credentials required for mutual authentication with the Mononoke VIP are encoded in a format that `native-tls` does not understand, some credential format conversion via the `openssl` crate is necessary.
Reviewed By: DurhamG
Differential Revision: D13055687
fbshipit-source-id: cc944abd579ce49928776646c0dcce567f99c3b6
Summary:
Let's stop depending on call-conduit here. This will make it work on Windows
and remove the Mercurial's dependency on conduit.
Technically we're replacing subprocess call to `arc call-conduit` with
a thread doing a pure graphql call (as we already have graphql client
implemented as hg extension).
Reviewed By: markbt
Differential Revision: D13280279
fbshipit-source-id: 56ee9027826f2dd2f71cbe0faa28cd2310660aaf
Summary:
In a future diff we want pushrebase to be able to rewrite commits
pulled from svn. In that situation, we need to be able to rewrite the svn
metadata so that the post-pushrebase commits are mapped to the svn rev numbers.
Let's add a truncatemeta command that will walk the latest commits in the
changelog and redo any that aren't mapped correctly.
Reviewed By: markbt
Differential Revision: D13074960
fbshipit-source-id: fe98879dd16cc0806d20ef6eab5f9d77f0e0c877
Summary:
D13267528 malformed the bookmark store files for testing by deleting a
random file. This makes the test flaky since sometimes a non critical file is
deleted and the exception is not raised. Therefore, lets malform the store more
reliably by truncating all the files in the store.
Reviewed By: ikostia
Differential Revision: D13271362
fbshipit-source-id: 684ffbcdc59e943476a6acc2c4152bf65cc12e1f
Summary:
Importing anything from python files with explicit breaks embedded
aproach we use on Windows.
For one of the tests, I did add `hg debugshell -c "from hgext import sparse;
print sparse.__file__` just to make sure the same file is used in practice.
Reviewed By: singhsrb
Differential Revision: D13258984
fbshipit-source-id: 4c86e1dccfd4f525a14019dc4b9586d798f2bffe
Summary:
D13133605 changed the python bookmark store API and broke the tests.
To fix, I have migrated the tests to use the new API. Also, since most of the
tests was just python code, I have moved the test from being a `.t` test to a
`.py` test.
Reviewed By: quark-zju
Differential Revision: D13267528
fbshipit-source-id: 69b5a7110f63ea60e1bac402064edbffbf64420f
Summary:
Make repacks flush the newly created pack files to disk before proceeding to
delete the old datapacks. This ensures the data is committed to disk before we
delete the old pack files. There had been user complaints of data corruption.
The changes affects repack only to avoid slowing down performance for other
operations with manipulates datapack and historypack.
Reviewed By: DurhamG
Differential Revision: D12920147
fbshipit-source-id: 907b64d7763a6212fb49487cfc3bc403f8e3dce2
Summary:
Turn BookmarkStore rust implementation into indexed-log backed.
Note that this no longer matches existing mercurial bookmark store
disk representation.
Reviewed By: DurhamG
Differential Revision: D13133605
fbshipit-source-id: 2e0a27738bcec607892b0edab6f759116929c8e1
Summary:
Before I implement a proper fix [1], let's just use the correct slashes.
[1]
Correct fix is de-verbatimization of canonicalized paths.
So, if `a-symlink->b` and `"C:\a".canonicalize()` produces `\\?\C:\b`, then doing `.push("c/d")` produces `\\?\C:\b\c/d`, where `c/d` is a *single* path component, becuase the path starts with `\\?\`. If there isn't such prefix, it's fine to push forward-slash-separated things into Windows paths.
Differential Revision: D13234288
fbshipit-source-id: 2ca0326bbd91ddc6ffd259153915037264292dc1
Summary:
If full is specified, or the maxage config option changes from one sync to
another, we need to actually pull in the new commits. Since the version number
won't necessarily have changed, we need to request a full copy of the cloud
workspace to work from.
Reviewed By: liubov-dmitrieva
Differential Revision: D13214274
fbshipit-source-id: b21594c04c05f065caf9f9dc494e6274debbde5c
Summary:
Stop comparing the local heads and bookmarks with the full cloud heads and
bookmarks when some heads or bookmarks have been omitted. Since they're
omitted, they won't be there.
Reviewed By: liubov-dmitrieva
Differential Revision: D13214275
fbshipit-source-id: 35a897f053f58d0793d384ff60b8202e80aec0c7
Summary: Sometimes, the script could provide useful context to the user, so allow its stderr to be printed.
Reviewed By: DurhamG
Differential Revision: D13202155
fbshipit-source-id: cb4c5c4ff0c696fa2959f0c038d391bf701949e1
Summary: I only tested the original diff of Windows, aparently.
Reviewed By: mitrandir77
Differential Revision: D13188952
fbshipit-source-id: 9dc33cb0eedb8d3c09cb7a734528f71afd7cbe8a
Summary:
We need to canonicalize `current_exe` to resolve symlinks on OSX.
Unfortunetely, on there's no way to just resolve symlinks and not
turn path into a `\\?\` on Windows, AFAIK.
Once the path is canonicalized on Windows, it starts with `\\?\` and
forward slashes are no longer recognized as valid separators.
Here's a demonstration:
```
> cat src\main.rs
use std::path::Path;
fn main() {
let p = Path::new("\\\\?\\C:\\Code\\fbsource\\fbcode\\scm\\hg\\mercurial/entrypoint.py");
for comp in p.components() { println!("{:?}", comp); }
println!("{:?} exists: {}", p, p.exists());
let p = Path::new("\\\\?\\C:\\Code\\fbsource\\fbcode\\scm\\hg\\mercurial\\entrypoint.py");
for comp in p.components() { println!("{:?}", comp); }
println!("{:?} exists: {}", p, p.exists());
let p = Path::new("C:\\Code\\fbsource\\fbcode\\scm\\hg\\mercurial/entrypoint.py");
for comp in p.components() { println!("{:?}", comp); }
println!("{:?} exists: {}", p, p.exists());
}
> cargo run
Prefix(PrefixComponent { raw: "\\\\?\\C:", parsed: VerbatimDisk(67) })
RootDir
Normal("Code")
Normal("fbsource")
Normal("fbcode")
Normal("scm")
Normal("hg")
Normal("mercurial/entrypoint.py")
"\\\\?\\C:\\Code\\fbsource\\fbcode\\scm\\hg\\mercurial/entrypoint.py" exists: false
Prefix(PrefixComponent { raw: "\\\\?\\C:", parsed: VerbatimDisk(67) })
RootDir
Normal("Code")
Normal("fbsource")
Normal("fbcode")
Normal("scm")
Normal("hg")
Normal("mercurial")
Normal("entrypoint.py")
"\\\\?\\C:\\Code\\fbsource\\fbcode\\scm\\hg\\mercurial\\entrypoint.py" exists: true
Prefix(PrefixComponent { raw: "C:", parsed: Disk(67) })
RootDir
Normal("Code")
Normal("fbsource")
Normal("fbcode")
Normal("scm")
Normal("hg")
Normal("mercurial")
Normal("entrypoint.py")
"C:\\Code\\fbsource\\fbcode\\scm\\hg\\mercurial/entrypoint.py" exists: true
```
Differential Revision: D13176266
fbshipit-source-id: 5f35a3263e058d179b237c80f28e4fdf44105576
Summary: Since we are using `hg.rust` with everything, we should also test it.
Differential Revision: D13121588
fbshipit-source-id: 3a4e07c1d70c279f008812d026bf9a7c74aa3b2e
Summary: in case of VIP TLS errors comes as socket errors.
Reviewed By: DurhamG
Differential Revision: D13159635
fbshipit-source-id: 80204718ea117e3694199a8686a151ff8aae2634
Summary:
This had a nasty bug where it manually computed the cached base text
using the full manifest text. When operating in a treeonly repository, this
resulted in corrupt deltas. The only reason we didn't hit it earlier is because
the secondary stackpush path was added and doesn't have this issue. But it
couldn't handle merge commits, which is why we hit this when it came time to add
a merge commit.
Now that the main repo that needed this optimization is treeonly, we don't need
this precaching anymore.
Reviewed By: mitrandir77
Differential Revision: D13154342
fbshipit-source-id: 5cb74167665a60a36e4a6d926f3f9f1c5c7bbef1
Summary:
There's a corruption that occurs because pushrebase caches a fulltext
instead of a tree. Let's write a test to expose it. The next diff will fix it.
Reviewed By: mitrandir77
Differential Revision: D13154341
fbshipit-source-id: e2ac040c59c677b414b6eddd39d7dc2ebb25f082
Summary:
Fix tests.
This makes sure that the grep process itself is not matched.
Reviewed By: farnz
Differential Revision: D13138516
fbshipit-source-id: d29829584c6fe34dd60345b2812682d4cb9d51e4
Summary:
Mercurial extensions are expected to be side-effect free at import time.
Move setup code to `uisetup` so they only get executed if the extension is enabled.
This helps chg's case where `lz4revlog` is always pre-imported.
Reviewed By: quark-zju
Differential Revision: D13118788
fbshipit-source-id: a7801057c01b6918bb4902a326b92f1c17f2707a
Summary:
When we have omitted syncing commits from the cloud workspace, give the user
advice on what to do to fetch them if they run `hg sl --all`.
Reviewed By: markbt
Differential Revision: D13085505
fbshipit-source-id: 624ce6cbf4cb2194ab5ffbc09c1ac3e073932249
Summary:
This is important on OSX where `current_exe` will return the symlink address if `hg.rust` is a symlink.
Therefore, if you create a symlink to the `hg.rust` in the repo (like tests do), repo Python code won't be picked up, and the system code will be.
Reviewed By: mitrandir77
Differential Revision: D13138333
fbshipit-source-id: ffdf27329609d77bee4b8a2eecc47e02cb2dd5c8
Summary:
During the sequence of events that led up to S169085 we tried an hg
push that resulted in an error that looked like this: "RuntimeError:
std::exception". I believe that error was caused by an uncaught
pyexception, replacing Python's current thread exception with a
useless one.
Catch the pyexception and deal with it appropriately.
Reviewed By: quark-zju
Differential Revision: D13112735
fbshipit-source-id: 301d899543ae95084b890c19b00322e69ded07b2
Summary: Now that `chg` is embedded into `hg.rust`, we need to explicitly disable it while tests run.
Reviewed By: quark-zju
Differential Revision: D13021560
fbshipit-source-id: 843045f3bc9d9866e421f0bf0aa132849e5b338f
Summary:
When `chg` decides, that the command is unsupported, it executes the real hg
instead. Now that the real `hg` and the `chg` can be the same binary, we need
to make sure that `chg` is definitely not re-executed. Passing `CHGDISABLE`
achieves that.
Reviewed By: markbt
Differential Revision: D13118766
fbshipit-source-id: e41a2c230d17c4c26e590d648e4d0b722cf41ee5
Summary:
In backuplockcheck we try to take the lock to see if it's currently in use. If
it isn't, then we end up taking the lock, but don't release it. We should
release it.
Reviewed By: liubov-dmitrieva
Differential Revision: D13085506
fbshipit-source-id: 4b38ec11385d754d07ad2d15ff14b43d37325263
Summary:
hgsubversion has some logic that tries to read the list of files in a
given directory. In a tree-only world, it turns the entire tree into a text
file, then splits it by line and bisects over it.
Let's instead use the walk function, which is optimized for this kind of thing
on trees.
Reviewed By: quark-zju
Differential Revision: D12969497
fbshipit-source-id: ef3af9e0022978d6a4922cbb464bfd14248f5501
Summary: For the sake of incremental builds and `make local`.
Reviewed By: quark-zju
Differential Revision: D13058780
fbshipit-source-id: f59dc235e75232b475ca0c2504b47ff0d4a64c8a
Summary:
Add "--dry-run" for fix-code.py and use it in test-check.
This avoids license header and version = "*" issues.
Reviewed By: ikostia
Differential Revision: D10213070
fbshipit-source-id: 9fdd49ead3dfcecf292d5f42c028f20e5dde65d3
Summary:
This is done by running `fix-code.py`. Note that those strings are
semvers so they do not pin down the exact version. An API-compatiable upgrade
is still possible.
Reviewed By: ikostia
Differential Revision: D10213073
fbshipit-source-id: 82f90766fb7e02cdeb6615ae3cb7212d928ed48d
Summary:
`version = "*"` is not a good pattern. This script reads Cargo.lock and writes
down the version explicitly. This helps the OSS build.
Reviewed By: ikostia
Differential Revision: D10213074
fbshipit-source-id: 39720d895a5788b68d3e9d183d5e8941abc8c048
Summary:
This is a shorter, rewritten version of "WritingExtensions".
"WritingExtensions" is too verbose and some code examples are outdated.
Reviewed By: markbt
Differential Revision: D10287050
fbshipit-source-id: f989172c4a683142983e17d3fa9dbddfd8252586
Summary:
Add a new config option: `commitcloud.max_sync_age`. When set, commit cloud
will not pull in any commits that are older than this when it is joining or
syncing. The commits are still nominally in the cloud workspace, we just
save join or sync time by not including the commits.
Reviewed By: liubov-dmitrieva
Differential Revision: D13062470
fbshipit-source-id: 17a4bdb4095766a83a4bf6d4151ae86b39edf59c
Summary: This commit reverts an earlier change in D12950765 that removed options from 'hg smartlog'
Reviewed By: kulshrax
Differential Revision: D13029915
fbshipit-source-id: f514dca841cf9a48a46255c4eb0b376d8f0d2761
Summary: Log the uploading time spend and the downloading size when hg is using lfs
Reviewed By: ikostia
Differential Revision: D12993962
fbshipit-source-id: c53b189a12c60eece47dbbab0852fcfea9471363
Summary:
tweakdefaults used to do this if configured.
After dicussion with the team, we decided that returning 1 was not a helpful design decision, so we're going to fold this in here and everywhere.
Reviewed By: singhsrb
Differential Revision: D13050548
fbshipit-source-id: 66e834ea503e4b1339e369495a9729b951024a6d
Summary:
These commands should be able to use the full workspace name as well
Also the fix for commit cloud test file is included
Reviewed By: markbt
Differential Revision: D13042297
fbshipit-source-id: ac8c907292e2d15b72f56ef1cc831add5523b990
