Summary: The TestDag struct will be used to do some more complicated tests.
Reviewed By: sfilipco
Differential Revision: D23494507
fbshipit-source-id: 11350f9e448725ae49f50a7b6f19efc57ad84448
Summary:
A few things here:
- The heads must be bytes.
- The arguments to wireproto must be strings (we used to encode / decode them,
but we shouldn't).
- The bookmark must be a string (otherwise it gets serialized as `"b\"foo\""`
and then it deserializes to that instead of `foo`).
Reviewed By: StanislavGlebik
Differential Revision: D23499846
fbshipit-source-id: c8a657f24c161080c2d829eb214d17bc1c3d13ef
Summary:
Replacing places where the tokio runtime is instantiated inside the edenapi
client crate.
Reviewed By: quark-zju
Differential Revision: D23468596
fbshipit-source-id: ef68718c7d5b89b6477a2946daaa51618b53d06a
Summary:
At open time, it's pointless to attempt to create new levels. So let's just
read the existing max_level and do not try to build max_level + 1.
This turns out to save 300ms in profiling result.
Reviewed By: sfilipco
Differential Revision: D23494509
fbshipit-source-id: 4ea326a3cc21792790ea0b87e5bf608a94ae382b
Summary:
With MultiLog, per-log meta was previously entirely ignored. However, they can
be useful for updated indexes. For example, application defines a new index,
and opens a Log via MultiLog. The application would expect the new index is
built only once. Without MultiLog, per-log meta is updated at open time in
place. With MultiLog, the updated index meta is not written back to the
multimeta so the new index would be rebuilt multiple times undesirably.
Update MultiLog to reuse the per-log meta if it's compatible so it can pick up
new indexes.
Reviewed By: sfilipco
Differential Revision: D23488212
fbshipit-source-id: c8b3e6b5589dbda2e76a143d15085862a93dae22
Summary:
The poisoned meta makes investigation harder. ex. `debugdumpindexlog` won't
work on those logs.
Reviewed By: sfilipco
Differential Revision: D23488213
fbshipit-source-id: b33894d8c605694b6adf5afdaed45707fbd7357e
Summary:
Change dag_ops benchmarks to use different IdDagStores. An example run shows:
benchmarking dag::iddagstore::indexedlog_store::IndexedLogStore
building segments (old) 856.803 ms
building segments (new) 127.831 ms
ancestors 54.288 ms
children (spans) 619.966 ms
children (1 id) 12.596 ms
common_ancestors (spans) 3.050 s
descendants (small subset) 35.652 ms
gca_one (2 ids) 164.296 ms
gca_one (spans) 3.132 s
gca_all (2 ids) 270.542 ms
gca_all (spans) 2.817 s
heads 247.504 ms
heads_ancestors 40.106 ms
is_ancestor 108.719 ms
parents 243.317 ms
parent_ids 10.752 ms
range (2 ids) 7.370 ms
range (spans) 23.933 ms
roots 620.150 ms
benchmarking dag::iddagstore::in_process_store::InProcessStore
building segments (old) 790.429 ms
building segments (new) 55.007 ms
ancestors 8.618 ms
children (spans) 196.562 ms
children (1 id) 2.488 ms
common_ancestors (spans) 545.344 ms
descendants (small subset) 8.093 ms
gca_one (2 ids) 24.569 ms
gca_one (spans) 529.080 ms
gca_all (2 ids) 38.462 ms
gca_all (spans) 540.486 ms
heads 103.930 ms
heads_ancestors 6.763 ms
is_ancestor 16.208 ms
parents 103.889 ms
parent_ids 0.822 ms
range (2 ids) 1.748 ms
range (spans) 6.157 ms
roots 197.924 ms
benchmarking dag::iddagstore::bytes_store::BytesStore
building segments (old) 724.467 ms
building segments (new) 90.207 ms
ancestors 23.812 ms
children (spans) 348.237 ms
children (1 id) 4.609 ms
common_ancestors (spans) 1.315 s
descendants (small subset) 20.819 ms
gca_one (2 ids) 72.423 ms
gca_one (spans) 1.346 s
gca_all (2 ids) 116.025 ms
gca_all (spans) 1.470 s
heads 155.667 ms
heads_ancestors 19.486 ms
is_ancestor 51.529 ms
parents 157.285 ms
parent_ids 5.427 ms
range (2 ids) 4.448 ms
range (spans) 13.874 ms
roots 365.568 ms
Overall, InProcessStore > BytesStore > IndexedLogStore. The InProcessStore
uses `Vec<BTreeMap<Id, StoreId>>` for the level-head index, which is more
efficient on the "Level" lookup (Vec), and more cache efficient (BTree).
BytesStore outperforms IndexedLogStore because it does not need to verify
checksum on every read access - the checksum was verified at store creation
(IdDag::from_bytes).
Note: The `BytesStore` is something optimized for serialization, and hasn't been sent.
Reviewed By: sfilipco
Differential Revision: D23438174
fbshipit-source-id: 6e5f15188e3b935659ccde25fac573e9b963b78f
Summary: This allows them to use the SyncableIdDag APIs.
Reviewed By: sfilipco
Differential Revision: D23438170
fbshipit-source-id: 7ec7288cfb8186b88f85f0212a913cb0dffe7345
Summary: Other IdDagStores can also use the API. This will be used in benchmarks.
Reviewed By: sfilipco
Differential Revision: D23438180
fbshipit-source-id: 565552b66372dcfbb268c397883f627491d6e154
Summary:
Similar to `IdDagStore::sync` -> `GetLock::persist`, `reload` is more related
to filesystem/internal state exchange, and should be protected by a lock. So
let's move the API there, and requires a lock.
Reviewed By: sfilipco
Differential Revision: D23438169
fbshipit-source-id: 4228106b7739a1a758677adfddd213ad54aa4b6a
Summary:
`NameDag::reload` is used in `flush` to get a "fresh" NameDag.
In a future diff the `IdDag::reload` API gets changed, so let's
remove NameDag's use of it.
Instead, let's just re-`open` the path again to get a fresh NameDag.
It's a bit more expensive but probably okay, and easier to understand.
`get_new_segment_size()` was added as an internal API to preserve tests.
This also solves an issue where `NameDag` cannot recover properly if its
`flush` fails, because the old `NameDag` state is not lost.
After removing `NameDag::reload`, `idMap::reload` is no longer used publicly
and was made private.
Reviewed By: sfilipco
Differential Revision: D23438179
fbshipit-source-id: 0a32556a2cd786919c233d7efcae1cb9cbc5fb09
Summary:
The word "sync" is bi-directional: flush + reload. It was indexedlog::Log's
behavior. However, in the IdDag context "sync" is confusing - it is actually
only used to write data out, with protection from lock. Rename to `persist`
to clarify it's memory -> disk. Besides, requires a reference to a lock object
as a lightweight prove that some lock is held.
Reviewed By: sfilipco
Differential Revision: D23438175
fbshipit-source-id: 3d9ccd7431691d1c4e2ee74f3c80d95f5e7243b5
Summary:
This removes the need of cloning `IdMap`.
SyncableIdMap is a bit tricky. I added some comments to clarify things.
Reviewed By: sfilipco
Differential Revision: D23438176
fbshipit-source-id: fe66071da07067ed6c53a6437790af1d81b28586
Summary:
Make the test cover IndexedLogIdDagStore. The only change is the parent index
returns children in a different order.
Reviewed By: sfilipco
Differential Revision: D23438173
fbshipit-source-id: bcfabcd329e45bbc5e7e773103fa42307c23c35d
Summary:
There aren't too many thigs that we can do with the responses that we get back
from the server. Thigs are somewhat application specific for this endpoint.
One option that is not available right now and might make sense to add is
limiting the number of entries that are printed for a given location.
Reviewed By: kulshrax
Differential Revision: D23456220
fbshipit-source-id: eb24602c3dea39b568859b82fc27b7f6acc77600
Summary:
To reduce the size over the wire on cases where we would be traversing the
changelog on the client, we want to allow the endpoint to return a whole parent
chain with their hashes.
Reviewed By: kulshrax
Differential Revision: D23456216
fbshipit-source-id: d048462fa8415d0466dd8e814144347df7a3452a
Summary:
Renaming all the LocationToHash related structures to CommitLocationToHash.
This is done for consistency. I realized the issue when the command for reading
the request from cbor was not what I was expecting it to be. The reason was that
the commit prefix was used inconsistently for LocationToHash.
Reviewed By: kulshrax
Differential Revision: D23456221
fbshipit-source-id: 0181dcaf81368b978902d8ca79c5405838e4b184
Summary:
The default archive behavior archives the entire working copy. That is
undesirable and easy to accidentally trigger in a large repository. Let's
prevent it and require users to specify what they want archived.
Reviewed By: quark-zju
Differential Revision: D23464818
fbshipit-source-id: c39a631d618c2007e442e691cda542400cf8f4c3
Summary:
Replacing uses of the custom Runtime in lfs with the global runtime in the
`async-runtime` crate.
Reviewed By: xavierd
Differential Revision: D23468347
fbshipit-source-id: 61d2858634a37eb2d7d807104702d24889ec047a
Summary:
debugstacktrace is broken right now on Python 3: it wants to write to stderr,
which expects `bytes`, but it tries to write a `str`. This fixes it.
Reviewed By: DurhamG
Differential Revision: D23447984
fbshipit-source-id: 5896ae858f6022276fa47e08636c700159a2a678
Summary: Make it possible to test other IdDagStores.
Reviewed By: sfilipco
Differential Revision: D23438178
fbshipit-source-id: e5fc1b20833c71dd7569c77c31c76a26a6e357fe
Summary:
Now SpanSet can easily support `push_front`, we can just use SpanSet
efficiently without SpanSetAsc.
Reviewed By: sfilipco
Differential Revision: D23385246
fbshipit-source-id: b2e0086f014977fa990d5142e6eee844293e7ca5
Summary: To remove SpanSetAsc, its API needs to be implemented on SpanSet.
Reviewed By: sfilipco
Differential Revision: D23385250
fbshipit-source-id: ebd9d537287b5c1cde6e2c52ffb6da57dbd71852
Summary: This will make it possible to `push_front` and remove SpanSetAsc special case.
Reviewed By: sfilipco
Differential Revision: D23385249
fbshipit-source-id: 63ac67e9bce7cb281236399b3fb86eba23bbf8a0
Summary:
This makes it easier to replace Vec<Span> with VecDeque<Span> in SpanSet for
efficient push_front and deprecates SpanSetAsc (which uses Id in a bit hacky
way - they are not real Ids).
Reviewed By: sfilipco
Differential Revision: D23385245
fbshipit-source-id: b612cd816223a301e2705084057bd24865beccf0
Summary:
One user reports very very slow rebase (tens of minutes and running). The
commit is not very large. Python 2 can complete the rebase in 6 seconds.
I tracked it down to this code path. Making the change makes Python 3
rebase fast too (< 10 seconds). I haven't tracked down exactly why Python
3 is slow yet (maybe N^2 a += b)?
Some numbers about the slow merge:
ipdb> p len(m3.atext)
17984924
ipdb> p len(m3.btext)
17948110
ipdb> p len(m3.a)
613353
ipdb> p len(m3.b)
612129
ipdb> p len(m3.base)
612135
Reviewed By: singhsrb
Differential Revision: D23441221
fbshipit-source-id: 14b725439f4ecd3352edca512cdde32958b2ce29
Summary:
Previously the `is_valid()` function only checks about ordering.
Make it also check "no mergeable adjacent spans" and `span.low<=span.high`.
To provide better debug messages, the function does assertions
directly without returning a bool.
Reviewed By: sfilipco
Differential Revision: D23385247
fbshipit-source-id: 84829e9242e47e68dc2a4b2a6775b13331eba959
Summary:
Previously, `SpanSet::from_sorted_spans` allows having adjacent spans like
`[1..=2, 3..=4]`, while `SpanSet::from_spans` would merge them into `[1..=4]`.
Change it so `SpanSet::from_sorted_spans` merges them too. This simplifies
the `contains` logic and could make some Sets more efficient.
Reviewed By: sfilipco
Differential Revision: D23385248
fbshipit-source-id: 85b5ba9533f15034779e93255085a4fa09c6328a
Summary:
See the test change. Partially successful auto restack should have bookmarks
moved.
Reviewed By: DurhamG
Differential Revision: D23441932
fbshipit-source-id: 07e509a70bcc5cf81f702d40ec1b8dc4a5a781ff
Summary:
Those commands are broken right now: they try to write bytes but don't use
`writebytes`.
Reviewed By: DurhamG
Differential Revision: D23450968
fbshipit-source-id: 5d554771459f81718d90e5bad9a4c439cbb05d97
Summary:
When Python 3 wants to upload a file-like object, it does something a bit
awkward: it sets the `Transfer-Encoding` to `chunked`, but doesn't actually
chunk the data. Also, for some reason ,it still sets the `Content-Length`. I'm
not sure where that is coming from.
The thing is, when you set `Transfer-Encoding` to `chunked`, you do need to
chunk, or the other end is going to get very confused.
Unfortunately, this is not what happens here (note that the "send" logs are
from enabling http tracing in Python here, and those logs are basically one
line before `.send()` into a socket, so the chunking doesn't appear to happen
elsewhere):
```
[torozco@devbig051]~/opsfiles_bin % echo "aaaa" | ~/fbcode/buck-out/gen/eden/scm/__hg-py3__/hg-py3.sh debuglfssend https://mononoke-lfs.internal.tfbnw.net/opsfiles_bin
send: b'PUT /opsfiles_bin/upload/11a77c3d96c06974b53d7f40a577e6813739eb5c811b2a86f59038ea90add772/5 HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-length: 5\r\nx-client-correlator: tQT3yBfFEzhVtqI5\r\naccept: application/mercurial-0.1\r\ncontent-type: application/x-www-form-urlencoded\r\nhost: mononoke-lfs.internal.tfbnw.net\r\ntransfer-encoding: chunked\r\nuser-agent: mercurial/4.4.2_dev git/2.15.1\r\n\r\n'
sendIng a read()able
send: b'aaaa\n'
reply: 'HTTP/1.1 400 Bad request\r\n'
header: Content-Type: text/html; charset=utf-8
header: Access-Control-Allow-Origin: *
header: proxy-status: client_read_error; e_upip="AcLKajO63Vab0hC4kzGZQsqck3P_YOu7HsBzshC-NCbuo31tlWWqCiVw5xVLh44LYYe7qioCPqYSb8-1cBpdvFDZb_t5oYRP1Q"; e_proxy="AcJjRKHG02qo6Bv6fEPCUVF7DpCyrq3rmSnXhRLWakKWREEvVpk4jc-tzDyG6l9jvn3vNo8PYPG_5hLtC3L1"
header: Date: Tue, 01 Sep 2020 13:10:35 GMT
header: Connection: close
header: Content-Length: 2959
```
What's a bit confusing to me here is where this Content-length header comes
from. Indeed, normally Python 3 will:
- Not infer a content-length for file-like objects (which is what we have)
https://fburl.com/ms94eq31
- Set Transfer-Encoding if no Content-Length is present:
https://fburl.com/f81g8v2j
So, it's a bit unexpected that a) we have a Content-Length (we shouldn't), and
that we b) also have a Transfer-Encoding header. That said, setting the
Content-Length does fix the problem, so that's what this diff does.
Reviewed By: DurhamG
Differential Revision: D23450969
fbshipit-source-id: e1f535ff3d0b49c0c914130593d9aebe89ba18ca
Summary:
As a follow up to the previous diff, let's also warn if dirstate includes
marker files that should not be included in any sparse profiles.
Reviewed By: DurhamG
Differential Revision: D23414361
fbshipit-source-id: 3d171328bf0ba5754e5bacde85f09abb4fed8603
Summary: There seems to be no need to use a shell.
Reviewed By: DurhamG
Differential Revision: D23124756
fbshipit-source-id: 7de1c23e2325fe88dc4c6a2c90563d06f109ed2f
Summary:
The Rust process utility avoids issues with interaction with Python and can do file
redirection on Windows.
Reviewed By: DurhamG
Differential Revision: D23124755
fbshipit-source-id: f72b88bafd19b3b41e53afbf6a4095d0d6bcb93a
Summary:
The Rust bindings handle the cross-platform differences and avoids issues
with Python / Rust interaction. Use it.
As we're here, extend the API to support cwd and env.
Reviewed By: DurhamG
Differential Revision: D23124171
fbshipit-source-id: fdc13f6eaeb25c05b53d385eb220af33dad984e1
Summary:
Spawning processes turns out to be tricky.
Python 2:
- "fork & exec" in plain Python is potentially dangerous. See D22855986 (c35b8088ef).
Disabling GC might have solved it, but still seems fragile.
- "close_fds=True" works on Windows if there is no redirection.
- Does not work well with `disable_standard_handle_inheritability` from `hgmain`.
We patched it. See `contrib/python2-winbuild/0002-windows-make-subprocess-work-with-non-inheritable-st.patch`.
Python 3:
- "subprocess" uses native code for "fork & exec". It's safer.
- (>= 3.8) "close_fds=True" works on Windows even with redirection.
- "subprocess" exposes options to tweak low-level details on Windows.
Rust:
- No "close_fds=True" support for both Windows and Unix.
- Does not have the `disable_standard_handle_inheritability` issue on Windows.
- Impossible to cleanly support "close_fds=True" on Windows with existing stdlib.
https://github.com/rust-lang/rust/pull/75551 attempts to add that to stdlib.
D23124167 provides a short-term solution that can have corner cases.
Mercurial:
- `win32.spawndetached` uses raw Win32 APIs to spawn processes, bypassing
the `subprocess` Python stdlib.
- Its use of `CreateProcessA` is undesirable. We probably want `CreateProcessW`
(unless `CreateProcessA` speaks utf-8 natively).
We are still on Python 2 on Windows, and we'd need to spawn processes correctly
from Rust anyway, and D23124167 kind of fills the missing feature of `close_fds=True`
from Python. So let's expose the Rust APIs.
The binding APIs closely match the Rust API. So when we migrate from Python to
Rust, the translation is more straightforward.
Reviewed By: DurhamG
Differential Revision: D23124168
fbshipit-source-id: 94a404f19326e9b4cca7661da07a4b4c55bcc395
Summary:
The Rust upstream took the "set F_CLOEXEC on every opened file" approach and
provided no support for closing fds at spawn time to make spawn lightweight [1].
However, that does not play well in our case:
- On Windows:
- stdin/stdout/stderr are not created by Rust, and inheritable by
default (other process like `cargo`, or `dotslash` might leak them too).
- a few other handles like "Null", "Afd" are inheritable. It's
unclear how they get created, though.
- Fortunately, files opened by Python or C in edenscm (ex. packfiles) seem to
be not inheritable and do not require special handling.
- On Linux:
- Files opened by Python or C are likely lack of F_CLOEXEC and need special
handling.
Implement logic to close file handlers (or set F_CLOEXEC) explicitly.
[1]: https://github.com/rust-lang/rust/issues/12148
Reviewed By: DurhamG
Differential Revision: D23124167
fbshipit-source-id: 32f3a1b9e3ae3a9475609df282151c9d6c4badd4
Summary:
It uses `sys.argv`, which might be rewritten by `debugshell`. Capture
`sys.argv` to make hgcmd more reliable.
Reviewed By: DurhamG
Differential Revision: D22993215
fbshipit-source-id: 5fa319e8023b656c6cdf96cb3229ea9f2c9b9b99
Summary: This allows us to run commands after changes were made to the repo.
Reviewed By: DurhamG
Differential Revision: D22993218
fbshipit-source-id: d9943dcda94da42970fb9107f48f4caa14b6a9d4
Summary:
Some code paths (ex. metalog.commit) use `util.timer()` as a way to get
seconds since epoch, and get 0 for tests. Other use-cases of `util.timer()`
are ad-hoc time measure for displaying speed / progress. They do not need high
precision or strong guarantee that the clock does not go backwards. Drop the
`time.perf_counter()` to meet the first use-case's expectation.
Reviewed By: singhsrb
Differential Revision: D23431253
fbshipit-source-id: 8bf2d1ed32e284e17285742e1d0fd7178f181fb3
Summary:
With segments backend, the revision numbers will be longer than commit hashes
and are confusing.
Reviewed By: DurhamG
Differential Revision: D23408971
fbshipit-source-id: e2057fa644fc7b6be4291f879eee3235bb4e687b
Summary:
Pulling from older repos (ex. years ago) could require GBs of commit text data.
Flush commit data if they exceed certain size.
This is for revlog compatibility.
In the future we probably just make commit text lazy to avoid this kind of issues.
Reviewed By: DurhamG
Differential Revision: D23408834
fbshipit-source-id: 273384f5a05be07877bb1c9871c17b53ba436233
Summary: This would be used to avoid excessive memory usage during pull.
Reviewed By: DurhamG
Differential Revision: D23408833
fbshipit-source-id: 8edd95ab8201697074f65cc118d14755a230567d
Summary:
`addcommits` is designed to be more efficiently if called with a batch of
commits. So let's buffer the commits to add then only call it once.
This avoids some N^2 behaviors, for example, the NameDag internally will
prepare "snapshot" of itself which involves coping the pending Rust vecs
about the segments and id <-> hash map.
The change makes `pull` usable from unusably slow:
Original Python Revlog backend:
```
In [1]: %trace repo.pull(bookmarknames=['master'],quiet=False)
5191 +466 | Apply Changegroup edenscm.mercurial.bundle2 line 516
| - Commits = 125 :
| - Range = a1d1b3ade136:2e3fe78af189 :
5191 +466 | changegroup.cg1unpacker.apply edenscm.mercurial.changegroup line 313
5192 +416 | Progress Bar: commits (progressbar)
5192 +415 | changelog.changelog.addgroup edenscm.mercurial.changelog line 536
5192 +409 | revlog.revlog.addgroup edenscm.mercurial.revlog line 2116
5215 +371 | changelog.changelog._addrevision (125 times) edenscm.mercurial.changelog line 558
```
DoubleWrite (Segments + Revlog) backend, Before:
```
In [2]: %trace repo.pull(bookmarknames=['master'],quiet=False)
2396 +154059 | Apply Changegroup edenscm.mercurial.bundle2 line 516
| - Commits = 323 :
| - Range = cb0b100180ba:5fb57c74f72e :
2396 +154059 | changegroup.cg1unpacker.apply edenscm.mercurial.changegroup line 313
2397 +151433 \ Progress Bar: commits (progressbar)
2397 +151433 | changelog2.changelog.addgroup edenscm.mercurial.changelog2 line 334
```
DoubleWrite (Segments + Revlog) backend, After:
```
In [2]: %trace repo.pull(bookmarknames=['master'],quiet=False)
4629 +512 | Apply Changegroup edenscm.mercurial.bundle2 line 516
| - Commits = 45 :
| - Range = cf23c6972934:1ff0c5f0e7ad :
4629 +512 | changegroup.cg1unpacker.apply edenscm.mercurial.changegroup line 313
4630 +494 | changelog2.changelog.addgroup edenscm.mercurial.changelog2 line 334
```
Reviewed By: DurhamG
Differential Revision: D23390435
fbshipit-source-id: dd97a5008dedd844d4134b87bfef190fa739a80b
Summary:
The users of addrevisoncb are gone.
This also removes the "alwayscache" parameter of "_addrevision".
Reviewed By: DurhamG
Differential Revision: D23390437
fbshipit-source-id: 7edd9dd0b93d4cb9d4f35d088a1aef719b450ec1
Summary: It is about legacy revlog formats that are no longer relevant.
Reviewed By: DurhamG
Differential Revision: D23390436
fbshipit-source-id: 58c2c432804181bcc6517d6c988777b843fc9ba4
Summary:
We have a few safeguards against creating full checkouts. However we have
sparse profiles that are not full, but that include very large directories
which normally should not be included.
This diff adds a logic that checks if a new sparse profile has any of the "marker"
files i.e. some files from a folder that should not be included. Operation
aborts if that the case, however there's always a way to workaround that.
Reviewed By: DurhamG
Differential Revision: D23414200
fbshipit-source-id: 626f392319eb1be8b35f39cadafb61f3c1dfefe3
Summary:
"hg diff" has --sparse option which diffs only files inside a sparse checkout.
The problem is that it doesn't work on eden checkouts because eden repo doesn't
have sparsematch() function.
This diff makes it so that if sparsematch() function doesn't exist then
--sparse option is just ignored.
The motivation for this change is
https://fb.workplace.com/groups/corehg/?post_id=687768245151742. There are some
diff calls that are triggered by arc lint that race with "hg update" and might download
loads of data on people's laptops. This diff doesn't fix the race, but it:
1) Makes sure we don't download too much data that are not in sparse profiles.
2) arc lint doesn't care about files outside of sparse profiles anyway, so
running --sparse make sense.
Reviewed By: DurhamG
Differential Revision: D23396918
fbshipit-source-id: 2a386fdbeab85187e2c2acab69cb86b74124d46f
Summary:
This is practically just 0 in our production setup during `pull`s. In the
future when the commit data become lazy, it's no longer possible to read the
files locally. So let's just don't scan the commits.
Reviewed By: DurhamG
Differential Revision: D23390438
fbshipit-source-id: 4c54c4aac5fd840205296ab86955ec1b8ab76607
Summary:
Mergedrivers can call dirstate.add directly and are adding paths with
"." and "..". Let's block those paths.
Reviewed By: quark-zju
Differential Revision: D23375469
fbshipit-source-id: 64e9f20169cfd50325ecd8ebcc1dd3be7a5cb202
Summary:
extdiff uses shutil.rmtree which calls os.rmdir with new python 3
options. Since we pathc os.rmdir, we need to support those options.
Reviewed By: quark-zju
Differential Revision: D23350968
fbshipit-source-id: 081d179dcd67b51ffdeb6b85899adf4e574a8d0f
Summary: Similar to D18528858 so module names do not need to be spelled twice.
Reviewed By: markbt
Differential Revision: D23091380
fbshipit-source-id: a2a261abc9c78c8805cea62b38498ba65398796d
Summary: This crate would fail to build without the "fb" feature because `serde_json` was listed as an optional dependency (but is used in a way that isn't conditional on the `fb` feature). This diff makes the dependency non-optional, and also silences several dead code warnings that are emitted when building without the "fb" feature.
Reviewed By: quark-zju
Differential Revision: D23386786
fbshipit-source-id: b00a8b0b8b0b978c1cfab2838629fcb388a076e9
Summary:
The `debugfsync` command calls fsync on newly modified files in svfs.
Right now it only includes locations that we know have constant number
of files.
The fsync logic is put in a separate crate to avoid slow compiles.
Reviewed By: DurhamG
Differential Revision: D23124169
fbshipit-source-id: 438296002eed14db599d6ec225183bf824096940
Summary:
A warning means that every tree fetched will be printed in the edenfs log,
which is way too much. Let's decrease this to a debug message.
Reviewed By: genevievehelsel
Differential Revision: D23385778
fbshipit-source-id: d77f1cac3efb945d4b95750822f2f12f48c75ffe
Summary: `len(repo)` can no longer predicate the next rev number. Use nodes instead.
Reviewed By: DurhamG
Differential Revision: D23307791
fbshipit-source-id: cc20e53f039eee2a714748352e8e98aab253095a
Summary:
Some functions might be called very frequently. For example,
`phases.phasecache.loadphaserevs` might be called 100k+ times.
That makes the tracing data harder to process.
Limit the count of spans to 1k by default so the data is cheaper to process,
and some highly repetitive cases can now be reasoned about. Note the limit
is only put on static Span Ids. If a span uses dynamic metadata or ask for
different Span Ids each time, they will not be limited.
In debugshell,
td = %trace repo.revs('smartlog()')
len(td.serialize())
dropped from 6MB to 0.87MB.
It's also possible to reason about:
td = %trace len(repo.revs('ancestors(.)'))
in debugshell (taking 30s, 98KB serialized, vs 21s without tracing), while
previously the result would be too large to show (`%trace` just hangs).
Reviewed By: DurhamG
Differential Revision: D23307793
fbshipit-source-id: 3c1e9885ce7a275c2abd8935a4e4539a4f14ce83
Summary: Set a default limit so the output won't be too long.
Reviewed By: DurhamG
Differential Revision: D23307792
fbshipit-source-id: 7e2ed99e96bbde06436a034e78f899fc2e3e03f8
Summary:
The debugshell command can be long running and contains uninteresting stuff.
Do not profile it.
Practically this hides showing the background statprof thread when using `%trace`.
Reviewed By: DurhamG
Differential Revision: D23278597
fbshipit-source-id: bad97de22e1be2be8b866bee705ea3a6755aa54b
Summary:
This allows entering ipdb for code like: `ipdb` or `ipdb()`. It can be handy to
debug something.
Reviewed By: DurhamG
Differential Revision: D23278599
fbshipit-source-id: 4355dd1944617aeb795450935789f01f66f094eb
Summary: This makes it possible to get tracing results, or run hg commands directly.
Reviewed By: DurhamG
Differential Revision: D23278601
fbshipit-source-id: e7dc92080d2881cb4155a481df5ca93f324828fc
Summary:
The `--trace` flag enables tracing Python modules.
For compatibility reasons, it also enables `--traceback`.
It can be used with debugshell to make `%trace` more useful.
Reviewed By: sfilipco
Differential Revision: D23278600
fbshipit-source-id: d6d0b34bd5c48111f8cd33d7df115f349b0e95b6
Summary:
I found this when I aborted an rebase Dxxx and trying rebasing again and it
complained about "nothing to rebase". It was caused by Dxxx resolving into
a hidden commit.
Reviewed By: sfilipco
Differential Revision: D23307794
fbshipit-source-id: f7a956b5300240089b6a4648f28cf4a152ee2433
Summary:
We shouldn't delete from a dictionary while iterating over it, instead we should iterate over a copy and then delete from the original.
`.items()` returns a view of the dict, while wrapping it in `list` makes a deep copy.
Reviewed By: DurhamG
Differential Revision: D23283668
fbshipit-source-id: a168eef1ed2a1ce02fe71b3f6e3aed090965d2a4
Summary:
Mononoke throws an error if we request the nullid. In the long term we
want to get rid of the concept of the nullid entirely, so let's just add some
Python level blocks to prevent us from attempting to fetch it. This way we can
start to limit how much Rust has to know about these concepts.
Reviewed By: sfilipco
Differential Revision: D23332359
fbshipit-source-id: 8a67703ba1197ead00d4984411f7ae0325612605
Summary:
Corp has a different concept of tier than prod. Let's load the corp
tier into our tier set as well.
Reviewed By: quark-zju
Differential Revision: D23354056
fbshipit-source-id: c9543b8253f042c7b1224578e0687b4bdf21738e
Summary:
The Python 3 email library internally stores the message as text, even
though our input and requested output is bytes. Let's make our own wrapper
around the parser to use ascii surrogateescape encoding so we can get the
actual bytes out later and not get universal newlines.
Based off the upstream 7b12a2d2eedc995405187cdf9a35736a14d60706,
which is basically a copy of the BytesParser implementation (https://github.com/python/cpython/blob/3.8/Lib/email/parser.py) with
newline=chr(10) added.
Reviewed By: quark-zju
Differential Revision: D23363965
fbshipit-source-id: 880f0642cce96edfdd22da5908c0b573887bed12
Summary:
`hg cloud rejoin` command is used in fbclone and it is supposed to print a
message on RegistrationError but this has been broken recently.
Reviewed By: markbt
Differential Revision: D23342773
fbshipit-source-id: 4f3318848953656dea65a2b5d4d832694f6b353c
Summary:
There are users who prefer run `hg cloud leave` if they notice they are
connected to commit cloud sync.
Proving more information and add a prompt might help them to change their mind.
For some users who left new fbclone will connect them back. So on next leave they can learn more information about Commit Cloud Workspaces.
Reviewed By: markbt
Differential Revision: D23346091
fbshipit-source-id: 72f170f7133cd64b772ec75ae29a85dc8809e351
Summary:
When updating to the null commit, the logic that computes the update
distance was broken. The null commit is pre-resolved to -1, which when passed to
a revset raw gets resolved as the tip commit. In large repositories this can
take a long time and use a lot of memory, since it's computing the difference
between tip and null.
Let's fix it to not pass the raw rev number, and also to handle the case of a 0
distance update.
Reviewed By: quark-zju
Differential Revision: D23358402
fbshipit-source-id: 3b0a1fe1bbcb07effba4d0ab2c092e66bdc02e67
Summary:
Pull Request resolved: https://github.com/facebookexperimental/eden/pull/46
See https://github.com/facebookexperimental/eden/runs/1034006668:
error: unused import: `env::set_var`
--> src/lfs.rs:1539:15
|
1539 | use std::{env::set_var, str::FromStr};
| ^^^^^^^^^^^^
|
note: the lint level is defined here
--> src/lib.rs:125:9
|
125 | #![deny(warnings)]
| ^^^^^^^^
= note: `#[deny(unused_imports)]` implied by `#[deny(warnings)]`
error: unnecessary braces around method argument
--> src/lfs.rs:2439:36
|
2439 | remote.batch_upload(&objs, { move |sha256| local_lfs.blobs.get(&sha256) })?;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: remove these braces
|
note: the lint level is defined here
--> src/lib.rs:125:9
|
125 | #![deny(warnings)]
| ^^^^^^^^
= note: `#[deny(unused_braces)]` implied by `#[deny(warnings)]`
error: aborting due to 2 previous errors
error: could not compile `revisionstore`.
I dropped `#![deny(warnings)]` as I don't think warnings like the above ones
should break the build. (denying specific warnings that we care about explicitly
might be a better approach)
Reviewed By: singhsrb
Differential Revision: D23362178
fbshipit-source-id: 02258f57727edfac9818cd29dda5e451c7ca80a7
Summary: Now that it is possible to control which features are enabled on manually-managed dependencies, we can reenable autocargo for `edenapi`. See D23216925, D23327844, and D23329351 (840e6dd6f6) for context.
Reviewed By: dtolnay
Differential Revision: D23335122
fbshipit-source-id: 8ce250c3a106d2a02f457f7ed531623dd866232f
Summary: The command does not crash but `-` lines are ignored.
Reviewed By: DurhamG
Differential Revision: D23357655
fbshipit-source-id: f48568bc193f947503bc19f3e192b33346c317e1
Summary:
Pull Request resolved: https://github.com/facebookexperimental/eden/pull/45
Fix referring to 'version' without proper codegen by making 'version' compile
without codegen. This fixes configparser test when version/src/lib.rs was not
generated.
Make unneeded deps without 'fb' feature optional.
This would hopefully fix the "EdenSCM Rust Libraries" GitHub workflow.
Reviewed By: DurhamG
Differential Revision: D23269864
fbshipit-source-id: f9e691fe0a75159c4530177b8a96dad47d2494a9
Summary: This makes the code simpler.
Reviewed By: sfilipco
Differential Revision: D23269858
fbshipit-source-id: bb9ac0bd1696f7429ca1856e6c63e04fabc2757a
Summary: This makes the code simpler.
Reviewed By: sfilipco
Differential Revision: D23269866
fbshipit-source-id: 30c9e9d218378c0d6df8b822b2a81df2b38f5b01
Summary: Will be used to simplify code.
Reviewed By: sfilipco
Differential Revision: D23269859
fbshipit-source-id: bed0c4dca075ff60900025642af1d84bdd03452d
Summary:
`impl<T> Trait for T` in the current Rust makes it impossible to have
`impl<Q> Trait for Q`. Avoid using it for IdConvert and PrefixLookup.
Reviewed By: sfilipco
Differential Revision: D23269861
fbshipit-source-id: a837f3984ff4e1bd5a3983dd1642b9f064f51a36
Summary:
`impl<T> Trait for T` in the current Rust makes it impossible to have
`impl<Q> Trait for Q`. Avoid using it for DagAlgorithm.
Reviewed By: sfilipco
Differential Revision: D23269860
fbshipit-source-id: 031e75e9bf1f1eec2b9e8f36220ef8b817a143a5
Summary: LowLevelAccess is a subset of NameDagStorage. Use the latter instead.
Reviewed By: sfilipco
Differential Revision: D23269865
fbshipit-source-id: 81ebb1e986d8b02c968a9a237ad9a97d4afd54bf
Summary:
If there are too many heads, the current `descendants` algorithm would visit
all "old" heads. For example, with this graph:
head9999 (N9999)
/
Z (master)
:
: (many heads)
:/
: head2 (N2)
:/
C head1 (N1)
|/
B head0 (N0)
|/
A
`A::head9999` or `Z::head9999` will visit N0, N1, ..., N9999, because
`descendands_up_to` is provided with `max_id = N9999` and Z as a vertex in the
master group, is before N0 in non-master. The current algorithm also means
`descendands_up_to` gets linearly slower as the user uses the repo more, which
is quite undesirable.
This diff changes `descendands_up_to` to take an `ancestors` set, which is
`::head9999` in this case, and iterate non-master flat segments in it. So it
will skip N0 to N9998 directly by finding the N9999 flat segment and only use
it. The number of heads will have a smaller impact on performance.
Another slowness is `draft::draft_heads`, if there are too many `draft_heads`,
the internal calculation of `::draft_heads` can be slow. Optimize it by
limiting `draft_heads` to `draft:`. Practically this affects `y::` revset as
`y::` is translated to `y::visible_heads` and `visible_heads` can be large.
`cargo bench --bench dag_ops -- '::-master'` shows significant difference:
Before:
range (master::draft) 18.112 s
range (recent_draft::drafts) 2.594 s
After:
range (master::draft) 72.542 ms
range (recent_draft::drafts) 14.932 ms
In my fbsource checkout there were 20k+ heads. The improvement of
`master::recent_draft` (`x::y`) is pretty visible, and `y::` is also improved:
% lhg debugbenchmarkrevsets -m -x 'p1(min(7e8c86ae % master))' -Y 'draft() & 7e8c86ae' -e 'x::y' -e 'y::' --no-default
# x: 168f5228e570fb6b2ff7f851bd82413102748d84 (p1(min(7e8c86ae % master)))
# y: 7e8c86aec68ebc6e0b8254afcb381315991fd21c (draft() & 7e8c86ae)
# before
| revset \ backend | segments | revlog | revlog-cpy |
|------------------|----------|--------|------------|
| x::y | 17ms | 0.1ms | 0.5ms |
| y:: | 3.3ms | 0.7ms | 1.3ms |
# after
| revset \ backend | segments | revlog | revlog-cpy |
|------------------|----------|--------|------------|
| x::y | 0.2ms | 0.1ms | 0.6ms |
| y:: | 1.0ms | 0.7ms | 1.3ms |
Reviewed By: sfilipco
Differential Revision: D23214387
fbshipit-source-id: 4d11db84cd28f4e04e8b991cbc650c9d5781fd27
Summary:
Lots of non-master heads is not an exercised graph in the benchmarks.
Add it as it practically happens. This will be used by the next change.
Reviewed By: sfilipco
Differential Revision: D23259879
fbshipit-source-id: 7fe290d14403e42e6d135bde56e2d5c8519ae530
Summary:
Currently the fuzz test only uses the master group. Let it exercise non-master
group too.
Reviewed By: DurhamG
Differential Revision: D23214388
fbshipit-source-id: 7108a1055fbdda2b012f93c5948fb83ef3b9a96f
Summary:
The calculation can take tens of milliseconds. Cache it.
Invalidate the cache on transaction commit.
This will improve perf on revsets like `descendants` that will use
`head()`.
Reviewed By: DurhamG
Differential Revision: D23196412
fbshipit-source-id: 2913310ebb97e1c0346198c1e2738799799c740a
Summary: Provide a way to see segments.
Reviewed By: sfilipco
Differential Revision: D23196408
fbshipit-source-id: b1418f945a5a3364ac73b0f97466d973dd4b6300
Summary:
Provide a way to print out all segments with resolved names. This will be used
in a debug command.
Reviewed By: sfilipco
Differential Revision: D23196410
fbshipit-source-id: 1712bfda0271aa548699fe4a6b8603c5ec07af7f
Summary:
Use the parent-child index to answer children query quickly.
`cargo bench --bench dag_ops -- children`:
Before:
children (spans) 606.076 ms
children (1 id) 124.105 ms
After:
children (spans) 602.999 ms
children (1 id) 10.777 ms
Reviewed By: sfilipco
Differential Revision: D23196411
fbshipit-source-id: 37195d5ccaa582d35314e0000352ef477287d38c
Summary: This will be used to optimize "children(single vertex)" query.
Reviewed By: sfilipco
Differential Revision: D23196409
fbshipit-source-id: 050c0859faf83b909e3174bb7c7bd6e7725165c0
Summary:
Update the parent index to store non-master group too. To make
"remove_non_master" work, the index contains a "child group" prefix that
allows efficient range invalidation.
This will allow answering "children(single vertex)" query more efficiently.
This diff does not expose an API to query the index yet.
Reviewed By: sfilipco
Differential Revision: D23196406
fbshipit-source-id: 9137da5ffa8306bdafbcabc06b6f0d23f38dcf57
Summary:
Practically, the input of `children` is often one vertex instead of a large set.
Add a benchmark for it.
It looks like:
children (spans) 606.076 ms
children (1 id) 124.105 ms
Reviewed By: sfilipco
Differential Revision: D23196407
fbshipit-source-id: 0645b59ac846836fd061386384f6386a57661741
Summary: They can be figured out at Hints initialization time. So they don't need to be mutable.
Reviewed By: sfilipco
Differential Revision: D23182518
fbshipit-source-id: 133375fdf27a2546a50b63fb130534acdadc5938
Summary:
Both IdSet and IdLazy set require both Dag and IdMap to construct.
This is step 1 torwards making Dag and IdMap immutable in hints.
A misspeall of "lhs" vs "hints" in the union set is discovered by the change
and fixed.
Reviewed By: sfilipco
Differential Revision: D23182520
fbshipit-source-id: 3d052de4b8681d3672ebc45d953d1e784f64b2a4
Summary:
It will be used in places (ex. tests) where a Dag is required but constructing
a real Dag is troublesome.
Reviewed By: sfilipco
Differential Revision: D23182517
fbshipit-source-id: 736911365778e5071c1e0b9615090a4e960392a0
Summary: This is more consistent with `id_map_snapshot`.
Reviewed By: sfilipco
Differential Revision: D23182519
fbshipit-source-id: 62b7fc8bfdc9d6b3a4639a6518ea084c7f3807dd
Summary:
Similar to descendants, the new range algorithm avoids potentially expensive
checks about whether high-level segments can be used or not. Practically this
is overall an improvement.
`cargo bench --bench dag_ops -- range`:
Before:
range (2 ids) 115.380 ms
range (spans) 243.666 ms
After:
range (2 ids) 123.274 ms
range (spans) 23.101 ms
It is 100x faster with the range x::y benchmark added later on `git.git`.
Reviewed By: sfilipco
Differential Revision: D23106175
fbshipit-source-id: 691e0418ba2b7ad9f52ac15b5cd6088ec28d5f48
Summary:
The old algorithm tries to make use high-level segments.
However, the code to test whether a high-level segment can be used is
often too expensive for the benefit. Often, high-level segments cannot
be used most of the time and it's similar to O(flat segments).
This diff adds a simpler algorithm that just iterates through the flat
segments. It's faster in most practical cases.
`cargo bench --bench dag_ops -- descendants` shows improvements too:
Before:
descendants (small subset) 436.515 ms
After:
descendants (small subset) 33.460 ms
Reviewed By: sfilipco
Differential Revision: D23106174
fbshipit-source-id: e6101483d8539b2b1c881be2ccfd0071f122352f
Summary: This will be used by upcoming changes.
Reviewed By: sfilipco
Differential Revision: D23106177
fbshipit-source-id: 9bf183f7464c06b801be64fd938db0babd544756
Summary: This internal struct will be used by upcoming changes.
Reviewed By: sfilipco
Differential Revision: D23106172
fbshipit-source-id: 6d5b9bc1c810984814d0912100acca38a2565a63
Summary:
Our internal build infra creates a workspace and workspaces don't like
it when two crates have the same name. Eden scm had third-party rust crates that
were simple redirects to the internal location, but had the same name. This
caused breakages once these crates became part of the edenfs open source build.
Let's rename them to avoid this issue.
Reviewed By: kulshrax
Differential Revision: D23252539
fbshipit-source-id: 9ff2fa160a19c6bc54e015c71f9da7044ce659a7
Summary:
Windows doesn't seem to follow the same write permission rules, so
let's just skip that part of the test for now.
Reviewed By: xavierd
Differential Revision: D23348515
fbshipit-source-id: bfcfa4f8adf94413047b045987e570ba98f9078d
Summary:
We'd like to keep track of hg cache size on users' machines. It's generally
useful, but it will be even more important with megarepo rollout. The job will be triggered by cron.
This is a very simple implementation, most of it was copy-pasted from
https://fburl.com/diffusion/g8ysyxo1.
Note that I intentionally didn't call hg to get hgcache size.
Since this binary will be triggered by cron, it will run as
root without a direct access to user's repository.
A few notes for the future:
1) Currentlly I'm planning to get hgcache path value from opsfiles.
I'm not sure if we'd need to change this later when dynamic configs are fully
rolled out. CC DurhamG
2) We might want to track size of pack files (are they still used at all?) and
indexed log files separately
Reviewed By: ikostia
Differential Revision: D23341149
fbshipit-source-id: 2a600d7a8034ac887014788f1024fb9866c3ef76
Summary:
Some user may think that the progress reporting is spammy, others can like it.
Make it configurable without disabling status completely.
We could explain the new config 'enableprogress' in the wiki for those who would like to disable the messages.
Also add 'enablestatus' under commitcloud section of config.
Reviewed By: markbt
Differential Revision: D23304206
fbshipit-source-id: 7735ae3284d19b9f1ea9da38b705a8932b34f91b
Summary:
It will be clearer to provide both names because all other commands (like `cloud
sl`, `cloud join`) use short names and if you would like to use the long one you would need to provide
'--raw-workspace' option and this is an advanced option.
Reviewed By: markbt
Differential Revision: D23274288
fbshipit-source-id: d751346a2e49cab21d1054eb196f7977c72c8764
Summary:
We have a thread blocking application. We have async libraries. This crate
provides common utilities for communicating between the blocking world and the
async world. It is intended to be a guide so that not all developers have to
get in depth understanding of Tokio in order to use async functions.
Reviewed By: quark-zju, xavierd
Differential Revision: D23222876
fbshipit-source-id: b9a61795bc917bfc664c9d6da95c9e5e2d506c79
Summary: The default method implementations on this trait were causing unused variable warnings. Prefix them with underscores to silence the warning and add `#![deny(warnings)]` to `revisionstore` to prevent future warnings from slipping through the cracks.
Reviewed By: singhsrb
Differential Revision: D23334309
fbshipit-source-id: d17b27ca0dd462e1613eac918fb595faa8637741
Summary: This diff fixes the issue that mkscratch does not build on Rust < 1.45.0
Reviewed By: xavierd
Differential Revision: D23325087
fbshipit-source-id: 7e737eb5bc0d75255a84c8ac353bf3a17c269300
Summary:
This was just a dumb issue. I added a test that depended on the
previous logic which didn't run on OSX.
Reviewed By: kulshrax, singhsrb
Differential Revision: D23324601
fbshipit-source-id: 259175c91c9eb54e8350f196770ff967b53ee841
Summary:
We want to eventually get rid of system and repo configs, but for now
they should take precedence over the dynamicconfigs. Previously we relied on
validation to remove any entries from dynamicconfig that interferes with a
system rc config, but in some code paths we didn't run that validation (like if
we loaded configs purely from Rust).
Let's just make dynamicconfig load before system configs. If validation doesn't
run, we might miss the case where dynamicconfig sets a value and the system rc
doesn't. But that's probably fine.
Reviewed By: quark-zju
Differential Revision: D23305711
fbshipit-source-id: 77b5f49d348cfa116694a641ed17e6d1184a81ab
Summary:
Dynamicconfigs are now part of our critical path. Let's remove the
option to not load them. This also let's us get rid of a circularl dependency
where loading dynamicconfigs required having already loaded some configs. This
will let us move dynammicconfig loading to be before system rc loading in a
later diff.
Reviewed By: sfilipco
Differential Revision: D23309090
fbshipit-source-id: 5138059a8ed944c3616007e7c1289b6a57be0e65
Summary:
An earlier diff moved all dynamicconfig loadding into Rust, but it lost
some of the config validation along the way. This allowed dynamicconfig values
to take precedence over system rcs when they shouldn't yet. Most notably the
remotefilelog.cachepath value.
Let's ensure validation is run on all load paths. A future diff will go a step
farther and move dynamicconfigs to be loaded before system configs so we can
ensure system configs always take precedence, until we can remove them entirely.
Reviewed By: quark-zju
Differential Revision: D23305712
fbshipit-source-id: 33a6b4c56d97fa2e2e8f3acc343a8a8868b797ef
Summary:
Python 2's `socket.error` type provided a tuple of (errno, error
string) that could be used for identifying the specific error type. In Python 3
this was switched to an object that doesn't expose such an interface. As a
result, accessing `socket.error` using an index will cause an exception, such
as:
```
File "/opt/fb/mercurial/edenscm/mercurial/keepalive.py", line 608, in safesend
if v[0] == errno.EPIPE: # Broken pipe
TypeError: 'BrokenPipeError' object is not subscriptable
```
Replace the use of indexing with accessing `.errno` instead, as this works for
both python 2 and python 3.
Differential Revision: D23294090
fbshipit-source-id: 5f901493c2db270817949d93840c6d469b17a560
Summary:
Dynamicconfig was throwing errors if hgrc.dynamic wasn't writable.
Let's eat those errors for normal read operations. We still treat it as an error
for straight hg debugdynamicconfig invocations.
Reviewed By: quark-zju
Differential Revision: D23301100
fbshipit-source-id: ed0bd1282d2c7ee747f0909c238a5fa07b7bc9bc
Summary:
We've seen user reports of this error. Let's add a test to demonstrate
it. The next diff will fix it.
Reviewed By: sfilipco
Differential Revision: D23309612
fbshipit-source-id: 6fb9e4e65d3351fa29812fc75095d054465cfe13
Summary:
On Windows, Rust's `std::fs::canonicalize` [1] will generate extended-length path that will include a `\\?\` prefix [2]. This has subsequently cause `encode` to generate a path that contains a question mark, which is an invalid path on Windows.
This diff teaches `encode` to handle extended-length path on Windows. It essentially converts the path back so it no longer contains the prefix.
[1] http://doc.rust-lang.org/1.45.2/std/fs/fn.canonicalize.html
[2] https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file#maximum-path-length-limitation
Reviewed By: wez
Differential Revision: D22956659
fbshipit-source-id: 54691e204d7cb481bdb40f62c6520c0f70c3f648
Summary:
In python 3 curses sometimes throws an error when weird keys are
pressed. I'm not certain exactly what key causes the problem, but let's just
prevent all such errors from crashing the process.
Reviewed By: quark-zju
Differential Revision: D23310301
fbshipit-source-id: a9684ce6f690d0753ff9956ef9f13c330eb0a77b
Summary:
Based on [user report](https://fb.workplace.com/groups/scm/permalink/3128221090560823/).
Note that slices in rust behave differently and if index exceeds slice size this will always be panic. My fix was based on assumption that behavior should be similar to python.
Reviewed By: quark-zju
Differential Revision: D23263922
fbshipit-source-id: 3d2a1a1b59f14e43b1f1a2b7102982b11637c0b4
Summary:
This could help simplify the graph a lot for repos with lots of merges. For
example, logging tags on linux.git looks like:
o fb893de3 Yesterday at 17:28 master
├─┬─┬─┬─┬─┬─┬─┬─┬─┬─╮
╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ o bcf87687 Aug 02 at 14:21 v5.8
╷ ╷ ╷ ╭─────┬─┬───┬─╯
╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ o 92ed3019 Jul 26 at 14:14 v5.8-rc7
╷ ╷ ╷ ╭─────┬─┬─┬─╯
╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ o ba47d845 Jul 19 at 15:41 v5.8-rc6
╷ ╷ ╷ ╭─┬─┬─┬─┬─╯
╷ ╷ ╷ ╷ ╷ ╷ ╷ o 11ba4688 Jul 12 at 16:34 v5.8-rc5
╷ ╷ ╷ ╭─┬─┬─┬─╯
╷ ╷ ╷ ╷ ╷ ╷ o dcb7fd82 Jul 05 at 16:20 v5.8-rc4
╷ ╷ ╷ ╭─┬─┬─┤
╷ ╷ ╷ ╷ ╷ o ╷ 9ebcfadb Jun 28 at 15:00 v5.8-rc3
╷ ╷ ╭─┬─┬─╯ ╷
╷ ╷ ╷ ╷ o ╷ 48778464 Jun 21 at 15:45 v5.8-rc2
╷ ╷ ╷ ╭─╯ ╷
╷ ╷ ╷ o ╷ b3a9e3b9 Jun 14 at 12:45 v5.8-rc1
╭─┬─┬─┼─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─┬─╮
╷ ╷ o ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ 3d77e6a8 May 31 at 16:49 v5.7
╭─┬─┴───────┬───────────┬─┬───┬─╮
╷ o ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ 9cb1fd0e May 24 at 15:32 v5.7-rc7
╷ ╰─────────┬─────────────┬─┬─┬─╮
╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ o b9bbe6ed May 17 at 16:48 v5.7-rc6
╭───────────┬─────────────┬─┬─┬─╯
╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ o 2ef96a5b May 10 at 15:16 v5.7-rc5
╭───────────┬─────────────┬─┬─╯
╷ ╷ ╷ ╷ o ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ 0e698dfa May 03 at 14:56 v5.7-rc4
╭───────────┴───────────┬─┬─╮
o ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ 6a8b55ed Apr 26 at 13:51 v5.7-rc3
╰─────────────────┬───────┬─╮
╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ o ae83d0b4 Apr 19 at 14:35 v5.7-rc2
╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╭─┤
╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ o ╷ 8f3d9f35 Apr 12 at 12:35 v5.7-rc1
╭─┬─┬───────┬─────┬─┬─┬─┬─┼─╮
╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ o ╷ ╷ 7111951b Mar 29 at 15:25 v5.6
╷ ╭─────────┬─────┬─┬─┬─┴───╮
╷ ╷ ╷ ╷ ╷ ╷ o ╷ ╷ ╷ ╷ ╷ ╷ ╷ 16fbf79b Mar 22 at 18:31 v5.6-rc7
╷ ╷ ╭───────┴─────┬─┬─┬─────╮
╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ o fb33c651 Mar 15 at 15:01 v5.6-rc6
╷ ╭─┬─────────────┬─┬─┬─────╯
╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ o ╷ 2c523b34 Mar 08 at 17:44 v5.6-rc5
╷ ╭─┬─────────────┬─┬─╯ ╷
╷ ╷ o ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ 98d54f81 Mar 01 at 14:38 v5.6-rc4
╷ ╭─┴─────────────┬─╮ ╷
╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ o ╷ f8788d86 Feb 23 at 16:17 v5.6-rc3
....
And with simplification turned on, it looks like:
o fb893de3 Yesterday at 17:28 master
├─╮
o ╷ bcf87687 Aug 02 at 14:21 v5.8
╷ ╷
o ╷ 92ed3019 Jul 26 at 14:14 v5.8-rc7
╷ ╷
o ╷ ba47d845 Jul 19 at 15:41 v5.8-rc6
╷ ╷
o ╷ 11ba4688 Jul 12 at 16:34 v5.8-rc5
╷ ╷
o ╷ dcb7fd82 Jul 05 at 16:20 v5.8-rc4
╷ ╷
o ╷ 9ebcfadb Jun 28 at 15:00 v5.8-rc3
╷ ╷
o ╷ 48778464 Jun 21 at 15:45 v5.8-rc2
├─╯
o b3a9e3b9 Jun 14 at 12:45 v5.8-rc1
╷
o 3d77e6a8 May 31 at 16:49 v5.7
╷
o 9cb1fd0e May 24 at 15:32 v5.7-rc7
╷
o b9bbe6ed May 17 at 16:48 v5.7-rc6
╷
o 2ef96a5b May 10 at 15:16 v5.7-rc5
╷
o 0e698dfa May 03 at 14:56 v5.7-rc4
╷
o 6a8b55ed Apr 26 at 13:51 v5.7-rc3
╷
o ae83d0b4 Apr 19 at 14:35 v5.7-rc2
╷
o 8f3d9f35 Apr 12 at 12:35 v5.7-rc1
╷
o 7111951b Mar 29 at 15:25 v5.6
╷
o 16fbf79b Mar 22 at 18:31 v5.6-rc7
╷
o fb33c651 Mar 15 at 15:01 v5.6-rc6
╷
o 2c523b34 Mar 08 at 17:44 v5.6-rc5
╷
o 98d54f81 Mar 01 at 14:38 v5.6-rc4
╷
o f8788d86 Feb 23 at 16:17 v5.6-rc3
....
Under the hood, the difference is how `reachableroots` gets calculated.
See also D22657197 (a5c36fd0b1) and D22368827 (da42f2c17e).
Since the old behavior almost always seems confusing to human. The new
config is turned on by default (but only takes effect if the "segments"
backend is used).
Reviewed By: sfilipco
Differential Revision: D23095468
fbshipit-source-id: f0fc631d2d9a00e3b36744e4236b43d230d10687
Summary:
The files in commit cloud `References` structures are optional. Handle them
not being present.
Reviewed By: quark-zju
Differential Revision: D23266786
fbshipit-source-id: ed7128bc7e6b762d3509d77b40a00b77885191b9
Summary: This makes it a bit easier to track down perf issues printed by RUST_LOGs.
Reviewed By: sfilipco
Differential Revision: D23095463
fbshipit-source-id: 78221a1992389f512fac6e6e633be6d19123e04a
Summary:
Use `git config core.autocrlf false` to silent warnings like:
```
$ git add alpha
+ warning: LF will be replaced by CRLF in alpha.
+ The file will have its original line endings in your working directory
```
Reviewed By: sfilipco
Differential Revision: D23270146
fbshipit-source-id: af3bf241edb9f615bcc285b51cc491385f208039
Summary: The command is needed to restore a deleted workspace
Reviewed By: markbt
Differential Revision: D23250376
fbshipit-source-id: e24a7cbc0aad004291853b4c34d7474789aa9c2b
Summary:
The fuzz tests need `TestContext::id_dag()`, which was removed by D20471712 (1fb5acf242).
Restore it so fuzz tests can run. This is mainly to check the new `range`
function.
The `range` fuzz test does find an issue caused by `>` written as `>=`
relatively quickly.
Reviewed By: sfilipco
Differential Revision: D23106176
fbshipit-source-id: e9540cc932503a9d54246d24c70bac829fcb13df
Summary: Ensure that the commit text is verified, but do not verify git hashes.
Reviewed By: DurhamG
Differential Revision: D23095464
fbshipit-source-id: e62341f6c7258c6f18b7cc75088c25dfc7040ab1
Summary:
The immediate goal is to run benchmarks on a commit graph provided by a git
repo without converting a whole (large) repo from git to hg. Note git repos can
be cloned in a shallow way so it only contains the commit graph. For example:
git clone https://github.com/torvalds/linux --filter=tree:0 -n
Note: The above command writes repositoryformat=1 in `.git/config`
which is not supported by libgit2. Manually editing it to repositoryformat=0
would enable libgit2 to read it for this crate's use-case.
In the longer term we might want to extend the support so refs/trees/files can
be read/written directly via the git repo based on this work. However that's
currently beyond scope.
Reviewed By: DurhamG
Differential Revision: D23095467
fbshipit-source-id: 868beb0c7de60453b47962639863eb8f7e3f5753
Summary: Migrate to concrete types so it can be typechecked.
Reviewed By: DurhamG
Differential Revision: D23095469
fbshipit-source-id: 27c6da30ca8a1329df544cd2ded7d9734593e48a
Summary:
Read git commit graph and migrate them to `dag::Dag`.
This allows using Rust dag abstractions on the git
commit graph.
Reviewed By: DurhamG
Differential Revision: D23095471
fbshipit-source-id: 2163701350ce82ce6e97074e56ad5877f3c9c158
Summary:
Add alternative paths will be faster if changelog2 is used, since they are
backed by native paths.
Add a config option to disable the fast paths if they cause issues.
Reviewed By: DurhamG
Differential Revision: D23036074
fbshipit-source-id: 489b6eac64148867c209d595623d0b9c21ad1d5a
Summary:
OSX doesn't support touch -d. Let's just skip that part of the test on
that platform. This fixes the OSX build.
Reviewed By: singhsrb
Differential Revision: D23253475
fbshipit-source-id: 0eccb884cbdd4bf0a4068fbf943ba7dac9df4e04
Summary:
Detect the "segments" backend and calculate the revset differently.
Practically, with collapse-obsolete disabled, the time of related revset
calculation drops from 0.14s to 0.03s in my fbsource repo.
The `obsolete()` set calculation is expensive (0.4-0.6s) and a bit more
expensive with the new DAG APIs, which will be addressed in upcoming
changes. EDIT: Addressed by D23036063.
Reviewed By: DurhamG
Differential Revision: D23036055
fbshipit-source-id: 71140a88599cc68bfa90d564c786da89b3ebd38b
Summary:
The `compact` template is rarely used and is coupled with rev numbers (ex. rev
number decides what "parents" to show). Use explicit templates. This makes the
test change easier to check.
Reviewed By: DurhamG
Differential Revision: D23036076
fbshipit-source-id: f2cc0f25191711fa7d846a8ad38aee8fb9171273
Summary:
The `notbackedup()` revset is used as part of `summary` that prints information
at the end of `smartlog`. It can take hundreds of milliseconds if there are
many heads. Detect segmented changelog and use a fast path for it.
Practically this reduces `summary` from 594ms to 91ms for me:
With segmented changelog (doublewrite backend) and new code path:
91 \ summary status.py:23
2 \ currentworkspace workspace.py:121
3 | _get (2 times) workspace.py:110
3 | read (2 times) config.py:195
3 | parse (2 times) config.py:116
2 | compile (14 times) util.py:1464
3 \ __init__ syncstate.py:44
82 \ revs localrepo.py:1203
With revlog and old code path:
594 \ summary status.py:23
2 \ currentworkspace workspace.py:121
4 | _get (2 times) workspace.py:110
3 | read (2 times) config.py:195
3 | parse (2 times) config.py:116
3 | compile (14 times) util.py:1464
3 \ __init__ syncstate.py:44
46 \ revs localrepo.py:1203
539 \ _iterfilter smartset.py:647
538 | <lambda> (1565 times) commitcloud/__init__.py:371
537 | __contains__ (1565 times) smartset.py:1039
533 | _consumegen (17355 times) smartset.py:1122
Reviewed By: markbt
Differential Revision: D23036075
fbshipit-source-id: 09dcc34f34a42814c6526e558d40b4d75ba9d75f
Summary: Expose the Rust API so `getdag` can choose to skip successors or predecessors.
Reviewed By: markbt
Differential Revision: D23036056
fbshipit-source-id: 30cd437c5420d2d10176e33ef9de98814046f4ce
Summary:
The new path does not calculate the complicated `successorssets`, and is
known to make wez's repo operations significantly faster (which, I suspect is
slowed by a very long chain).
The new code is about 3x faster on my repo too:
# before
In [1]: list(repo.nodes('draft()'))
In [2]: %time len(m.mutation.obsoletenodes(repo))
CPU times: user 246 ms, sys: 42.2 ms, total: 288 ms
Wall time: 316 ms
Out[2]: 1127
# after
In [1]: list(repo.nodes('draft()'))
In [2]: %time len(m.mutation.obsoletenodes(repo))
CPU times: user 74.3 ms, sys: 7.92 ms, total: 82.3 ms
Wall time: 82.3 ms
Out[2]: 1127
Reviewed By: markbt
Differential Revision: D23036063
fbshipit-source-id: afd6ac122bb5d8d513b5cdc033e04d2c377286eb
Summary:
Optimize get_dag:
- Avoid parsing mutation entries once they are parsed, by keeping an in-memory
`parent_map`.
- Pass `heads` to `add_heads` so the segments are less fragmented, cycle break
helper is more efficient.
The `heads` optimization is effective. Practically this makes `get_dag` about 2x faster.
This has a subtle change on cycle handling - full cycle without any non-cycle heads will
be ignored. Practically cycles are rare so it might be okay.
Together with improvements on the `dag` side, `get_dag` is about 4x faster.
Reviewed By: markbt
Differential Revision: D23036062
fbshipit-source-id: 3dc407b562f7ebf2543a87c5cd651ad6a2339d67
Summary:
If there is no new master segments, it's still possible to have new non-master
segments. Fix the loop condition so we don't skip building non-master segments.
Reviewed By: sfilipco
Differential Revision: D23095465
fbshipit-source-id: 46eb9d5b5f2b04241981558646e0bc090652abce
Summary:
I noticed that high-level segments are somehow not built for non-master vertexes.
Add a test to demonstrate the issue.
Reviewed By: DurhamG, sfilipco
Differential Revision: D23095466
fbshipit-source-id: c5a6da14bdfabcf7c432f6c6dfe096c71cc10ee9
Summary: This is useful to investigate internals of dag calculations.
Reviewed By: sfilipco
Differential Revision: D23095473
fbshipit-source-id: 4750c1b4ffad32b1317051d17db9659aaaed59c4
Summary:
Follow up of the previous change by actually using the flat segments to build
segments. This significantly improved the perf. `cargo bench --bench dag_ops`
shows:
building segments (old) 774.109 ms
building segments (new) 143.879 ms
Besides, a `O(N^2)` update to `head_ids` is changed. It improves performance
when the graph has many heads (ex. the mutation graph).
Reviewed By: sfilipco
Differential Revision: D23036080
fbshipit-source-id: 033565700f253c6f20e30a00adb6b579921d6679
Summary:
While testing the `obsolete()` set, I found an in-memory segmented DAG takes
10x time to build than a HashMap DAG.
Part of the inefficiency is to use a translated "parent_func" that round-trips
through Id and Vertex, used by segment building logic. This diff makes
`IdMap::assign_head` return flat segments, so we don't need a translated
"parent_func" to build flat segments.
This diff only adds checks to make sure the parent_func (Id version) matches
the segments. The next diff switches the segment building to not use the
translated parent_func.
Reviewed By: sfilipco
Differential Revision: D23036060
fbshipit-source-id: 99137f4b5be455cdf43218ba23eb3954b6d9e05a
Summary:
This affects the `tonodes` API in the Python world. Practically this will bind
the main commit graph to sets like draft, public.
The `ToSet` requirement on `DagAlgorithm` has to be removed to avoid stack
overflow of rustc resolving constraints.
Reviewed By: sfilipco
Differential Revision: D23036077
fbshipit-source-id: 912b924e29611680ab6b2ee4dbcd7ab39824409a
Summary: This will be useful for the `obsolete()` set.
Reviewed By: sfilipco
Differential Revision: D23036072
fbshipit-source-id: 2f944ef31cf19f902622d90545fa02b7dda89221
Summary:
If two sets have different IdMap, their Ids cannot be compared directly
for correctness.
Reviewed By: sfilipco
Differential Revision: D23036068
fbshipit-source-id: e800e8273b95c1f8174236e0f30445db7fd44556
Summary: This is similar to the previous change. This allows "binding" IdMaps to sets.
Reviewed By: sfilipco
Differential Revision: D23036058
fbshipit-source-id: ec1b1ec73e949ad4865aecf17bfcc5c1ca723e0d
Summary:
This trades a bit performance (calculating the snapshot) for correctness (no
pointer reuse issues) and convenience (set captures dag information with them
and enables use-cases like converting NameSet from another dag to the
current dag without requiring extra `dag` objects).
Reviewed By: sfilipco
Differential Revision: D23036067
fbshipit-source-id: 2e691f09ad401ba79dbc635e908d79e54dadca5e
Summary:
If `x` and `y` come from a same graph, `x & y` is more efficient than
`y & x` if `y` is larger. However, if `x` and `y` are from different
graphs, the `FULL` hint can no longer accurately predict which one
is larger. Therefore the swap should be avoided.
Reviewed By: sfilipco
Differential Revision: D23036081
fbshipit-source-id: fe3970fc38c853b36689bfd0ee1dec20643ace78
Summary:
For sets like `obsolete()`, `merge()`, they could have a fast "contains" path:
Just check the given commit without calculating a full set. It's also possible
to have a relatively efficient code path to return StaticSet (for obsolete()),
or IdStaticSet (for merge(), by checking flat segments). This diff adds a
`MetaSet` that allows defining two fast paths separately.
This will be used for the `obsolete()` set in upcoming changes.
Reviewed By: sfilipco
Differential Revision: D23036059
fbshipit-source-id: 06e6f90e7e9511626a12cfa729c306ff539256d2
Summary:
Before this change, `flush` with empty changes but `master` moves will cause an
error, because the `parents_func` only contains "pending changes", aka. new
vertexes. The `parents_func` does not know `master` and `master` is needed to
re-assign them from the non-master to the master group.
With the snapshot API, things become easier. We just take a snapshot before
reloading, and use the snapshot to answer parent_names.
Reviewed By: sfilipco
Differential Revision: D22970569
fbshipit-source-id: 99a25857ba98792edff69985c16df118a560ffb0
Summary:
This API allows the underlying Dag to provide a snapshot. The snapshot can then
be used in places that do not want a lifetime (ex. NameSet).
Reviewed By: sfilipco
Differential Revision: D22970579
fbshipit-source-id: ededff82009fd5b4583f871eef084ec907b45d33
Summary:
Make it possible to snapshot a Dag. This is useful for cases where another
struct wants access to the Dag without lifetimes. Namely, the LazySet can
might want to keep a snapshot of Dag.
Reviewed By: sfilipco
Differential Revision: D22970568
fbshipit-source-id: 508c38d3ffac2ffcd2e682578c3c5e5787ea3bcf
Summary:
The only intended use of the inverse DAG is to implement the Python dag
interface in `dagutil.py`. D22519589 (2d4d44cf3d) stack changed it so the Python dag
interface becomes optional. Therefore there is no need to keep the inverse DAG
interface, which is a bit tricky on sorting.
Reviewed By: sfilipco
Differential Revision: D22970581
fbshipit-source-id: 58a126b41d992e75beaf76ece25cb578ee84760b
Summary:
The nameset serves as a bridge for Rust NameSet sets. It's different from the
Rust IdSet in a way that it supports all kinds of Rust NameSet (lazy or
non-lazy).
Unlike the native Rust binding, the added nameset uses rev numbers and fit in
the Python smartset framework.
Reviewed By: sfilipco
Differential Revision: D23036066
fbshipit-source-id: 060b3927dda6cd2275af21b093729c7e0e88ee7c
Summary: The Rust "flush(masternodes)" API does not handle nullid. Filter it out from Python.
Reviewed By: sfilipco
Differential Revision: D22970578
fbshipit-source-id: 671fe950948067a0b3f97c5b65ff2b9b7ed4b631
Summary:
By default, `torevs` calls Python iteration for non-list, non-spans Python
objects. The `idset` object has the `spans` which can be used as a fast
path.
Reviewed By: DurhamG
Differential Revision: D22970580
fbshipit-source-id: f491404ba803c4468c17cd74daaea90f46b8b38b
Summary:
This allows certain code paths to use `dageval` with the idea that `dageval` is
going to be faster.
Reviewed By: sfilipco
Differential Revision: D22970576
fbshipit-source-id: ba4536a55691de63640e574c898320629c6d7b2f
Summary: This allows migrating between a few changelog formats we have.
Reviewed By: DurhamG
Differential Revision: D22970571
fbshipit-source-id: d6b577ae5beb72a43fff999c26c35fcdc33e8f83
Summary:
This will be used for migrating revlog DAG to segmented changelog. It does not
migrate commit text data (which can take 10+ minutes).
Reviewed By: DurhamG, sfilipco
Differential Revision: D22970582
fbshipit-source-id: 125a8726d48e15ceb06edb139d6d5b2fc132a32c
Summary: For now it just prints some details about the changelog backend.
Reviewed By: DurhamG, sfilipco
Differential Revision: D22970573
fbshipit-source-id: 719a5e5bb6f3856df3c9357e47daa9e7c8584952
Summary:
This option is needed to validate Mononoke Smartlog against the original
infinitepush Commit Cloud Smartlog. This option is advanced and can be removed
after full migration to the Mononoke backend.
Reviewed By: markbt
Differential Revision: D23241251
fbshipit-source-id: e550334b104d18bb58d39acb8540ebdc9e711c4e
Summary:
We've been using a hard coded list to determine whether
mononoke is available, and that list is falling behind the current
state of our backend migration.
This commit removes the hard coded list in favor of testing
`remotefile.reponame`. If that is configured then it holds
the mononoke reponame.
I'm making the assumption that it being set implies that mononoke
is available for that repo. That may not be 100% true, but
it appears to be effectively true for the intersection of repos
known to `fbclone` and the migrated set of repos.
If this code decides that mononoke is supported, the behavior
in EdenFS is to then attempt to use the SCS to fetch tree
data using the returned `repo.name`. That appears to be the
only way that this information is used today.
Reviewed By: quark-zju
Differential Revision: D23214471
fbshipit-source-id: 17b6475b891df5423dca0c18ddae0838795f713a
Summary:
Dynamicconfigs compares the timestamp of config files with the current
timestamp to determine when to regenerate. If the timestamp of the config file
is newer than the current timestamp, Rust throws an exception. Let's handle that
case and treat it as if the file was just created instead of crashing.
Reviewed By: quark-zju
Differential Revision: D23230216
fbshipit-source-id: ca185de7dfca46953e04ec08c84668eda6d749bd
Summary:
If a cloud sync requires both fetching from the cloud and uploading new state to the cloud,
commit the transaction between the two steps, so that a successful cloud fetch is not
rolled back by failure to send to the cloud.
While we're here, limit the number of sync attempts to 3 in one go.
Reviewed By: farnz
Differential Revision: D23211846
fbshipit-source-id: fa97165d94eee973284ff3d00466387b3041306c
Summary: Avoid trying to send local obsmarkers if evolution is not enabled.
Reviewed By: farnz
Differential Revision: D23210800
fbshipit-source-id: dc247a18e92f6f5454eeed520854dd6254f66257
Summary:
Large numbers of pending obsmarkers can cause commit cloud `update_references`
requests to fail.
Prevent this from happening by only syncing the most recent obsmarkers.
Obsmarkers are in the process of being deprecated and removed, so this
shouldn't be a problem for long.
Reviewed By: farnz
Differential Revision: D23211621
fbshipit-source-id: 56a2e103722c3c162eacdb62638ff8ff614d5815
Summary: This fixes the Windows build.
Reviewed By: farnz
Differential Revision: D23212195
fbshipit-source-id: 159f3ddebf6a97f52f9b6c80ef19315c8f4b0c85
Summary: Update bindings to expose the DoubleWrite backend and the DescribeBackend API.
Reviewed By: sfilipco
Differential Revision: D22970574
fbshipit-source-id: bdb52ff21dd0b9ffa0be214b4a4824025f460092
Summary:
This allows importing from other DAGs. It will be used to import revlog DAG to
the new segmented format.
Reviewed By: sfilipco
Differential Revision: D22970572
fbshipit-source-id: 0a183e7b64831574cc9c60d4639124d02d19cf43
Summary:
This allows dag to use renderdag in tests to verify graph result. Previously
it was hard because dag <-> renderdag would form circular dependency.
It also make it possible to implement more efficient and integrated fast paths
for graph rendering.
Reviewed By: sfilipco
Differential Revision: D22970570
fbshipit-source-id: 526497339bd7aa8898d1af4aa9cf6d2a6797aae0
Summary: This will be used to describe what the commit graph backend is.
Reviewed By: sfilipco
Differential Revision: D22970577
fbshipit-source-id: 753efdbdd4466730ece758d9f4789fbd21e2801b
Summary:
This allows us to try segmented changelog while maintaining revlog
compatibility.
Reviewed By: sfilipco
Differential Revision: D22970583
fbshipit-source-id: 7c43cdadd76300e76e89f38aac5ed3ecc0cff728
Summary: The value of linkrev is invalid with segmented changelog. Do not resolve it.
Reviewed By: sfilipco
Differential Revision: D23036079
fbshipit-source-id: 8f8b097458bc694327db6ba4e2dc4107bdf44157
Summary:
Phabricator reports some repos as being backed by git, and the phrevset
code path in that case uses the python 2 hex encoding. Let's just use the hex
function directly.
Reviewed By: quark-zju
Differential Revision: D23207771
fbshipit-source-id: 88de3153e52a3db456c17ab4ca4b9c9dd6049855
Summary: Rather than using a Python program, simulate having no editor during tests using `internal:none`.
Reviewed By: quark-zju, ikostia
Differential Revision: D23152446
fbshipit-source-id: 5560f58885ee5959c62f0ac8bcf0483b9c3072f6
Summary:
The commandserver was setting args as bytes when they should be
strings. This was breaking arg parsing for users using hglib, which communicates
with hg via the commandserver.
Reviewed By: sfilipco, singhsrb
Differential Revision: D23202623
fbshipit-source-id: f71e4145211069b2f7ed6935fe86585061cbf8b3
Summary:
The old discovery logic is incompatible with the rust changelog because
the changelog is not based on revlog.
Reviewed By: sfilipco
Differential Revision: D23036065
fbshipit-source-id: 633fdf8726d40cb14e63c3df2f5573d35cb1640a
Summary:
They are faster if the new dag backend is used. For example, `headsancestors`
will be backed by a fast native path, where the `head(ancestors(x))` revset
would be unusably slow if the new dag backend is used.
Reviewed By: sfilipco
Differential Revision: D23036069
fbshipit-source-id: 75df1e2240520a6e560bc5d8414ba2f4fb7a4674
Summary:
It read the sharedpath manually but needed to read it as utf8, not
bytes.
Reviewed By: quark-zju
Differential Revision: D23194858
fbshipit-source-id: aa73c2cc782070ba6c4d2c441d6f5338842b137f
