Summary:
Now that namedag recovers nicely if the IdMap has more assigned ids than the IdDag, use it to build segmented changelogs.
This removes one source of divergence between client and Mononoke.
Reviewed By: quark-zju
Differential Revision: D32783287
fbshipit-source-id: 94a030b94e810d406102db92bab249dc76675dac
Summary:
For invocations of `ensure!` where the argument is one of the comparison operators `==` `<=` `<` `!=` `>=` `>` and the lhs and rhs of the operator both have a `Debug` impl, we now render both sides as part of the error message.
### Example:
```
ensure!(flags.len() <= 40);
```
```
ensure!(kind == Kind::File);
```
Before:
```
Condition failed: `flags.len() <= 40`
Condition failed: `kind == Kind::File`
```
After:
```
Condition failed: `flags.len() <= 40` (99 vs 40)
Condition failed: `kind == Kind::File` (Symlink vs File)
```
Reviewed By: zertosh
Differential Revision: D32598267
fbshipit-source-id: 70fdab6d8f0e3ceceb98e651eab0a73795cadaeb
Summary:
This release helps catch places that the `anyhow!` or `bail!` macros are invoked with a missing fmt argument, such as:
```
- anyhow!("unsupported redirect operation {:?}")
+ anyhow!("unsupported redirect operation {:?}", op)
```
All instances of this in fbcode were fixed in D32266567 (2f440eac7e).
Reviewed By: wqfish
Differential Revision: D32564287
fbshipit-source-id: e9cf5a1376e6c83343de3f3c6ba2925d88334a30
Summary:
Previously, only `add_heads_and_flush` can add vertexes in the master group.
The `flush` logic is a bit unfriendly for Mononoke use-case since it also requires
`open` and locking. Those might not apply to Mononoke, or at least add extra
overhead.
Update the `add_heads` API to take `VertexListWithOptions` to make it a step
closer to `add_heads_and_flush`. The goal is to make Mononoke use the `add_heads`
API while keeping its own logic about how to write to the shared storage.
Note: This diff only changes the function signature. It does not update the
actual logic yet.
Reviewed By: DurhamG
Differential Revision: D32499010
fbshipit-source-id: 4e74fb32484de5e387ddbdf29f8a6594ec269b6f
Summary:
It could be confusing that `iter()` emits Ids in DESC order.
Rename it to clarify.
Reviewed By: DurhamG
Differential Revision: D32227945
fbshipit-source-id: 0891fc19a4d027d2940e76160207d4eef90840c5
Summary:
The 2021 edition has some new traits in the prelude, which means they no longer require an explicit import in every module that wants to use them.
Details: https://doc.rust-lang.org/edition-guide/rust-2021/prelude.html
Reviewed By: metajack
Differential Revision: D31823254
fbshipit-source-id: 7532a198347b032f0cf9277c916d54db268581ba
Summary: Generated by `sed -i 's/edition = "2018"/edition = "2021"/' common/rust/cargo_from_buck/project_configs/*.toml` and `cargo autocargo` following the autocargo code change in D31830985 and its corresponding MSDK rebuild in D31832311.
Reviewed By: zertosh
Differential Revision: D31832698
fbshipit-source-id: db79024cad311467d5b7c13b7abedc6464f09366
Summary:
See previous diff for context. The reorders the `use` statements to make
things a bit more consistent.
Reviewed By: yancouto
Differential Revision: D31751166
fbshipit-source-id: 09c4ddba69ddeae1fc1b996d93fbfa6063e3407d
Summary:
The current rustfmt we use cannot handle `#[cfg(...)] use ...::{...}` properly.
Let's format the files that do not have `#[cfg] use` or are generated first.
The change is done by Ruby code:
list = Dir['{lib,edenscmnative}/**/*.rs'].select {|p|!File.read(p)["generated"]}
system 'rustfmt', *list
$fn = nil
reverts = `hg diff`.lines.map do |l|
$fn = l[/diff --git a\/fbcode\/eden\/scm\/([^ ]*) b\//, 1] || $fn
$fn if l[/-\s*#\[cfg/]
end.compact.uniq
system 'hg', 'revert', *reverts
Reviewed By: yancouto
Differential Revision: D31751037
fbshipit-source-id: dbe0b39e35345b7d1c54cbdf67b8aafd19df9c70
Summary: Bump in preparation for a new release
Reviewed By: brianc118
Differential Revision: D31657479
fbshipit-source-id: eeb2a08c54f6db75b6e515ca0966637f31b906a8
Summary:
Add back the octopus merge support for revlog.
This recommits D30686451 (b13579fdf9) and D30686450 (7eb11cb392) as-is, with updates to test files.
Original commit changeset: 9f213766e7c4
Reviewed By: StanislavGlebik
Differential Revision: D30784681
fbshipit-source-id: ace0c317652ad8b657c8edd9a0130532dad53078
Summary:
Support octopus merge defined in the following format:
- The revlog flag has `1 << 12` set.
- Extra `stepparents` is set to `hexnode1,hexnode2,...` format.
This is mainly used to support revlog from stream clone.
Reviewed By: StanislavGlebik
Differential Revision: D30686450
fbshipit-source-id: d5aa2f18a02f5f0d7aa033210fb4f79b729c0d26
Summary:
Extend the struct so we can support more than 2 parents.
The size of the sturct is now 16 bytes, from 8 bytes. This might have some
performance overhead.
Not using `Box<[u8]>` because that will make the struct 24 bytes.
Reviewed By: StanislavGlebik
Differential Revision: D30686451
fbshipit-source-id: c0f8d0472c7e578f34d771dacecffc91585650c3
Summary:
Makes the revlog index provides dummy graph integrity checks so it can
be used as a generic object in the Python bindings.
Reviewed By: andll
Differential Revision: D30682542
fbshipit-source-id: 25c6e8640de46188d7bf45a927e11e0779a8ad40
Summary:
The use of dyn traits of the Thrift-generated server traits was emitting future compatibility warnings with recent versions of rustc, due to a fixed soundness hole in the trait object system:
```
error: the trait `x_account_aggregator_if::server::XAccountAggregator` cannot be made into an object
|
= this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
note: for a trait to be "object safe" it needs to allow building a vtable to allow the call to be resolvable dynamically; for more information visit <https://doc.rust-lang.org/reference/items/traits.html#object-safety>
```
This diff pulls in https://github.com/dtolnay/async-trait/releases/tag/0.1.51 which results in the Thrift-generated server traits no longer hitting the problematic pattern.
Reviewed By: zertosh
Differential Revision: D29979939
fbshipit-source-id: 3e6e976181bfcf35ed453ae681baeb76a634ddda
Summary: Will be used by the next change.
Reviewed By: andll
Differential Revision: D29434722
fbshipit-source-id: 74dbec506fb0985379480815380118cd41058aec
Summary: This will be used by upcoming changes.
Reviewed By: sfilipco
Differential Revision: D27629312
fbshipit-source-id: 6c56e73caf4e1a398ac3a8b4294bd9f380af3764
Summary:
This will be used by "add_heads" logic to detect what vertexes to insert
and might reduce remote fetches.
Reviewed By: sfilipco
Differential Revision: D27572359
fbshipit-source-id: d0bf027a69d180663a1587dfde613cb76b05072a
Summary: This will be used by upcoming changes.
Reviewed By: sfilipco
Differential Revision: D27343884
fbshipit-source-id: 0938b1fb3d90b35f9d51c468cffca53e3f421bb8
Summary:
AsyncVfs provides async vfs interface.
It will be used in the native checkout instead of current use case that spawns blocking tokio tasks for VFS action
Reviewed By: quark-zju
Differential Revision: D26801250
fbshipit-source-id: bb26c4fc8acac82f4b55bb3f2f3964a6d0b64014
Summary: This diff removes the split between manually managed and autocargo managed Cargo.toml files in `eden/scm/lib`, now all files are autogenerated.
Reviewed By: quark-zju
Differential Revision: D26830884
fbshipit-source-id: 3a5d8409a61347c7650cc7d8192fa426c03733dc
Summary:
Previously, `MetaSet` was constructed with default `Hints`. That disables fast
paths. Revise the API so MetaSet requires an explicit `Hints` to address the
issue.
Reviewed By: sfilipco
Differential Revision: D26203557
fbshipit-source-id: 9e7658af8723b06d0efdcad1ab4671c79e907326
Summary:
This just renames types so `IdSet` is the recommended name and `SpanSet`
remains an implementation detail.
Reviewed By: sfilipco
Differential Revision: D26203560
fbshipit-source-id: 7ca0262f3ad6d874363c73445f40f8c5bf3dc40e
Summary: In upcoming changes, we're moving Id to a separate crate. This makes that easier.
Reviewed By: sfilipco
Differential Revision: D25857918
fbshipit-source-id: 6e2163f6fa171d4cd3be4fc0c4c248fd87ba739c
Summary:
Similar to the previous change. `VerLink` tracks compatibility more accurately.
- No false positives comparing to the current `map_id` approach.
- Less false negatives comparing to the previous `Arc::ptr_eq` approach.
The `map_id` is kept for debugging purpose.
Reviewed By: sfilipco
Differential Revision: D25607513
fbshipit-source-id: 7d7c7e3d49f707a584142aaaf0a98cfd3a9b5fe8
Summary:
`VerLink` tracks compatibility more accurately.
- No false positives comparing to the current `dag_id` approach.
- Less false negatives comparing to the previous `Arc::ptr_eq` approach.
The `dag_id` is kept for debugging purpose.
Note: By the current implementation, `dag.flush()` will make `dag`
incompatible from its previous state. This is somewhat expected, as
`flush` might pick up any changes on the filesystem, reassign non-master. Those
can be actually incompatible. This might be improved in the future to detect
reload changes by using some extra information.
Reviewed By: sfilipco
Differential Revision: D25607511
fbshipit-source-id: 3cfc97610504813a3e5bb32ec19a90495551fd3a
Summary:
See the previous diff for context. The new API will be used to check if two
dags are compatible.
Note: It can cause false positive on compatibility checks, which need a
more complex solution. See D25607513 in this stack.
Reviewed By: sfilipco
Differential Revision: D25598079
fbshipit-source-id: f5fc9c03d73b42fadb931038fe2e078881be955f
Summary:
It turns out `Arc::ptr_eq` is becoming unreliable, which will cause fast paths
to be not used, and extreme slowness in some cases (ex. `public & nodes`
iterating everything in `public`).
This diff adds an API for an IdMap to tell us its identity. That identity is
then used to replace the unreliable `Arc::ptr_eq`.
For an in-memory map, we just assign a unique number (per process) for its
identity on initialization. For an on-disk map, we use the type + path to
represent it.
Note: strictly speaking, this could cause false positives about
"maps are compatible", because two maps initially cloned from each other
can be mutated differently and their map_id do not change. That will
be addressed in upcoming diffs introducing a more complex but precise way to
track compatibility.
Reviewed By: sfilipco
Differential Revision: D25598076
fbshipit-source-id: 98c58f367770adaa14edcad20eeeed37420fbbaa
Summary: Make the parent function used by various graph building functions async.
Reviewed By: sfilipco
Differential Revision: D25353612
fbshipit-source-id: 31f173dc82f0cce6022cc2caae78369fdc821c8f
Summary: Make IdConvert async and migrate all its users.
Reviewed By: sfilipco
Differential Revision: D25350915
fbshipit-source-id: f05c89a43418f1180bf0ffa573ae2cdb87162c76
Summary: This will make it easier to make IdConvert async.
Reviewed By: sfilipco
Differential Revision: D25345232
fbshipit-source-id: b8967ea51a6141a95070006a289dd724522f8e18
Summary:
Update DagAlgorithm and all its users to async. This makes it easier to make
IdConvert async.
Reviewed By: sfilipco
Differential Revision: D25345236
fbshipit-source-id: d6cf76723356bd0eb81822843b2e581de1e3290a
Summary: This makes it easier to make DagAlgorithm async.
Reviewed By: sfilipco
Differential Revision: D25345234
fbshipit-source-id: 5ca4bac38f5aac4c6611146a87f423a244f1f5a2
Summary: Use async function for the PrefixLookup trait.
Reviewed By: sfilipco
Differential Revision: D24840820
fbshipit-source-id: d22cac9f11b06e3127fa956e3f116cf232214125
Summary:
Change the main API of NameSet to async. Use the `nonblocking` crate to bridge
the sync and async world for compatibility. Future changes will migrate
Iterator to async Stream.
Reviewed By: sfilipco
Differential Revision: D24806696
fbshipit-source-id: f72571407a5747a4eabe096dada288656c9d426e
Summary:
The trait requires an `IdMap` snapshot to be locally ready. That's not easy for
all possible implementations. Drop it to simplify things.
Reviewed By: sfilipco
Differential Revision: D24399501
fbshipit-source-id: 4d85f77c99208cda30b2a543a0bb5b295f49a65c
Summary:
Previously phase calculation was done via a simple ancestor check. This
was very slow in cases that required going far back into the graph. Going a year
back could take a number of seconds.
To fix it, let's take the Rust phaseset logic and rework it to make only_both
produce an incremental public nodes set. In a later diff we can switch the
phaseset function to use this as well, but right now phaseset returns IdSet, and
that would need to be changed to Set, which may have consequences. So I'll do it
later.
Reviewed By: quark-zju
Differential Revision: D24096539
fbshipit-source-id: 5730ddd45b08cc985ecd9128c25021b6e7d7bc89
Summary:
Generated by formatting with rustfmt 2.0.0-rc.2 and then a second time with fbsource's current rustfmt (1.4.14).
This results in formatting for which rustfmt 1.4 is idempotent but is closer to the style of rustfmt 2.0, reducing the amount of code that will need to change atomically in that upgrade.
---
*Why now?* **:** The 1.x branch is no longer being developed and fixes like https://github.com/rust-lang/rustfmt/issues/4159 (which we need in fbcode) only land to the 2.0 branch.
---
Reviewed By: zertosh
Differential Revision: D23568779
fbshipit-source-id: 477200f35b280a4f6471d8e574e37e5f57917baf
Summary:
Both IdSet and IdLazy set require both Dag and IdMap to construct.
This is step 1 torwards making Dag and IdMap immutable in hints.
A misspeall of "lhs" vs "hints" in the union set is discovered by the change
and fixed.
Reviewed By: sfilipco
Differential Revision: D23182520
fbshipit-source-id: 3d052de4b8681d3672ebc45d953d1e784f64b2a4
Summary: This is more consistent with `id_map_snapshot`.
Reviewed By: sfilipco
Differential Revision: D23182519
fbshipit-source-id: 62b7fc8bfdc9d6b3a4639a6518ea084c7f3807dd
Summary:
This trades a bit performance (calculating the snapshot) for correctness (no
pointer reuse issues) and convenience (set captures dag information with them
and enables use-cases like converting NameSet from another dag to the
current dag without requiring extra `dag` objects).
Reviewed By: sfilipco
Differential Revision: D23036067
fbshipit-source-id: 2e691f09ad401ba79dbc635e908d79e54dadca5e
Summary:
This API allows the underlying Dag to provide a snapshot. The snapshot can then
be used in places that do not want a lifetime (ex. NameSet).
Reviewed By: sfilipco
Differential Revision: D22970579
fbshipit-source-id: ededff82009fd5b4583f871eef084ec907b45d33
Summary: If parent_revs gets an out-of-bound rev, it should fail.
Reviewed By: sfilipco
Differential Revision: D23036071
fbshipit-source-id: 7fae0fd5adf07ac3c933a29d7d06289d8d740c60
Summary:
If the text starts with `\0`, the `\0` should be considered as part of the
uncompressed text instead of a separated header.
Reviewed By: sfilipco
Differential Revision: D22970575
fbshipit-source-id: 49e8a1a1ea42a3c4cf153b70f59fd0558dcfcede
Summary:
The parent handling is unsound when there are revs that are skipped. Fix it by
reasoning about commit hashes for parents.
Reviewed By: sfilipco
Differential Revision: D23036078
fbshipit-source-id: 8f710171471025cd48b3bd8f6ea57c68330eb8b8
