Summary: This would be used to avoid excessive memory usage during pull.
Reviewed By: DurhamG
Differential Revision: D23408833
fbshipit-source-id: 8edd95ab8201697074f65cc118d14755a230567d
Summary:
Mergedrivers can call dirstate.add directly and are adding paths with
"." and "..". Let's block those paths.
Reviewed By: quark-zju
Differential Revision: D23375469
fbshipit-source-id: 64e9f20169cfd50325ecd8ebcc1dd3be7a5cb202
Summary: Similar to D18528858 so module names do not need to be spelled twice.
Reviewed By: markbt
Differential Revision: D23091380
fbshipit-source-id: a2a261abc9c78c8805cea62b38498ba65398796d
Summary: This crate would fail to build without the "fb" feature because `serde_json` was listed as an optional dependency (but is used in a way that isn't conditional on the `fb` feature). This diff makes the dependency non-optional, and also silences several dead code warnings that are emitted when building without the "fb" feature.
Reviewed By: quark-zju
Differential Revision: D23386786
fbshipit-source-id: b00a8b0b8b0b978c1cfab2838629fcb388a076e9
Summary:
The `debugfsync` command calls fsync on newly modified files in svfs.
Right now it only includes locations that we know have constant number
of files.
The fsync logic is put in a separate crate to avoid slow compiles.
Reviewed By: DurhamG
Differential Revision: D23124169
fbshipit-source-id: 438296002eed14db599d6ec225183bf824096940
Summary:
Some functions might be called very frequently. For example,
`phases.phasecache.loadphaserevs` might be called 100k+ times.
That makes the tracing data harder to process.
Limit the count of spans to 1k by default so the data is cheaper to process,
and some highly repetitive cases can now be reasoned about. Note the limit
is only put on static Span Ids. If a span uses dynamic metadata or ask for
different Span Ids each time, they will not be limited.
In debugshell,
td = %trace repo.revs('smartlog()')
len(td.serialize())
dropped from 6MB to 0.87MB.
It's also possible to reason about:
td = %trace len(repo.revs('ancestors(.)'))
in debugshell (taking 30s, 98KB serialized, vs 21s without tracing), while
previously the result would be too large to show (`%trace` just hangs).
Reviewed By: DurhamG
Differential Revision: D23307793
fbshipit-source-id: 3c1e9885ce7a275c2abd8935a4e4539a4f14ce83
Summary: Set a default limit so the output won't be too long.
Reviewed By: DurhamG
Differential Revision: D23307792
fbshipit-source-id: 7e2ed99e96bbde06436a034e78f899fc2e3e03f8
Summary:
The `--trace` flag enables tracing Python modules.
For compatibility reasons, it also enables `--traceback`.
It can be used with debugshell to make `%trace` more useful.
Reviewed By: sfilipco
Differential Revision: D23278600
fbshipit-source-id: d6d0b34bd5c48111f8cd33d7df115f349b0e95b6
Summary:
Corp has a different concept of tier than prod. Let's load the corp
tier into our tier set as well.
Reviewed By: quark-zju
Differential Revision: D23354056
fbshipit-source-id: c9543b8253f042c7b1224578e0687b4bdf21738e
Summary:
Pull Request resolved: https://github.com/facebookexperimental/eden/pull/46
See https://github.com/facebookexperimental/eden/runs/1034006668:
error: unused import: `env::set_var`
--> src/lfs.rs:1539:15
|
1539 | use std::{env::set_var, str::FromStr};
| ^^^^^^^^^^^^
|
note: the lint level is defined here
--> src/lib.rs:125:9
|
125 | #![deny(warnings)]
| ^^^^^^^^
= note: `#[deny(unused_imports)]` implied by `#[deny(warnings)]`
error: unnecessary braces around method argument
--> src/lfs.rs:2439:36
|
2439 | remote.batch_upload(&objs, { move |sha256| local_lfs.blobs.get(&sha256) })?;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: remove these braces
|
note: the lint level is defined here
--> src/lib.rs:125:9
|
125 | #![deny(warnings)]
| ^^^^^^^^
= note: `#[deny(unused_braces)]` implied by `#[deny(warnings)]`
error: aborting due to 2 previous errors
error: could not compile `revisionstore`.
I dropped `#![deny(warnings)]` as I don't think warnings like the above ones
should break the build. (denying specific warnings that we care about explicitly
might be a better approach)
Reviewed By: singhsrb
Differential Revision: D23362178
fbshipit-source-id: 02258f57727edfac9818cd29dda5e451c7ca80a7
Summary: Now that it is possible to control which features are enabled on manually-managed dependencies, we can reenable autocargo for `edenapi`. See D23216925, D23327844, and D23329351 (840e6dd6f6) for context.
Reviewed By: dtolnay
Differential Revision: D23335122
fbshipit-source-id: 8ce250c3a106d2a02f457f7ed531623dd866232f
Summary:
Pull Request resolved: https://github.com/facebookexperimental/eden/pull/45
Fix referring to 'version' without proper codegen by making 'version' compile
without codegen. This fixes configparser test when version/src/lib.rs was not
generated.
Make unneeded deps without 'fb' feature optional.
This would hopefully fix the "EdenSCM Rust Libraries" GitHub workflow.
Reviewed By: DurhamG
Differential Revision: D23269864
fbshipit-source-id: f9e691fe0a75159c4530177b8a96dad47d2494a9
Summary: This makes the code simpler.
Reviewed By: sfilipco
Differential Revision: D23269866
fbshipit-source-id: 30c9e9d218378c0d6df8b822b2a81df2b38f5b01
Summary: Will be used to simplify code.
Reviewed By: sfilipco
Differential Revision: D23269859
fbshipit-source-id: bed0c4dca075ff60900025642af1d84bdd03452d
Summary:
`impl<T> Trait for T` in the current Rust makes it impossible to have
`impl<Q> Trait for Q`. Avoid using it for IdConvert and PrefixLookup.
Reviewed By: sfilipco
Differential Revision: D23269861
fbshipit-source-id: a837f3984ff4e1bd5a3983dd1642b9f064f51a36
Summary:
`impl<T> Trait for T` in the current Rust makes it impossible to have
`impl<Q> Trait for Q`. Avoid using it for DagAlgorithm.
Reviewed By: sfilipco
Differential Revision: D23269860
fbshipit-source-id: 031e75e9bf1f1eec2b9e8f36220ef8b817a143a5
Summary: LowLevelAccess is a subset of NameDagStorage. Use the latter instead.
Reviewed By: sfilipco
Differential Revision: D23269865
fbshipit-source-id: 81ebb1e986d8b02c968a9a237ad9a97d4afd54bf
Summary:
If there are too many heads, the current `descendants` algorithm would visit
all "old" heads. For example, with this graph:
head9999 (N9999)
/
Z (master)
:
: (many heads)
:/
: head2 (N2)
:/
C head1 (N1)
|/
B head0 (N0)
|/
A
`A::head9999` or `Z::head9999` will visit N0, N1, ..., N9999, because
`descendands_up_to` is provided with `max_id = N9999` and Z as a vertex in the
master group, is before N0 in non-master. The current algorithm also means
`descendands_up_to` gets linearly slower as the user uses the repo more, which
is quite undesirable.
This diff changes `descendands_up_to` to take an `ancestors` set, which is
`::head9999` in this case, and iterate non-master flat segments in it. So it
will skip N0 to N9998 directly by finding the N9999 flat segment and only use
it. The number of heads will have a smaller impact on performance.
Another slowness is `draft::draft_heads`, if there are too many `draft_heads`,
the internal calculation of `::draft_heads` can be slow. Optimize it by
limiting `draft_heads` to `draft:`. Practically this affects `y::` revset as
`y::` is translated to `y::visible_heads` and `visible_heads` can be large.
`cargo bench --bench dag_ops -- '::-master'` shows significant difference:
Before:
range (master::draft) 18.112 s
range (recent_draft::drafts) 2.594 s
After:
range (master::draft) 72.542 ms
range (recent_draft::drafts) 14.932 ms
In my fbsource checkout there were 20k+ heads. The improvement of
`master::recent_draft` (`x::y`) is pretty visible, and `y::` is also improved:
% lhg debugbenchmarkrevsets -m -x 'p1(min(7e8c86ae % master))' -Y 'draft() & 7e8c86ae' -e 'x::y' -e 'y::' --no-default
# x: 168f5228e570fb6b2ff7f851bd82413102748d84 (p1(min(7e8c86ae % master)))
# y: 7e8c86aec68ebc6e0b8254afcb381315991fd21c (draft() & 7e8c86ae)
# before
| revset \ backend | segments | revlog | revlog-cpy |
|------------------|----------|--------|------------|
| x::y | 17ms | 0.1ms | 0.5ms |
| y:: | 3.3ms | 0.7ms | 1.3ms |
# after
| revset \ backend | segments | revlog | revlog-cpy |
|------------------|----------|--------|------------|
| x::y | 0.2ms | 0.1ms | 0.6ms |
| y:: | 1.0ms | 0.7ms | 1.3ms |
Reviewed By: sfilipco
Differential Revision: D23214387
fbshipit-source-id: 4d11db84cd28f4e04e8b991cbc650c9d5781fd27
Summary:
Lots of non-master heads is not an exercised graph in the benchmarks.
Add it as it practically happens. This will be used by the next change.
Reviewed By: sfilipco
Differential Revision: D23259879
fbshipit-source-id: 7fe290d14403e42e6d135bde56e2d5c8519ae530
Summary:
Currently the fuzz test only uses the master group. Let it exercise non-master
group too.
Reviewed By: DurhamG
Differential Revision: D23214388
fbshipit-source-id: 7108a1055fbdda2b012f93c5948fb83ef3b9a96f
Summary: Provide a way to see segments.
Reviewed By: sfilipco
Differential Revision: D23196408
fbshipit-source-id: b1418f945a5a3364ac73b0f97466d973dd4b6300
Summary:
Provide a way to print out all segments with resolved names. This will be used
in a debug command.
Reviewed By: sfilipco
Differential Revision: D23196410
fbshipit-source-id: 1712bfda0271aa548699fe4a6b8603c5ec07af7f
Summary:
Use the parent-child index to answer children query quickly.
`cargo bench --bench dag_ops -- children`:
Before:
children (spans) 606.076 ms
children (1 id) 124.105 ms
After:
children (spans) 602.999 ms
children (1 id) 10.777 ms
Reviewed By: sfilipco
Differential Revision: D23196411
fbshipit-source-id: 37195d5ccaa582d35314e0000352ef477287d38c
Summary: This will be used to optimize "children(single vertex)" query.
Reviewed By: sfilipco
Differential Revision: D23196409
fbshipit-source-id: 050c0859faf83b909e3174bb7c7bd6e7725165c0
Summary:
Update the parent index to store non-master group too. To make
"remove_non_master" work, the index contains a "child group" prefix that
allows efficient range invalidation.
This will allow answering "children(single vertex)" query more efficiently.
This diff does not expose an API to query the index yet.
Reviewed By: sfilipco
Differential Revision: D23196406
fbshipit-source-id: 9137da5ffa8306bdafbcabc06b6f0d23f38dcf57
Summary:
Practically, the input of `children` is often one vertex instead of a large set.
Add a benchmark for it.
It looks like:
children (spans) 606.076 ms
children (1 id) 124.105 ms
Reviewed By: sfilipco
Differential Revision: D23196407
fbshipit-source-id: 0645b59ac846836fd061386384f6386a57661741
Summary: They can be figured out at Hints initialization time. So they don't need to be mutable.
Reviewed By: sfilipco
Differential Revision: D23182518
fbshipit-source-id: 133375fdf27a2546a50b63fb130534acdadc5938
Summary:
Both IdSet and IdLazy set require both Dag and IdMap to construct.
This is step 1 torwards making Dag and IdMap immutable in hints.
A misspeall of "lhs" vs "hints" in the union set is discovered by the change
and fixed.
Reviewed By: sfilipco
Differential Revision: D23182520
fbshipit-source-id: 3d052de4b8681d3672ebc45d953d1e784f64b2a4
Summary:
It will be used in places (ex. tests) where a Dag is required but constructing
a real Dag is troublesome.
Reviewed By: sfilipco
Differential Revision: D23182517
fbshipit-source-id: 736911365778e5071c1e0b9615090a4e960392a0
Summary: This is more consistent with `id_map_snapshot`.
Reviewed By: sfilipco
Differential Revision: D23182519
fbshipit-source-id: 62b7fc8bfdc9d6b3a4639a6518ea084c7f3807dd
Summary:
Similar to descendants, the new range algorithm avoids potentially expensive
checks about whether high-level segments can be used or not. Practically this
is overall an improvement.
`cargo bench --bench dag_ops -- range`:
Before:
range (2 ids) 115.380 ms
range (spans) 243.666 ms
After:
range (2 ids) 123.274 ms
range (spans) 23.101 ms
It is 100x faster with the range x::y benchmark added later on `git.git`.
Reviewed By: sfilipco
Differential Revision: D23106175
fbshipit-source-id: 691e0418ba2b7ad9f52ac15b5cd6088ec28d5f48
Summary:
The old algorithm tries to make use high-level segments.
However, the code to test whether a high-level segment can be used is
often too expensive for the benefit. Often, high-level segments cannot
be used most of the time and it's similar to O(flat segments).
This diff adds a simpler algorithm that just iterates through the flat
segments. It's faster in most practical cases.
`cargo bench --bench dag_ops -- descendants` shows improvements too:
Before:
descendants (small subset) 436.515 ms
After:
descendants (small subset) 33.460 ms
Reviewed By: sfilipco
Differential Revision: D23106174
fbshipit-source-id: e6101483d8539b2b1c881be2ccfd0071f122352f
Summary: This will be used by upcoming changes.
Reviewed By: sfilipco
Differential Revision: D23106177
fbshipit-source-id: 9bf183f7464c06b801be64fd938db0babd544756
Summary: This internal struct will be used by upcoming changes.
Reviewed By: sfilipco
Differential Revision: D23106172
fbshipit-source-id: 6d5b9bc1c810984814d0912100acca38a2565a63
Summary:
Our internal build infra creates a workspace and workspaces don't like
it when two crates have the same name. Eden scm had third-party rust crates that
were simple redirects to the internal location, but had the same name. This
caused breakages once these crates became part of the edenfs open source build.
Let's rename them to avoid this issue.
Reviewed By: kulshrax
Differential Revision: D23252539
fbshipit-source-id: 9ff2fa160a19c6bc54e015c71f9da7044ce659a7
Summary:
We have a thread blocking application. We have async libraries. This crate
provides common utilities for communicating between the blocking world and the
async world. It is intended to be a guide so that not all developers have to
get in depth understanding of Tokio in order to use async functions.
Reviewed By: quark-zju, xavierd
Differential Revision: D23222876
fbshipit-source-id: b9a61795bc917bfc664c9d6da95c9e5e2d506c79
Summary: The default method implementations on this trait were causing unused variable warnings. Prefix them with underscores to silence the warning and add `#![deny(warnings)]` to `revisionstore` to prevent future warnings from slipping through the cracks.
Reviewed By: singhsrb
Differential Revision: D23334309
fbshipit-source-id: d17b27ca0dd462e1613eac918fb595faa8637741
Summary:
We want to eventually get rid of system and repo configs, but for now
they should take precedence over the dynamicconfigs. Previously we relied on
validation to remove any entries from dynamicconfig that interferes with a
system rc config, but in some code paths we didn't run that validation (like if
we loaded configs purely from Rust).
Let's just make dynamicconfig load before system configs. If validation doesn't
run, we might miss the case where dynamicconfig sets a value and the system rc
doesn't. But that's probably fine.
Reviewed By: quark-zju
Differential Revision: D23305711
fbshipit-source-id: 77b5f49d348cfa116694a641ed17e6d1184a81ab
Summary:
Dynamicconfigs are now part of our critical path. Let's remove the
option to not load them. This also let's us get rid of a circularl dependency
where loading dynamicconfigs required having already loaded some configs. This
will let us move dynammicconfig loading to be before system rc loading in a
later diff.
Reviewed By: sfilipco
Differential Revision: D23309090
fbshipit-source-id: 5138059a8ed944c3616007e7c1289b6a57be0e65
Summary:
Dynamicconfig was throwing errors if hgrc.dynamic wasn't writable.
Let's eat those errors for normal read operations. We still treat it as an error
for straight hg debugdynamicconfig invocations.
Reviewed By: quark-zju
Differential Revision: D23301100
fbshipit-source-id: ed0bd1282d2c7ee747f0909c238a5fa07b7bc9bc
Summary: This makes it a bit easier to track down perf issues printed by RUST_LOGs.
Reviewed By: sfilipco
Differential Revision: D23095463
fbshipit-source-id: 78221a1992389f512fac6e6e633be6d19123e04a
Summary:
The fuzz tests need `TestContext::id_dag()`, which was removed by D20471712 (1fb5acf242).
Restore it so fuzz tests can run. This is mainly to check the new `range`
function.
The `range` fuzz test does find an issue caused by `>` written as `>=`
relatively quickly.
Reviewed By: sfilipco
Differential Revision: D23106176
fbshipit-source-id: e9540cc932503a9d54246d24c70bac829fcb13df
Summary: Migrate to concrete types so it can be typechecked.
Reviewed By: DurhamG
Differential Revision: D23095469
fbshipit-source-id: 27c6da30ca8a1329df544cd2ded7d9734593e48a
Summary:
Read git commit graph and migrate them to `dag::Dag`.
This allows using Rust dag abstractions on the git
commit graph.
Reviewed By: DurhamG
Differential Revision: D23095471
fbshipit-source-id: 2163701350ce82ce6e97074e56ad5877f3c9c158
Summary:
The new path does not calculate the complicated `successorssets`, and is
known to make wez's repo operations significantly faster (which, I suspect is
slowed by a very long chain).
The new code is about 3x faster on my repo too:
# before
In [1]: list(repo.nodes('draft()'))
In [2]: %time len(m.mutation.obsoletenodes(repo))
CPU times: user 246 ms, sys: 42.2 ms, total: 288 ms
Wall time: 316 ms
Out[2]: 1127
# after
In [1]: list(repo.nodes('draft()'))
In [2]: %time len(m.mutation.obsoletenodes(repo))
CPU times: user 74.3 ms, sys: 7.92 ms, total: 82.3 ms
Wall time: 82.3 ms
Out[2]: 1127
Reviewed By: markbt
Differential Revision: D23036063
fbshipit-source-id: afd6ac122bb5d8d513b5cdc033e04d2c377286eb
