Summary:
This makes it possible to actually delete keys from the index, which can be
handy for certain use-cases. For example, `dag` wants to rewrite non-master
ids and segments in certain cases.
Reviewed By: sfilipco
Differential Revision: D18825298
fbshipit-source-id: 3b69ca5fc763ba0c57bd16a28a349f2f829a0759
Summary:
This just moves parameters to a structure without changing the actual logic.
This is to make it possible to add a "remove" API to index.
Some variables are prefixed with `old_` or `new_` to make the code more
consistent.
Reviewed By: sfilipco
Differential Revision: D18825297
fbshipit-source-id: 27bd8438e66125c906ebf2044d6ff978b99c5986
Summary:
Make cpython-ext free from business logic. This adds a bit overhead.
But it should be fine given the error cases are considered rare.
Reviewed By: markbt
Differential Revision: D19186692
fbshipit-source-id: daaffc1369a36c781d1badea822bf62a144eb54e
Summary: We no longer use the `failure` crate. Rename related methods and traits.
Reviewed By: markbt
Differential Revision: D19186694
fbshipit-source-id: 1c83a90ee12db431b7d8e09a9c2cf1d43a4c0f93
Summary:
We no longer use the `failure` crate. Rename the module.
`cpython-failure` is dropped since it was merged into `cpython-ext`.
Reviewed By: markbt
Differential Revision: D19186693
fbshipit-source-id: 410d1491bcadd8d3272c7e2df08ecbe66fc0fef2
Summary:
The `map_pyerr<PE>` API is kind of hard to use correctly since the
code that calls `map_pyerr` does not really know the best suitable error type.
The existing code also "encourage"s the use of stdlib error types like
RuntimeError, ValueError, etc since the code is shorter, which is not great.
Error types from `mercurial.error` are better choices.
With previous diffs in this stack, we now decide the Python error type based
on the Rust error type. It's no longer necessary to specify an error type
with `map_pyerr<PE>`. So let's just drop the type parameter.
A `RustError` was added for error types that `cpython-ext` does not know how
to convert. We should avoid leaking it to the Python land by just implementing
how to convert `anyhow::Error` to `PyErr` in `cpython-ext`.
Reviewed By: markbt
Differential Revision: D19169527
fbshipit-source-id: f4563c36174cd51201b526bbc92a3f1c8a3da864
Summary:
Convert indexedlog error and IOError to dedicated Python types so the Python
world can match them meaningfully by type and handle them accordingly.
Reviewed By: markbt
Differential Revision: D19169210
fbshipit-source-id: 1e7bfdf7cbaf917098efdcd63d600483a9427d33
Summary:
When there are many processes (ex. import helpers). Writing to blackbox can be
slow due to flock contention.
Change blackbox "flush" decision from timestamp based to buffer-size based to
mitigate the issue.
Reviewed By: DurhamG
Differential Revision: D19198470
fbshipit-source-id: 71f87e5e3b45ef4db8efda22ca89fd8ce1fae7fa
Summary:
The current implementation iterates through all entries, which will include
deleted entries. Rewrite it to use index lookups so it does not print
deleted entries.
Reviewed By: sfilipco
Differential Revision: D18838993
fbshipit-source-id: 58d3f2da27cd3e91bc10cbb04bcdca3b7ff07dbb
Summary:
Address review comment in D18640899. I went a bit further that makes COUNT,
ALL, and BITS consistent.
Reviewed By: sfilipco
Differential Revision: D18820721
fbshipit-source-id: e354a3aecf0d8cbe66c43d56198933cdc6420241
Summary:
Address review comment in D18640898.
Id::min_value and Id::max_value become unused and are removed.
Reviewed By: sfilipco
Differential Revision: D18820722
fbshipit-source-id: d9682bf5918307166571fa35aa15cc9eee1a7140
Summary:
Address review comment in D18640899. This makes the word `id` universally
refer to a same concept within the crate.
Reviewed By: sfilipco
Differential Revision: D18820723
fbshipit-source-id: 6803192db7e1304a72100568f8f29b90f25c7779
Summary: There is no need to insert duplicated entries to IdMap.
Reviewed By: sfilipco
Differential Revision: D18670158
fbshipit-source-id: 812ea421c9f36cd12b4aa2be5b2560abec627a52
Summary: This emulates what IdMap the server will send to the client at initial clone.
Reviewed By: sfilipco
Differential Revision: D18670160
fbshipit-source-id: 496142c1e149df9accf816e3d0a1b8316ff64c67
Summary: This makes it easier to check what's stored in IdMap.
Reviewed By: sfilipco
Differential Revision: D18670159
fbshipit-source-id: 91387d01da7d52a0d349e87ffa0aa8983bbe2213
Summary:
This diff implements the following part in a local-only environment:
Id -> Slice: Id -> Request --> Response -> Slice
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Slice -> Id: Slice -> Request --> Response -> Id
^^^^^^^^^^^^^^^^^^^^^^^^^^
Reviewed By: sfilipco
Differential Revision: D18664210
fbshipit-source-id: 740b9cdfe533e7bcc9bbbf2a53c8bcc5bc3648bf
Summary:
This diff implements the following part in a local-only environment:
Id -> Slice: Id -> Request --> Response -> Slice
^^^^^^^^^^^^^
Slice -> Id: Slice -> Request --> Response -> Id
^^^^^^^^^^^^^^^^
Reviewed By: sfilipco
Differential Revision: D18664205
fbshipit-source-id: 47a35c6f95a58ab29c014ac3c6799f16e6114da3
Summary:
In the future the client will have a sparse IdMap, and needs the server (who
has a complete IdMap)'s help to convert between Id and Slice (commit hash)
for commits in the MASTER group.
This diff defines basic abstractions for communications between a sparse IdMap
and a complete IdMap. The basic flows are:
To convert Id to Slice: Id -> Request --> Response -> Slice
To convert Slice to Id: Slice -> Request --> Response -> Id
(`->`: client-side; `-->`: server-side)
Reviewed By: sfilipco
Differential Revision: D18664206
fbshipit-source-id: 93a6b671be30ab1247c71d814817a2ea06b6bc66
Summary: This diff makes EdenFS to be able to use EdenApi to import blobs and trees when requested objects are not present in local hgcache.
Reviewed By: chadaustin
Differential Revision: D18605547
fbshipit-source-id: 4acd2e07cfd9de6b6775ded30ea22a4478b9f1e4
Summary: This will allow us to use EdenApi in EdenFS to fetch trees and blobs.
Reviewed By: xavierd
Differential Revision: D18622844
fbshipit-source-id: 59a9091e9f2fdbcae078da2fb24ee9c0dd18505b
Summary:
Add an option `experimental:use-edenapi` to `EdenConfig`.
See the next diff for usage.
Reviewed By: chadaustin
Differential Revision: D18605549
fbshipit-source-id: 2786c21bb38a76229078662cc5c1ddf906d1be4a
Summary: Ideas for improvements. Put them in the code for future consideration.
Reviewed By: quark-zju
Differential Revision: D18870140
fbshipit-source-id: a2a84d94d72303adc64614b06e36588ff7778aab
Summary:
The list functionality is required by EdenFs. We want this functionality
to be well supported by the Manifest.
Reviewed By: quark-zju
Differential Revision: D18870143
fbshipit-source-id: 1ebaa713ff521226e6ace22cbd35cc841d967298
Summary:
Consistency in naming. The general idea is to have Metadata types that don't
contain paths. Then we will have File, Directory and eventually FsNode that
will contain paths.
Reviewed By: quark-zju
Differential Revision: D18870141
fbshipit-source-id: a1f09add7f1c3dd4fa0348693cd3ce2fd5767fa7
Summary:
This rename is going to make it easier to import and use outside of
manifest specific crates.
Reviewed By: quark-zju
Differential Revision: D18870142
fbshipit-source-id: 2f3ea460170308162ee834efc038b2dcedd9e233
Summary:
Provide a way to repair the MetaLog. It's not just repairing 2 indexedlog
structures, but also checking the relationships between them.
Reviewed By: xavierd
Differential Revision: D18737904
fbshipit-source-id: fcf8ae56a1fdbb0561765701d962dfad4a8b5bd4
Summary:
The environment variable `EDENSCM_TRACE_OUTPUT` specifies where to write the
tracing output for the current command. Environment variables are inherited
by subprocesses by default. That is undesirable because another hg command
(triggered by hook, or background maintaince) will rewrite the trace output.
Avoid it by unsetting EDENSCM_TRACE_OUTPUT for subprocesses.
Reviewed By: xavierd
Differential Revision: D18892973
fbshipit-source-id: 575b6c0df2e7a0775172893e4aa72ca33fa4658c
Summary:
The previously behavior silently uses the empty Root upon "id not found" cases,
which hides data corruption issues captured by the next changes.
Reviewed By: xavierd
Differential Revision: D18737906
fbshipit-source-id: 0f2cfe1b05f11e9d2584d96a14afcb1aed80da4d
Summary: Otherwise the repair will fail with "unable to lock directory" errors.
Reviewed By: xavierd
Differential Revision: D18737911
fbshipit-source-id: 2c47b5b9a071e0db2d20fc385ef466aef1e732f6
Summary: This makes it possible for other crates to implement the `Repair` trait.
Reviewed By: xavierd
Differential Revision: D18737909
fbshipit-source-id: 3d098e583b45e84eb4bd01ef45569e9694b63ed6
Summary:
This simplifies the logic a bit by getting a free `repair` method from
indexedlog.
Reviewed By: xavierd
Differential Revision: D18737908
fbshipit-source-id: 4988c1a83b7709b751cd1899c5663acc0c42e313
Summary:
The goal is to make it easier for different structures to implement `repair` by
just providing `OpenOptions`.
Reviewed By: xavierd
Differential Revision: D18737903
fbshipit-source-id: af3c18038c5bb2f91f53ab53926ec097e1d87e6b
Summary: Uses `folly::IOBuf`'s `TakeOwnership` instead of `CopyBuffer` to achieve zero-copy of file content we get from Rust.
Reviewed By: chadaustin
Differential Revision: D18264900
fbshipit-source-id: 45baa70cb771a2bff0b31feaa1b85569625218eb
Summary:
This isolates the core types in manifest so that it is harder to
create unsound dependencies with specific implementations.
Reviewed By: quark-zju
Differential Revision: D18843133
fbshipit-source-id: 4b866ad84d2e7d0ff2dc4ec6bd65f66548c3fe4a
Summary: Signal more clearly that the iteration in the cursor is depth first.
Reviewed By: quark-zju
Differential Revision: D18843135
fbshipit-source-id: 8ee2c612963eaf6fec3ae48f75ebf16bd177fed4
Summary: Items feels a bit generic as a name.
Reviewed By: quark-zju
Differential Revision: D18843136
fbshipit-source-id: 29a904f3a8a1996760e71bd78610bdbfef4ad65b
Summary: No longer strictly related to files.
Reviewed By: quark-zju
Differential Revision: D18843128
fbshipit-source-id: c7853e08aa5d0256616e777c5bb6dd477b9ebc97
Summary:
`hgid` can be easily derived from the link member variable. Using link is
preferable because link is the owner of the data. It makes the logic more
specific, less bug prone to use the true source of the data.
Reviewed By: quark-zju
Differential Revision: D18843137
fbshipit-source-id: 443f1e5eb4498fa4f2ff9904dba7cc3dc4305add
Summary:
This structure is not public. It is private to the manifest crate so there is
no point to have those modifiers.
Reviewed By: quark-zju
Differential Revision: D18843134
fbshipit-source-id: 0794bbbf578863cabbd85ada87669b791ebcc72d
Summary:
The purpose of this change is to disambiguate from the other Directory struct
that we have in the Manifest. Right now that other is in manifest/lib.rs
Reviewed By: quark-zju
Differential Revision: D18843130
fbshipit-source-id: f25a60f3040cc19abcab99ac5f2cdbff83ccf7ee
Summary:
Directory<'a> which wraps a link is closely linked with the Link abstraction.
It can be seem as a convenience wrapper around Link.
Reviewed By: quark-zju
Differential Revision: D18843129
fbshipit-source-id: 2b677d71101bff3708a65e03bbcddf5074d51387
Summary:
Diff is a core operation for manifests. The Manifest trait should require that
it is implemented.
Reviewed By: quark-zju
Differential Revision: D18843132
fbshipit-source-id: 5e3febab669470e266878d934561dfc323b71c7f
Summary: The conversion from tuple is not used much. I don't like them that much either.
Reviewed By: quark-zju
Differential Revision: D18843131
fbshipit-source-id: fb06b72a279b59ee8af5e43d83a10e39a576730b
Summary:
Querying all the files in a Manifest is core functionality that every Manifest
implementation should implement. The main question is how do we describe that.
We want to return an interator over the files of the manifest. The problem is
that the manifest is a trait and functions on traits can't return traits.
The solution is to Box the results. Not pretty but allows us to move the code
forward.
Reviewed By: quark-zju
Differential Revision: D18822854
fbshipit-source-id: ca44b24816203049f747ad67af48e33370861024
Summary:
The purpose of the rename is to signal that this function may potentially do
a lot of work, even calling and waiting on network requests.
Reviewed By: quark-zju
Differential Revision: D18822855
fbshipit-source-id: 98471944c8e6c7cffa7a2a11f2f733274802b376
Summary:
Removing the contructor for File that takes dependency on
tree implementation details.
Reviewed By: kulshrax
Differential Revision: D18822858
fbshipit-source-id: aed33b146dfdbdb23721a0f6e96977025d71a627
Summary: The types in file.rs being in a separate file confuse people.
Reviewed By: quark-zju
Differential Revision: D18822856
fbshipit-source-id: 29aa10c76bb5ef74f1fc0bf726e6b3cfaf19b428
Summary:
In the future we might have revlog or more complicated storage for maintaining
the IdMap. Define a trait for it.
Reviewed By: sfilipco
Differential Revision: D18664208
fbshipit-source-id: 242611d78852e8d82a52ab52192db87c13b848e7
Summary: This is the reverse of first_ancestor_nth. It will be used later.
Reviewed By: sfilipco
Differential Revision: D18664209
fbshipit-source-id: 8d3706cdc080086ec60da60870fde76c8933f6a8
Summary: This will be used in an upcoming change.
Reviewed By: sfilipco
Differential Revision: D18664207
fbshipit-source-id: 3030bb02e172b95c6dbabda74c318e7f16212f59
Summary:
With the ONLY_HEAD flag, ancestors can be sped up sometimes. Since ancestors
is used widely in other DAG calculations, perf improvements are visible across
the benchmark. The trade-off is to spend a bit more time on building segments
to figure out ONLY_HEAD.
After this diff:
building segments 576.625 ms
ancestors 47.119 ms
children 436.967 ms
common_ancestors (spans) 257.096 ms
descendants (small subset) 367.316 ms
gca_one (2 ids) 146.315 ms
gca_one (spans) 253.059 ms
gca_all (2 ids) 236.075 ms
gca_all (spans) 266.695 ms
heads 220.766 ms
heads_ancestors 36.258 ms
is_ancestor 98.967 ms
parents 219.173 ms
parent_ids 9.282 ms
range (2 ids) 99.859 ms
range (spans) 205.154 ms
roots 429.047 ms
Before this and previous diff:
building segments 542.564 ms
ancestors 316.075 ms
children 428.612 ms
common_ancestors (spans) 452.850 ms
descendants (small subset) 537.483 ms
gca_one (2 ids) 526.112 ms
gca_one (spans) 446.726 ms
gca_all (2 ids) 780.590 ms
gca_all (spans) 490.906 ms
heads 214.005 ms
heads_ancestors 217.558 ms
is_ancestor 256.769 ms
parents 213.170 ms
parent_ids 9.313 ms
range (2 ids) 259.435 ms
range (spans) 358.934 ms
roots 423.005 ms
Reviewed By: sfilipco
Differential Revision: D18642797
fbshipit-source-id: 27f1ee5797a92142b2c19195eb26baef58769553
Summary: The pre-calculated flag can be used for some fast paths.
Reviewed By: sfilipco
Differential Revision: D18642796
fbshipit-source-id: bc16b3517cc29e23478f2027e83e2a2282be0d0a
Summary:
The bindings APIs are out of date.
The latest idea is to write non-master commits to disk, in a different group.
There are no "segments in memory". Therefore:
- Rename `builddisk` to `build`.
- Make sure `build` updates internal states by calling `reload`.
- Drop `buildmem`.
- In case `builddisk` gets called very frequently, reduce its overhead by
doing nothing if all commits are built already.
The constructor is revised to match the latest interface (no
"max segment level" and "set_new_segment_size" needs to be called).
Reviewed By: sfilipco
Differential Revision: D18640897
fbshipit-source-id: 76a5d0db363713dc5b763503c238c1bfa61ea02e
Summary: Make it possible to build non-master segments.
Reviewed By: sfilipco
Differential Revision: D18641507
fbshipit-source-id: 4d0c46cdea240849af37d28219b6023578514323
Summary:
`build_segments_volatile(id)` should build flat segments so it includes `id`.
Right now it can fail that in a corner case. Fix it.
Reviewed By: sfilipco
Differential Revision: D18641509
fbshipit-source-id: ad1de694e4d31c51f29e9bc5800291066bf16dbb
Summary:
The IdMap has assertions about 1:1 mapping between Id and Slice.
Practically, a non-master commit can be merged into a master commit.
That means a commit hash might map to 2 Ids: [master-id, non-master-id].
Allow that to happen (temporarily).
Note: The non 1:1 mapping causes more headache than benefit and
D18838995 restores 1:1 mapping. But it still requires temporary
non-1:1 mapping to work.
Reviewed By: sfilipco
Differential Revision: D18641508
fbshipit-source-id: bc4bc831d5ab5ebfc2588bafd48114ae49d31e0f
Summary: panic on drop is a terrible idea. It can cause SIGILL kill without meaningful messages or meaningful backtrace. Remove it.
Reviewed By: xavierd
Differential Revision: D18640894
fbshipit-source-id: 3ab52ded9875e17e891acd8f9b45d82964615f01
Summary: This makes the code harder to crash in abnormal cases.
Reviewed By: sfilipco
Differential Revision: D18640895
fbshipit-source-id: ee81dd7739739d3d15052bdb7bb2ba80ed48cd0a
Summary:
This allows us to allocate Ids for non-master commits in a separate range
so they wouldn't cause fragmentation for commits in the master group.
Reviewed By: sfilipco
Differential Revision: D18640896
fbshipit-source-id: c19c53342583ded8795849584cf5f31bc86b2818
Summary: Migrate the codebase to use the new Id type instead of an alias of u64.
Reviewed By: sfilipco
Differential Revision: D18640898
fbshipit-source-id: 415db9afeda1f8960e18c69e70f53a9eeb63cbbe
Summary:
Background:
- Tests and Nuclide haven't removed usage of revision numbers.
- If we build segments for non-master commits on demand, there are performance
concerns about very long release branches.
- It's very desirable to keep `ancestors(master)` continuous. In other words,
`ancestors(master)` is a single integer span. This affects performance for
DAG algorithms a lot.
To solve the problems, I'm going to use separate ranges for master and
non-master commit Ids (so assigning Ids to non-master commits wouldn't affect
what numbers can be used for the master group). To make the code easier to
read, I'm changing `type Id = u64` to `struct Id` where the struct is
group-aware.
This diff introduces related types and basic operations.
Reviewed By: sfilipco
Differential Revision: D18640899
fbshipit-source-id: 8f7ecbee6e0d1703d1225f18e92742981dbf3757
Summary:
When changing a file to a folder in a commit this function would try to query
the storage using the path and the id of the file. Storage would not have
such an entry and would cause the operation to fail.
Fixing the issue by filtering out ids of files in the section for known
manifests. Filtering already occurs in the section for transmitted manifest.
Reviewed By: quark-zju
Differential Revision: D18797368
fbshipit-source-id: 2ed727b5eaaab2f17cd988d6c8a3a1055ac8bd2a
Summary:
These are not currently used by mercurial and are superseded
by the newer more general `watchman_client` crate on crates.io:
https://docs.rs/watchman_client/0.2.0/watchman_client/
Some of the implementation of that crate was based on the work I'm
removing here.
I was going to update hg_watchman_client to be implemented in terms
of the new crate, but since it isn't used and it would only be a thin
wrapper, I figure that it is best to delete this code instead so that
we can integrate it in terms of the new API when we're ready.
Reviewed By: singhsrb
Differential Revision: D18777161
fbshipit-source-id: efacb749e6d35ff4cc1fdc7d99cdca9ed42c9cfe
Summary:
It would be nice if the insert method would give back the path that
it failed to insert.
Reviewed By: dtolnay
Differential Revision: D18739978
fbshipit-source-id: 4c16d09750ade2f01397161129c31bcf0059a957
Summary:
The Mercurial code has the bad habbit of inserting files in what can
appear impossible locations. Sometimes files with directory names and
sometimes directories in file locations. This happens because the initial
code would do additions before deletions on Manifest implementations that
were rudimentary. As soon as we introduce validation various code paths
surface.
I tried to fix the codepaths that modify manifests but it's a losing game.
I fixed the issue that appeared in tests and a couple of issues people
reported but more situations crop up.
This is giving up on the python code.
Reviewed By: quark-zju
Differential Revision: D18737678
fbshipit-source-id: 0c97128ff67e5ba2334942b6afc404aa64a2e5f4
Summary:
`path = ...` is enough. This should resolve the following build issues:
error: failed to select a version for the requirement `serde_bser = "^0.1.0"`
candidate versions found which didn't match: 0.2.0
location searched: ...
required by package `watchman_client v0.1.0 (...)
... which is depended on by `hg_watchman_client v0.1.0 (...)`
Reviewed By: lukaspiatkowski
Differential Revision: D18738176
fbshipit-source-id: 725840895a8e988b35000a48cf92018b14cb4dee
Summary:
We have seen cases on Windows where the hg process gets stuck in
CreateToolhelp32Snapshot. Let's be defensive and always exit the loop.
Reviewed By: singhsrb
Differential Revision: D18729720
fbshipit-source-id: fb8602ce231eec01b6b42c6759849d56e5db2030
Summary:
This helps test if there are uncommitted changes. It can help detecting
programming errors. For example, is metalog changed before starting a
transaction?
Reviewed By: xavierd
Differential Revision: D18538148
fbshipit-source-id: 9c82838d8174e6a93fde7108503a025660191cbf
Summary:
The current `commit` code path errors out without telling much about what
conflicts are. Improve it by showing what are actually conflicting, and allows
users to replace the resolver function to do customized merges.
Reviewed By: xavierd
Differential Revision: D18519996
fbshipit-source-id: 78f42ada1df45659783ce1eba30a19f6fc6b3b62
Summary:
Remove some unnecessary forward declarations, and use full paths in includes
so autodeps can recognize which rule these headers come from.
Reviewed By: fanzeyi
Differential Revision: D18652106
fbshipit-source-id: e2c84e1aaf846c91c18242050c4015eca8c501d3
Summary:
Now that the fb-mercurial sources are available in the Eden repository, update
our CMake build files to always build them. This moves the build logic from
the centralized `FBMercurialFeatures.cmake` file into `CMakeLists.txt` files
in the appropriate subdirectories.
Reviewed By: chadaustin
Differential Revision: D18588011
fbshipit-source-id: ded9decde5c2ec766aae0bb0f4f5b021d1044a98
Summary:
This diff replaces eden's dependencies on failure::Error with anyhow::Error.
Failure's error type requires all errors to have an implementation of failure's own failure::Fail trait in order for cause chains and backtraces to work. The necessary methods for this functionality have made their way into the standard library error trait, so modern error libraries build directly on std::error::Error rather than something like failure::Fail. Once we are no longer tied to failure 0.1's Fail trait, different parts of the codebase will be free to use any std::error::Error-based libraries they like while still working nicely together.
Reviewed By: xavierd
Differential Revision: D18576093
fbshipit-source-id: e2d862b659450f2969520d9b74877913fabb2e5d
Summary:
Add Event::Tags so we can tag commands with one or more names.
Add index for the tags so we can search them more efficiently.
As we're here, update hgcommands to read EDENSCM_BLACKBOX_TAGS and write
Event::Tags automatically just after Event::Start.
Differential Revision: D18610733
fbshipit-source-id: c69b6e3d34b317e92a1c4d76f29e832b6b01c4fa
Summary:
Previously, the blackbox index was hardcoded on the timestamp, and is not
Event-aware. This diff changes the index to parse Event and build up indexes
for various common events (ex. Start, Finish).
For now, the index just includes start, finish timestamps and duration.
Differential Revision: D18610734
fbshipit-source-id: bf572f3138c8c18d3aab06eadcc83dd5b0b9cf02
Summary:
Previously, `match_pattern` only returns a `bool`. This diff implements an
enhanced version `capture_pattern` to be able to capture matched objects.
This will be used to match query patterns and apply optimization.
Differential Revision: D18610732
fbshipit-source-id: 39e989e7633b834c04997bf8665494556f3b62b2
Summary: Migrate callsites to use the new pattern matching filtering APIs.
Differential Revision: D18610738
fbshipit-source-id: 242182f4e6a58bbff5dddbb177287921c85519bc
Summary:
See the previous diff. As part of migrating away from IndexFilter. There needs
to be a way to filter commands by absolute clock time using pattern matching
against Events. To achieve that, add the timestamp field to Start and Finish
events.
Differential Revision: D18610735
fbshipit-source-id: 9e1898a87bc93e47d0f0c77342428b5c42ef6f24
Summary:
The existing query API `filter` has 2 issues:
- It does querying and filtering together, making it hard to express certain
queries. For example: "for commands taking more than 10 seconds, get all
network requests used by those commands".
- The `IndexFilter` abstraction overlaps with `pattern`s. `IndexFilter` is
built upon the `Entry` structure, and does not work for `Event`s. In the
future, we want a `Event::Tags` to tag commands, and build indexes on it.
That's not very compatible with `IndexFilter`.
To address the issues, I'm going to:
- Decouple querying and filtering by splitting it into 2 methods glued by
`SessionId`s.
- Deprecate `IndexFilter`. Make `patterns` the only way to filter entries.
- Add dedicated indexes for certain events.
This diff adds the related APIs towards that direction.
Differential Revision: D18610737
fbshipit-source-id: c6ab92fd6fa7bb2ee7e14f567d72ac98b287c143
Summary:
The type `Entry<T>` provides flexibility on serializing different types.
Practically, only the `Event` type is used. Let's remove `T` so `blackbox` can
assume it's the `Event` type.
Differential Revision: D18610736
fbshipit-source-id: 102a5f61c758fa83cd1994c7f813576dcf5b59d5
Summary:
Add tracing spans so we can get some insights about statistics from the pure
Rust land.
Reviewed By: xavierd
Differential Revision: D18625646
fbshipit-source-id: 15951fe2f8270a70f647a0f4e09819bbc33919c5
Summary: This diff adds Rust structs to represent the data EdenFS wants from a manifest so we can pass these to EdenFS.
Reviewed By: wez
Differential Revision: D18365440
fbshipit-source-id: b474ceb698d9230e1b02c2cce1bec2a32bd3f94f
Summary:
Some clean up.
`CByte` should clean up `vec` in `Drop` since we are going to have `CBytes` in other struct's field and this will allow Rust automatically manage this memory.
Reviewed By: chadaustin
Differential Revision: D18365441
fbshipit-source-id: 7038bc6ff457415ee8234cc4c6b7df0f70005cfe
Summary: `manifest::Tree` wants to have a store that implements `TreeStore`. However at current we don't want to introduce dependency between `manifest` and `contentstore`. So we create this wrapper struct to provide the trait implementation. This will allow us to use `manifest` to parse manifest data we get from hg store.
Reviewed By: chadaustin, xavierd
Differential Revision: D18365354
fbshipit-source-id: 3687032c7f51570ef51ccc4004efc87f64ef2188
Summary: This diff adds `Option<HgId>` to `FsNode::Directory` because EdenFS needs the tree hash for the directories in the tree.
Reviewed By: sfilipco
Differential Revision: D18307205
fbshipit-source-id: 93db944ff1686a1c5927aa9a876a0fd008edbf9a
Summary: I got unused warning every time I build Mononoke.
Reviewed By: singhsrb
Differential Revision: D18573764
fbshipit-source-id: 23921f581bdc74655041a2413f80cb159b4ba010
Summary:
Edenapi uses a group of impls like the following as the canonical way to construct ApiError.
```
impl From<Context<ApiErrorKind>> for ApiError {
```
Downstream code would write:
```
use failure::ResultExt;
let stream_trees = config
.get_or_default("edenapi", "streamtrees")
.context(ApiErrorKind::BadConfig("edenapi.streamtrees"))?;
```
This relies on the way that ResultExt::context returns a unique type that is different from failure::Error.
This diff introduces a dedicated extension trait to allow the same code in the caller to continue to work even without a failure::Context<T> type, which will be required for dropping our dependency on failure::Fail and failure.
```
pub trait ApiErrorContext<T> {
fn context(self, kind: ApiErrorKind) -> ApiResult<T>;
}
```
We also introduce a public constructor for ApiError and replace indirect construction that used to look like this:
```
error.context(ApiErrorKind::Curl).into()
```
with this instead:
```
ApiError::new(ApiErrorKind::Curl, error)
```
which is the same number of characters but clearer. The argument order matches [std::io::Error::new](https://doc.rust-lang.org/std/io/struct.Error.html#method.new).
Reviewed By: kulshrax
Differential Revision: D18574668
fbshipit-source-id: 0a56297bb942a26d75a62ca39fc16abeb4486345
Summary: The CommitCloudHttpError seems to have been working around a limitation of failure's derive that no longer applies to thiserror.
Reviewed By: xavierd
Differential Revision: D18523907
fbshipit-source-id: 54efaedc405911a412ce842dff6285fb80d875b8
Summary:
Now that fmt is available in Folly builds (D14813810), use it to reduce binary code size in Folly Logger. This is done by moving most of the formatting logic behind the type-erased `vformat` API. Previously it was instantiated for all combinations of formatting argument types used in calls to `FB_LOGF` and `XLOGF` in a program.
The effect of this change can be illustrated by looking at symbol sizes as given by `nm -S -td` for the following test function:
```
void test_log() {
FB_LOGF(logger, WARN, "num events: {:06d}, duration: {:6.3f}", 1234, 5.6789);
}
```
compiled in `opt` mode.
`nm` before:
```
0000000004236736 0000000000000231 T test_log()
0000000004236992 0000000000001002 W std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > folly::LogStreamProcessor::formatLogString<int, double>(folly::Range<char const*>, int const&, double const&)
```
`nm` after:
```
0000000004237536 0000000000000231 T test_log()
0000000004237792 0000000000000251 W std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > folly::LogStreamProcessor::formatLogString<int, double>(folly::Range<char const*>, int const&, double const&)
0000000004238048 0000000000000740 W folly::LogStreamProcessor::vformatLogString[abi:cxx11](folly::Range<char const*>, fmt::v5::format_args, bool&)
```
Before we had one 1002 byte instantiation of `formatLogString<int, double>`. With this change it was reduced 4x to 251 bytes and non-template function `vformatLogString` was added which is shared among all logging calls. The size of `test_log` remained unchanged. There are even bigger savings from Folly Formatter instantiations which are no longer needed, e.g.
```
0000000004238032 0000000000001363 W _ZNK5folly13BaseFormatterINS_9FormatterILb0EJRKiRKdEEELb0EJS3_S5_EEclIZNKS7_8appendToINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEENSt9enable_ifIXsr12IsSomeStringIT_EE5valueEvE4typeERSH_EUlNS_5RangeIPKcEEE_EEvSK_
```
So in total this change results in ~5x per-call/instantiation binary size. It is possible to reduce binary size even further but it is not done in the current diff to keep it manageable.
In addition to binary size improvements, switching to fmt will potentially
* allow catching errors in format strings at compile time,
* simplify future migration to C++20 [`std::format`](http://eel.is/c++draft/format).
Reviewed By: simpkins
Differential Revision: D15485589
fbshipit-source-id: 06db4436839f11c2c3dbed7b36658e2193343411
Summary:
This diff replaces code of the form:
```
use failure::Fail;
#[derive(Fail, Debug)]
pub enum ErrorKind {
#[fail(display = "something failed {} times", _0)]
Failed(usize),
}
```
with:
```
use thiserror::Error;
#[derive(Error, Debug)]
pub enum ErrorKind {
#[error("something failed {0} times")]
Failed(usize),
}
```
The former emits an implementation of failure 0.1's `Fail` trait while the latter emits an impl of `std::error::Error`. Failure provides a blanket impl of `Fail` for any type that implements `Error`, so these `Error` impls are strictly more general. Each of these error types will continue to have exactly the same `Fail` impl that it did before this change, but now also has the appropriate `std::error::Error` impl which sets us up for dropping our various dependencies on `Fail` throughout the codebase.
Reviewed By: Imxset21
Differential Revision: D18523700
fbshipit-source-id: 0e43b10d5dfa79820663212391ecbf4aeaac2d41
Summary:
`globset` supports multiple matching strategies, including literal prefix
(backed by AhoCorasick), or regexp, etc.
In theory patterns like `foo/**` (where `*` cannot match `/`) can use `foo`
prefix strategy. However, the implementation detail of `globset` wouldn't
accept it as a prefix. But `foo/*` (where `*` can match `/`) can be treated as
a prefix. Transform the former pattern to the latter to hint `globset` to use
the optimal strategies.
Reviewed By: sfilipco
Differential Revision: D18500298
fbshipit-source-id: 39e604d6157a919b75c392488b6d42375e518c16
Summary:
This diff is preparation for migrating off of failure::Fail / failure::Error for errors in favor of errors that implement std::error::Error. The Fallible terminology is unique to failure and in non-failure code we should be using Result<T>. To minimize the size of the eventual diff that removes failure, this codemod replaces all use of Fallible with Result by:
- In modules that do not use Result<T, E>, we import `failure::Fallible as Result`;
- In modules that use a mix of Result<T, E> and Fallible<T> (only 5) we define `type Result<T, E = failure::Error> = std::result::Result<T, E>` to allow both Result<T> and Result<T, E> to work simultaneously.
Reviewed By: Imxset21
Differential Revision: D18499758
fbshipit-source-id: 9f5a54c47f81fdeedbc6003cef42a1194eee55bf
Summary:
simpkins encountered a case where the treematcher does not work with many
patterns. It turns out `globset` has a hard-coded regex size limit (10MB).
Implement proper error handling so we can detect such issues and fallback to
slower paths.
Reviewed By: sfilipco
Differential Revision: D18500299
fbshipit-source-id: 0122ba9b0246c1536b2069a40e13261ee47f8bba
Summary:
Merge the fb-mercurial code into the Eden repository, under the
`eden/scm` subdirectory.
Reviewed By: quark-zju
Differential Revision: D18445774
fbshipit-source-id: fc3307f9937e0c7e1c8f7d03c5102c4fe5dedb10
Summary:
In preparation for merging fb-mercurial sources to the Eden repository,
move everything from the top-level directory into an `eden/scm`
subdirectory.
