Summary:
We want to migrate the tests to run using treemanifest. As part of
that, we want to first transition to using treemanifest without actually
changing the hash, so we can check that the tests still work first, then update
the hashes second.
This diff adds the flatcompat mode and enables it by default. A future diff will
start enabling treemanifest for existing tests.
Reviewed By: quark-zju
Differential Revision: D15030252
fbshipit-source-id: 06c82be749282d62f1d9cfb43246695c427f8165
Summary:
After removing the Hyper based EdenAPI client, there is at present only one client implementation. As such, the Python bindings have been updated to use that implementation directly rather than a `Box<dyn EdenApi>` trait object.
In spite of this, it would be nice to ensure that this trait remains object safe so that if we add a new implementation in the future, we can easily go back to using a trait object. (Otherwise, it is very tempting to add `impl Trait` or generic methods to the trait which would be difficult to remove later.)
This diff adds a simple compile-time check that ensures the trait is object safe.
Reviewed By: quark-zju
Differential Revision: D15289514
fbshipit-source-id: e880f22fa7f48b52467aedf41036711a1713deb9
Summary:
The Hyper-based EdenAPI client proved to be problematic in practice due to various difficult to debug issues with Tokio, Hyper, TLS, and h2. We have kept it around for the time being while building out the Curl based client in case we wanted to revert back to a pure Rust solution.
Today, the Curl client works well, and future work on the EdenAPI will involve adding more functionality to it. Given that both the Hyper and Curl clients implement the EdenAPI trait, modifications to the trait involve updating both clients. So far this has been acceptable because the updates have been minor, but we would now like to add substantial new functionality to the trait (namely tree fetching), and adding this functionality to the Hyper client would take nontrivial effort.
Given that we aren't using this client at all, let's just delete it. We can always bring it back from the repo's history if we need it in the future.
Reviewed By: quark-zju
Differential Revision: D15289196
fbshipit-source-id: d9c0c3cfc5087c3e080a9919dd9e627b9657676c
Summary:
The direct motivation is the recent `ui` prefix change made `abort:`
untranslated.
The i18n translations are unmaintained. As we move more logic to Rust, things
would be worse.
I18n tests does not seem to provide much value. Therefore just remove them.
Reviewed By: kulshrax
Differential Revision: D15291618
fbshipit-source-id: 1a244ae32dc8e2f099aba46cbd45553bbb477b2c
Summary: Given that the on-the-wire representation of file and tree data is essentially the same, we can reuse our existing structs for file fetching for tree fetching. As such, we should rename these structs to make it clear that they are not just for file data. (This also involves updating some comments/variable names to make sure everything is consistent.)
Reviewed By: singhsrb
Differential Revision: D15286690
fbshipit-source-id: 8c7fa4392057e90a9f19beb17b0bbcbf04b7e8e7
Summary:
For now, only pack-file based stores implement MutableHistoryStore, but once
the trait is implemented for stores that are updated in place, returning a Path
on close will not make much sense.
Reviewed By: kulshrax
Differential Revision: D15285970
fbshipit-source-id: 011db2b60c11c1eebfe11881cfc5ebafa1676704
Summary:
While it makes sense that closing a datapack returns the path, it doesn't
really mean anything for an update in place store. Let's change the API to
return an Option<PathBuf>.
Reviewed By: kulshrax
Differential Revision: D15285969
fbshipit-source-id: 804acd75607e86a0bc875910f6aaa300a5526558
Summary:
With the MutableDeltaStore and MutableHistoryStore traits, this one is no
longer used outside of revisionstore. Let's stop exporting it.
Reviewed By: kulshrax
Differential Revision: D15284473
fbshipit-source-id: 61d87a1843cfed51dddce36e314128ac8244a2dc
Summary: The edenapi is now independant of the storage type for history data.
Reviewed By: kulshrax
Differential Revision: D15284355
fbshipit-source-id: 72a5db42bb0fb19ee03155b13914202581ab5966
Summary:
The python mutablehistorypack can implement MutableHistoryStore, so let's do
it.
Reviewed By: kulshrax
Differential Revision: D15284357
fbshipit-source-id: e55818e2432573f0fbe4f5b73c592f7f6418785c
Summary:
This allows for a MutableHistoryPack to be used where a MutableHistoryStore
will be required. Once an IndexedLog based history store is implemented we will
be able to switch between the 2 more easily.
Reviewed By: kulshrax
Differential Revision: D15284356
fbshipit-source-id: 91d75ddc6991c26eace67d77679bb8d5806cf8b8
Summary: This will help in abstracting the kind of store that is being written to.
Reviewed By: kulshrax
Differential Revision: D15284358
fbshipit-source-id: ab6a6d23978480ca65587b745ae39ac6ed98cca9
Summary:
The type of store where data is stored is now fully abstracted to the python
bindings. For now, edenapi will write to the pending mutabledatapack, but we
can now switch it easily to any other store implementing MutableDeltaStore,
including an IndexedLogDataStore.
Reviewed By: kulshrax
Differential Revision: D15266191
fbshipit-source-id: 638cf90a567ef170e0302376312c4b82e6d6b6da
Summary:
The python mutabledatapack is a delta store, and therefore we can implement the
MutableDeltaStore trait for it.
Reviewed By: kulshrax
Differential Revision: D15266192
fbshipit-source-id: 6cc04e4a5e792d037d17c706a374f6864a2902a9
Summary:
This will allow to transparently use the IndexedLogDataStore or a datapack in
the edenapi code.
Reviewed By: kulshrax
Differential Revision: D15266194
fbshipit-source-id: 6396118a5c8107a8c91e5fc83fe4297d4321d10c
Summary:
This will be used to abstract writing to a MutableDataPack or
IndexedLogDataStore (or both).
Reviewed By: kulshrax
Differential Revision: D15266193
fbshipit-source-id: 99f2383555addbafea81a2752e8d6759a1c1c5e7
Summary: As we add more functionality to the Eden API, we will have a lot more request structs. These structs are only used by the HTTP data fetching code, and should not be used by actual business logic. As such, while these types need to be public (so that both Mononoke and Mercurial can use them), they should not be re-exported at the top level.
Reviewed By: quark-zju
Differential Revision: D15268439
fbshipit-source-id: e7d1405d2ac234892baedbf7dbf3e133d187cb45
Summary: Adds tracing for mergedriver and for the individual generators.
Reviewed By: quark-zju
Differential Revision: D15223926
fbshipit-source-id: c0c6e1a6814f59fda3ddd75eee901a60a2335c1b
Summary:
Set the `component` to `"commitcloud"` for commit cloud statuses and
messages, rather than using custom highlight functions.
Reviewed By: quark-zju
Differential Revision: D15201944
fbshipit-source-id: 7635942a5ca029209711a2b89c32cc5fd677d22f
Summary:
Add optional prefix keyword arguments to `ui.write` (and thus `ui.status`,
`ui.warn`, etc.). These prefixes can be used to indicate an error condition,
some notice to the user, or to mark a message as coming from a particular
component. The prefixes are labelled so that they can be colored to stand
out from the surrounding text. The default colors are red for errors, yellow
for notices, and cyan for components.
Add a component parameter to `error.Abort` (and thus any exception that
inherits from it) that can be used by callers to mark which component is
responsible for the exception. When the error is printed, the component is
passed as the component prefix of the error message.
Reviewed By: mitrandir77
Differential Revision: D15201946
fbshipit-source-id: 02e3da40e9563704fa1d7ce81366e6e7f66c7f34
Summary:
D15188668 added the `--batch` option to the `metaedit` command along
with updating the tests. Unfortunately, the test is broken on OSX. This commit
fixes the test to be compatible with OSX.
Differential Revision: D15273960
fbshipit-source-id: 57a67b9d7ab2b471ee205db4fdfc7f3e6e2e553b
Summary:
While closing the repo, hg_memcache_client is terminated, and later, the
pending packfiles are commited. Unfortunately, the commit phase is also when
the content of the pending packfiles will be sent to memcache, and since the
connection is at this point closed, nothing is sent then.
Reviewed By: quark-zju
Differential Revision: D15247639
fbshipit-source-id: 8719cb8bbc3c9571b9cfe250ba4be2b745e4ba15
Summary:
A tree manifest entry must always end with a line feed. It is somewhat
redundant but that's how the serialization is defined. Sometimes that last
line feed is missing in our data. I don't know why.
Reviewed By: quark-zju
Differential Revision: D15110860
fbshipit-source-id: c4ac5075e22a8b8851f6b246d22af8ab68f42a74
Summary:
This is a quality of life improvement for working with the storage layer.
We probably don't gain a whole lot by statically linking the store and it is
useful to have some flexibility in the storage layer.
Differential Revision: D15110859
fbshipit-source-id: 6102acafa21dd1dbaeed0f8fc3147538a8c301d1
Summary:
With the mutablestores, this code was starting to be too messy and hard to
follow. Let's simplify it a bit.
Reviewed By: quark-zju
Differential Revision: D15200748
fbshipit-source-id: 2c0cc5a4ff5057f7a9590fcc602afc2b1cc72dcd
Summary:
The `finally` block relies on this name being in scope. If the exception
happens before this name is initialized, the finally block also throws and we
loose the original root cause exception.
Reviewed By: singhsrb, HarveyHunt
Differential Revision: D15244583
fbshipit-source-id: fd8be11987c3028f775a62cb3ae308b30a9d5e75
Summary:
If a chg server was started by a low priority process, it will inherit the low
priority and can be painfully slow.
On the other hand, if the chg client was low priority, the user or the automation
wants the process to be low priority intentionally. Therefore connecting to an
existing chg server might bypass intention.
Therefore, detect the low priority case, and refuse to start or connect to a
chg server.
Previous attempts to workaround slow chg server were removed as they are
probably no longer necessary.
Reviewed By: singhsrb
Differential Revision: D15223518
fbshipit-source-id: 718e807820d481eac204f2293f949c3b315f923d
Summary:
In a previous diff I did a last minute refactor that apparently broke a
test. I needed to update this line as well to swap the arguments.
Reviewed By: singhsrb
Differential Revision: D15219418
fbshipit-source-id: c66a2cf7bbbec80ebbbd6ca818151de5f0c2d2c5
Summary: Update the tests to expect the new output added by D15173846.
Reviewed By: quark-zju
Differential Revision: D15212315
fbshipit-source-id: 82bb3b4e67a28eb8905d35fcaa8251947163f521
Summary:
When remotefilelog.fetchpacks is enabled, it's possible that 100 packfiles of
100MB each are present. In this case, every new packfiles that
hg_memcache_client would write will force an incremental repack, which will
only reduce the number of packfiles by a small number.
Let's have a simple heuristic that tries to bring the number of packfiles to be
lower than 50.
Reviewed By: DurhamG
Differential Revision: D15203771
fbshipit-source-id: 18c39487d5ac087d4879004993c1c1add087249c
Summary: Update the test code to return a 3-tuple as required after D15173846.
Reviewed By: singhsrb
Differential Revision: D15212317
fbshipit-source-id: 5c5ecaae858a3eaab23f624c11f0dda3ac74a870
Summary:
Previously we would only look for `/etc/scratch.toml`,
which doesn't exist on windows and would resolve ambiguously
depending on the current working directory drive letter.
This diff sets up scratch to look for a site configuration
in a similar location to where we deploy the watchman
configuration on windows systems.
Reviewed By: simpkins
Differential Revision: D15208158
fbshipit-source-id: 3b7e26f14f5b35c6fe7e196eba4e482b26bdeb01
Summary:
`visibility.add` expects to be given a list of nodes. From reset
we pass a single node which makes the underlying code break it up.
Reviewed By: singhsrb
Differential Revision: D15207234
fbshipit-source-id: 9e0dd0cc8dde0eac69d20816923b39f20963237c
Summary:
This allows us to pass flags to the step building Rust extensions.
For example, `make RFLAG=--debug local` would build extensions in debug mode,
which might take less time.
Reviewed By: singhsrb
Differential Revision: D15204831
fbshipit-source-id: 8906884e6617e20abaabb0d1f2b4e33bb9e304ba
Summary:
I wanted this feature in multiple cases. For example, I have renamed
`segdag::SegDag` to `segment::Dag`, and want to edit commit message for
`D15055347::D15055347`, a 9-diff stack. Editing them one by one is
painful.
Reviewed By: singhsrb
Differential Revision: D15188668
fbshipit-source-id: c7cc11aca0a5e16992b5246a74346a35bec00770
Summary:
Add simple algorithms to select all ancestors of a single node, or calculate a
"random" gca of two nodes.
Mercurial supports more "advanced" operations, like calculating ancestors of
multiple nodes, or calculating all ancestors of more than 2 nodes. We'll
see if those are necessary and maybe build fast or slow paths for them.
Reviewed By: sfilipco
Differential Revision: D15055347
fbshipit-source-id: c8c2bac2797d0389adb58c89b67e3ddfb62eb06f
Summary:
We now have enough building blocks to put things together.
The tests are taken from the slides. Both examples and some corner cases, plus
two maybe-interesting synthetic cases.
There are probably more details to test. But this should give us some level of
confidence.
Reviewed By: sfilipco
Differential Revision: D15055346
fbshipit-source-id: a76b70fec0ec7e88378830f251f997d147416db0
Summary:
High-level segments are built on top of lower level segments.
We simply scan through them, and greedily pick the longest ones.
Reviewed By: sfilipco
Differential Revision: D15055348
fbshipit-source-id: 3b72dc766abd46669b787187b7d1d5f7171c026a
Summary:
To build flat segments, we take a `get_parents(id) -> [id]` function and an end
`id`. Then scan through the missing ids and try to make them segments greedly.
Reviewed By: sfilipco
Differential Revision: D15055351
fbshipit-source-id: 21a503d4c3894583a314c6dfd4c7b87fafb95d95
Summary:
segment::Dag wants a `get_parents` function that speaks Ids instead of slices,
as segment::Dag works entirely on Ids.
Provide a function to translate `get_parents` on byte slices to Ids.
Reviewed By: sfilipco
Differential Revision: D15055350
fbshipit-source-id: 795367cf809f068c0cad2515af02c93e14960236
Summary:
Assigning IDs affects performance of segments. Therefore
The logic is abstracted in a way that the callsite only needs to provide a
`get_parents(slice) -> [slice]` function, and a slice to begin with. This
is intended to make it reusable for multiple cases:
- drawdag get_parents, for tests
- revlog get_parents
- Mononoke get_parents
Reviewed By: sfilipco
Differential Revision: D15055349
fbshipit-source-id: d6475737eb87f5ab7d7bd8123a8f4ae2b6d108e8
Summary:
This library parses an ASCII DAG. It is similar to mercurial/drawdag.py, which
was added by me in [1].
There are some (intentional) differences from the Python drawdag:
- Stricter. Confusing DAG characters like `+` or crossing lines are forbidden.
- Do not special handle `o` as a name.
- Do not try to be compatible with `hg log -G` output.
- Do not support special comments (yet).
- Support both left to right and bottom to top directions.
This library tries to be abstract. i.e. it does not have actual logic about
how to make a commit. Its intended users are Mononoke and scmdag, which have
different ways to make commits.
Since this is a library that is intended to be used only for tests. I didn't spend too
much effort to optimize its performance.
[1]: https://www.mercurial-scm.org/repo/hg/rev/a31634336471
Reviewed By: kulshrax
Differential Revision: D15039768
fbshipit-source-id: 4c33d44759ecf59aadc3d443a84db07d702dc69b
Summary:
The segment::Dag structure stores all levels of segments. The "segment" concept
is introduced by D14937221.
This diff adds empty structures and the serialization format.
Reviewed By: sfilipco
Differential Revision: D15019662
fbshipit-source-id: 8136acd45dc8526391e94c5ae98b609d4f8b392a
Summary:
This updates our apply updates logic to record how many files and how
many bytes were written to disk.
Reviewed By: quark-zju
Differential Revision: D15173846
fbshipit-source-id: c26d2b17f45e407f837c78c93fc440e960d4f922
Summary:
We already had tracing for tree prefetches, but in some cases the
fetching happens ondemand and goes through a different code path. Now that we
have pack file receive size tracking, let's annotate this appropriately.
Differential Revision: D15172482
fbshipit-source-id: 2db1dc8464ed9e0b8f6987718c448e391744f445
