Summary: The edenapi is now independant of the storage type for history data.
Reviewed By: kulshrax
Differential Revision: D15284355
fbshipit-source-id: 72a5db42bb0fb19ee03155b13914202581ab5966
Summary:
The python mutablehistorypack can implement MutableHistoryStore, so let's do
it.
Reviewed By: kulshrax
Differential Revision: D15284357
fbshipit-source-id: e55818e2432573f0fbe4f5b73c592f7f6418785c
Summary:
This allows for a MutableHistoryPack to be used where a MutableHistoryStore
will be required. Once an IndexedLog based history store is implemented we will
be able to switch between the 2 more easily.
Reviewed By: kulshrax
Differential Revision: D15284356
fbshipit-source-id: 91d75ddc6991c26eace67d77679bb8d5806cf8b8
Summary: This will help in abstracting the kind of store that is being written to.
Reviewed By: kulshrax
Differential Revision: D15284358
fbshipit-source-id: ab6a6d23978480ca65587b745ae39ac6ed98cca9
Summary:
The type of store where data is stored is now fully abstracted to the python
bindings. For now, edenapi will write to the pending mutabledatapack, but we
can now switch it easily to any other store implementing MutableDeltaStore,
including an IndexedLogDataStore.
Reviewed By: kulshrax
Differential Revision: D15266191
fbshipit-source-id: 638cf90a567ef170e0302376312c4b82e6d6b6da
Summary:
The python mutabledatapack is a delta store, and therefore we can implement the
MutableDeltaStore trait for it.
Reviewed By: kulshrax
Differential Revision: D15266192
fbshipit-source-id: 6cc04e4a5e792d037d17c706a374f6864a2902a9
Summary:
This will allow to transparently use the IndexedLogDataStore or a datapack in
the edenapi code.
Reviewed By: kulshrax
Differential Revision: D15266194
fbshipit-source-id: 6396118a5c8107a8c91e5fc83fe4297d4321d10c
Summary:
This will be used to abstract writing to a MutableDataPack or
IndexedLogDataStore (or both).
Reviewed By: kulshrax
Differential Revision: D15266193
fbshipit-source-id: 99f2383555addbafea81a2752e8d6759a1c1c5e7
Summary: As we add more functionality to the Eden API, we will have a lot more request structs. These structs are only used by the HTTP data fetching code, and should not be used by actual business logic. As such, while these types need to be public (so that both Mononoke and Mercurial can use them), they should not be re-exported at the top level.
Reviewed By: quark-zju
Differential Revision: D15268439
fbshipit-source-id: e7d1405d2ac234892baedbf7dbf3e133d187cb45
Summary: Adds tracing for mergedriver and for the individual generators.
Reviewed By: quark-zju
Differential Revision: D15223926
fbshipit-source-id: c0c6e1a6814f59fda3ddd75eee901a60a2335c1b
Summary:
Set the `component` to `"commitcloud"` for commit cloud statuses and
messages, rather than using custom highlight functions.
Reviewed By: quark-zju
Differential Revision: D15201944
fbshipit-source-id: 7635942a5ca029209711a2b89c32cc5fd677d22f
Summary:
Add optional prefix keyword arguments to `ui.write` (and thus `ui.status`,
`ui.warn`, etc.). These prefixes can be used to indicate an error condition,
some notice to the user, or to mark a message as coming from a particular
component. The prefixes are labelled so that they can be colored to stand
out from the surrounding text. The default colors are red for errors, yellow
for notices, and cyan for components.
Add a component parameter to `error.Abort` (and thus any exception that
inherits from it) that can be used by callers to mark which component is
responsible for the exception. When the error is printed, the component is
passed as the component prefix of the error message.
Reviewed By: mitrandir77
Differential Revision: D15201946
fbshipit-source-id: 02e3da40e9563704fa1d7ce81366e6e7f66c7f34
Summary:
D15188668 added the `--batch` option to the `metaedit` command along
with updating the tests. Unfortunately, the test is broken on OSX. This commit
fixes the test to be compatible with OSX.
Differential Revision: D15273960
fbshipit-source-id: 57a67b9d7ab2b471ee205db4fdfc7f3e6e2e553b
Summary:
While closing the repo, hg_memcache_client is terminated, and later, the
pending packfiles are commited. Unfortunately, the commit phase is also when
the content of the pending packfiles will be sent to memcache, and since the
connection is at this point closed, nothing is sent then.
Reviewed By: quark-zju
Differential Revision: D15247639
fbshipit-source-id: 8719cb8bbc3c9571b9cfe250ba4be2b745e4ba15
Summary:
A tree manifest entry must always end with a line feed. It is somewhat
redundant but that's how the serialization is defined. Sometimes that last
line feed is missing in our data. I don't know why.
Reviewed By: quark-zju
Differential Revision: D15110860
fbshipit-source-id: c4ac5075e22a8b8851f6b246d22af8ab68f42a74
Summary:
This is a quality of life improvement for working with the storage layer.
We probably don't gain a whole lot by statically linking the store and it is
useful to have some flexibility in the storage layer.
Differential Revision: D15110859
fbshipit-source-id: 6102acafa21dd1dbaeed0f8fc3147538a8c301d1
Summary:
With the mutablestores, this code was starting to be too messy and hard to
follow. Let's simplify it a bit.
Reviewed By: quark-zju
Differential Revision: D15200748
fbshipit-source-id: 2c0cc5a4ff5057f7a9590fcc602afc2b1cc72dcd
Summary:
The `finally` block relies on this name being in scope. If the exception
happens before this name is initialized, the finally block also throws and we
loose the original root cause exception.
Reviewed By: singhsrb, HarveyHunt
Differential Revision: D15244583
fbshipit-source-id: fd8be11987c3028f775a62cb3ae308b30a9d5e75
Summary:
If a chg server was started by a low priority process, it will inherit the low
priority and can be painfully slow.
On the other hand, if the chg client was low priority, the user or the automation
wants the process to be low priority intentionally. Therefore connecting to an
existing chg server might bypass intention.
Therefore, detect the low priority case, and refuse to start or connect to a
chg server.
Previous attempts to workaround slow chg server were removed as they are
probably no longer necessary.
Reviewed By: singhsrb
Differential Revision: D15223518
fbshipit-source-id: 718e807820d481eac204f2293f949c3b315f923d
Summary:
In a previous diff I did a last minute refactor that apparently broke a
test. I needed to update this line as well to swap the arguments.
Reviewed By: singhsrb
Differential Revision: D15219418
fbshipit-source-id: c66a2cf7bbbec80ebbbd6ca818151de5f0c2d2c5
Summary: Update the tests to expect the new output added by D15173846.
Reviewed By: quark-zju
Differential Revision: D15212315
fbshipit-source-id: 82bb3b4e67a28eb8905d35fcaa8251947163f521
Summary:
When remotefilelog.fetchpacks is enabled, it's possible that 100 packfiles of
100MB each are present. In this case, every new packfiles that
hg_memcache_client would write will force an incremental repack, which will
only reduce the number of packfiles by a small number.
Let's have a simple heuristic that tries to bring the number of packfiles to be
lower than 50.
Reviewed By: DurhamG
Differential Revision: D15203771
fbshipit-source-id: 18c39487d5ac087d4879004993c1c1add087249c
Summary: Update the test code to return a 3-tuple as required after D15173846.
Reviewed By: singhsrb
Differential Revision: D15212317
fbshipit-source-id: 5c5ecaae858a3eaab23f624c11f0dda3ac74a870
Summary:
Previously we would only look for `/etc/scratch.toml`,
which doesn't exist on windows and would resolve ambiguously
depending on the current working directory drive letter.
This diff sets up scratch to look for a site configuration
in a similar location to where we deploy the watchman
configuration on windows systems.
Reviewed By: simpkins
Differential Revision: D15208158
fbshipit-source-id: 3b7e26f14f5b35c6fe7e196eba4e482b26bdeb01
Summary:
`visibility.add` expects to be given a list of nodes. From reset
we pass a single node which makes the underlying code break it up.
Reviewed By: singhsrb
Differential Revision: D15207234
fbshipit-source-id: 9e0dd0cc8dde0eac69d20816923b39f20963237c
Summary:
This allows us to pass flags to the step building Rust extensions.
For example, `make RFLAG=--debug local` would build extensions in debug mode,
which might take less time.
Reviewed By: singhsrb
Differential Revision: D15204831
fbshipit-source-id: 8906884e6617e20abaabb0d1f2b4e33bb9e304ba
Summary:
I wanted this feature in multiple cases. For example, I have renamed
`segdag::SegDag` to `segment::Dag`, and want to edit commit message for
`D15055347::D15055347`, a 9-diff stack. Editing them one by one is
painful.
Reviewed By: singhsrb
Differential Revision: D15188668
fbshipit-source-id: c7cc11aca0a5e16992b5246a74346a35bec00770
Summary:
Add simple algorithms to select all ancestors of a single node, or calculate a
"random" gca of two nodes.
Mercurial supports more "advanced" operations, like calculating ancestors of
multiple nodes, or calculating all ancestors of more than 2 nodes. We'll
see if those are necessary and maybe build fast or slow paths for them.
Reviewed By: sfilipco
Differential Revision: D15055347
fbshipit-source-id: c8c2bac2797d0389adb58c89b67e3ddfb62eb06f
Summary:
We now have enough building blocks to put things together.
The tests are taken from the slides. Both examples and some corner cases, plus
two maybe-interesting synthetic cases.
There are probably more details to test. But this should give us some level of
confidence.
Reviewed By: sfilipco
Differential Revision: D15055346
fbshipit-source-id: a76b70fec0ec7e88378830f251f997d147416db0
Summary:
High-level segments are built on top of lower level segments.
We simply scan through them, and greedily pick the longest ones.
Reviewed By: sfilipco
Differential Revision: D15055348
fbshipit-source-id: 3b72dc766abd46669b787187b7d1d5f7171c026a
Summary:
To build flat segments, we take a `get_parents(id) -> [id]` function and an end
`id`. Then scan through the missing ids and try to make them segments greedly.
Reviewed By: sfilipco
Differential Revision: D15055351
fbshipit-source-id: 21a503d4c3894583a314c6dfd4c7b87fafb95d95
Summary:
segment::Dag wants a `get_parents` function that speaks Ids instead of slices,
as segment::Dag works entirely on Ids.
Provide a function to translate `get_parents` on byte slices to Ids.
Reviewed By: sfilipco
Differential Revision: D15055350
fbshipit-source-id: 795367cf809f068c0cad2515af02c93e14960236
Summary:
Assigning IDs affects performance of segments. Therefore
The logic is abstracted in a way that the callsite only needs to provide a
`get_parents(slice) -> [slice]` function, and a slice to begin with. This
is intended to make it reusable for multiple cases:
- drawdag get_parents, for tests
- revlog get_parents
- Mononoke get_parents
Reviewed By: sfilipco
Differential Revision: D15055349
fbshipit-source-id: d6475737eb87f5ab7d7bd8123a8f4ae2b6d108e8
Summary:
This library parses an ASCII DAG. It is similar to mercurial/drawdag.py, which
was added by me in [1].
There are some (intentional) differences from the Python drawdag:
- Stricter. Confusing DAG characters like `+` or crossing lines are forbidden.
- Do not special handle `o` as a name.
- Do not try to be compatible with `hg log -G` output.
- Do not support special comments (yet).
- Support both left to right and bottom to top directions.
This library tries to be abstract. i.e. it does not have actual logic about
how to make a commit. Its intended users are Mononoke and scmdag, which have
different ways to make commits.
Since this is a library that is intended to be used only for tests. I didn't spend too
much effort to optimize its performance.
[1]: https://www.mercurial-scm.org/repo/hg/rev/a31634336471
Reviewed By: kulshrax
Differential Revision: D15039768
fbshipit-source-id: 4c33d44759ecf59aadc3d443a84db07d702dc69b
Summary:
The segment::Dag structure stores all levels of segments. The "segment" concept
is introduced by D14937221.
This diff adds empty structures and the serialization format.
Reviewed By: sfilipco
Differential Revision: D15019662
fbshipit-source-id: 8136acd45dc8526391e94c5ae98b609d4f8b392a
Summary:
This updates our apply updates logic to record how many files and how
many bytes were written to disk.
Reviewed By: quark-zju
Differential Revision: D15173846
fbshipit-source-id: c26d2b17f45e407f837c78c93fc440e960d4f922
Summary:
We already had tracing for tree prefetches, but in some cases the
fetching happens ondemand and goes through a different code path. Now that we
have pack file receive size tracking, let's annotate this appropriately.
Differential Revision: D15172482
fbshipit-source-id: 2db1dc8464ed9e0b8f6987718c448e391744f445
Summary: This will let us get a feel for download speeds.
Reviewed By: kulshrax
Differential Revision: D15168153
fbshipit-source-id: 86291b8eb3393d54271d574250291ed691a64a86
Summary: Records how many memcache misses we hit.
Reviewed By: kulshrax
Differential Revision: D15168154
fbshipit-source-id: 53e105999d0af566666fcfaecc5bcaf414c5e804
Summary:
Adds tracing support for recording how many bytes we download for lfs,
including the download speed.
Reviewed By: kulshrax
Differential Revision: D15167587
fbshipit-source-id: fb76bbf15cd44dcef2601d9e750794816d273120
Summary:
The prefetch code was not refreshing the pack stores after writing
stuff into packs. This meant that in some cases we would execute a prefetch,
then when we went to read an entry from the store it wouldn't be found. That
would trigger a one-by-one download, which would happen to trigger a pack store
refresh (since remotecontentstore had a refresh) which would cause the rest of
the prefetched data to then become visible.
This fixes it to make prefetch responsible for refreshing the pack data.
This also caused issues where prefetching lfs files requires the hg fetching to
get the metadata, then it reads that metadata and does an lfs fetch. Since the
original metadata wasn't visible (since the packs had not refreshed), this
resulted in infinite recursion until the default 100ms refresh time passed.
Differential Revision: D15171701
fbshipit-source-id: 190b1da542675265efaad75a2a6826cbe3c9631c
Summary:
`bug29988wrapper` is a non-camel-case type, which fits the Mercurial Python
code style, but the Rust compiler warns about it. Suppress the Rust warning.
Reviewed By: quark-zju
Differential Revision: D15197827
fbshipit-source-id: 54d0b4020f8e43e3bc24f9ab37e0ede55dc919e9
Summary:
File that tracks accessed bookmarks doesn't interact with anything else and doesn't really need repo.wlock. However if does use wlock, hg commands like `hg log -r`, which normally are read-only, start waiting for the lock and thus are blocked by other writing commands, that might be running in different checkouts of the same repo.
Let's use a different lock for this feature.
Reviewed By: simpkins
Differential Revision: D15165788
fbshipit-source-id: f04c7196d51db67069c6420545be24d2b7c0af27
Summary:
`infinitepush.sqlindex` depends on `mysql.connector`, which is only needed on
infinitepush servers using the sqlstore, and may not be otherwise available.
Delay the import of this module until we know for sure that it is needed.
Reviewed By: ikostia
Differential Revision: D15197758
fbshipit-source-id: 34f166ceab2530c875a4539089e544755f199b7b
Summary:
Do not keep all revlogs in memory. Detect memory usage and drop filelogs on
demand.
Reviewed By: ikostia
Differential Revision: D15186221
fbshipit-source-id: 8311d479e8dd4f8f449981873a98940410c20c9e