Summary:
In the Mononoke integration tests (which are .t tests), we've started seeing
this waring show up.
We probably shouldn't have mutation enabled for most of those tests anyway
(save for the tests that actually exercise mutation), so I'm turning that off.
That said, we'd still see the warning. So, this diff refactors mutation a
little bit to only warn if we do have something to send to the server.
Reviewed By: ikostia
Differential Revision: D22016685
fbshipit-source-id: e9e8dbcc44a0fa048dca3ca7ce0961631058a2d6
Summary: Last usage was removed in D20342689.
Reviewed By: xavierd
Differential Revision: D22011195
fbshipit-source-id: 58455ad3dc77438704e80c69eb728a319ee71a39
Summary: In its current state, FORCE is a restart mode that cannot be combined with other modes (specifically `graceful`). Removing this coupling from `force` will allow us to at first attempt a graceful restart, but if the daemon is not healthy, then do a force restart, all within one call.
Reviewed By: wez
Differential Revision: D21873166
fbshipit-source-id: 079a777ff6e7b219198cbef15335244b9865f5f3
Summary:
Move the LFS server's `StreamBody` into `gotham_ext`, along with some changes to decouple it from the LFS server.
In particular, the `Content-Length` header and post-request `Sender` have been made optional fields that can be set via a builder-style interface. The LFS server's `StreamBody` has been renamed to `LfsStreamBody` and is now a thin wrapper around `gotham_ext`'s `StreamBody` that preserves the old behavior.
Reviewed By: krallin
Differential Revision: D21988855
fbshipit-source-id: a9bf9c04bb791388d761fc705ebc38472a713b65
Summary: I would like to use these utilities when building segmented changelog.
Reviewed By: krallin
Differential Revision: D21876432
fbshipit-source-id: 9022627e224bfcb155b47d696371d24e538e6f39
Summary:
Useful for keeping the insert logic working nicely even when we double insert
or have other inconsistencies.
Reviewed By: krallin
Differential Revision: D21853048
fbshipit-source-id: 958cbc0435f330749c5aae1ea0f41ecd03b01468
Summary:
If the caller provides a service identity in `repo_create_commit`, use that to obtain
write access to the repository.
Reviewed By: StanislavGlebik
Differential Revision: D21995724
fbshipit-source-id: ba218e392b979938d598a37494b0ffe496f28722
Summary: Now that apiserver is gone, we can remove the legacy parts of mononoke_api.
Reviewed By: krallin
Differential Revision: D21976615
fbshipit-source-id: 139d7ebd4a7185693f6a4fe4e5c234c143f18498
Summary:
Add the `move_bookmark` method to `mononoke_api`.
This attempts to re-use code from `repo_client`. This code isn't really designed to be called from this API, so the fit is very poor. In a future diff we will fix this up.
The `repo_client` code for force-moving a bookmark does not support running hooks for that bookmark. For now we will prevent API users from moving bookmarks that have hooks configured. This will also be addressed in a future diff.
Reviewed By: krallin
Differential Revision: D21904979
fbshipit-source-id: 42bf840489e5b04f463c69c752bcaa5174630c21
Summary:
There is no need to migrate if the obsstore is a proxy to the mutation store.
This avoids a crash reading the attribute `_all`.
Reviewed By: DurhamG
Differential Revision: D21999245
fbshipit-source-id: bfa6439982834c6835412fc175132526525af548
Summary: The test breaks, let's not enable mutation for it.
Reviewed By: DurhamG
Differential Revision: D21998151
fbshipit-source-id: 9c06485287d906837444f39faf90886933decc9b
Summary:
Some of the hg changeset structures use `comments` as the name for commit
message. But this is confusing - let's rename
Reviewed By: HarveyHunt
Differential Revision: D21974571
fbshipit-source-id: e7c5c5ad8db9b2f1343abe9101fc56a6d4287548
Summary:
Unawaited coroutines are pernicious and not errors by default. In
advance of enabling asynchronous tests, fail our tests when Python
warnings are shown.
We record warnings and report them after the test body instead of
enabling an "error" warning filter because exceptions thrown by
finalizers are silently swallowed, and most important warnings occur
during finalization.
Reviewed By: genevievehelsel
Differential Revision: D21955210
fbshipit-source-id: b1fc52ddfa9f9befbde6800f85f090862af0ef15
Summary:
The remotenames extension was enabled everywhere in production. It is
unsupported if remotenames is disabled.
Ideally the cpp tests should use the in-repo configuration instead of
maintaining its own hardcoded hgrc. For now I just did the minimal change to
unblock tests.
Reviewed By: fanzeyi
Differential Revision: D21986274
fbshipit-source-id: b632da074baf4026f331381ae8f2f8479a34e1b3
Summary:
When kicking off the background process, we were depending on the
current working directory being inside the repo. This is not always true, like
when the current command was invoked like `hg -R xxx log -r .`. Let's invoke the
debugdynamicconfig command with the explicit path.
Reviewed By: quark-zju
Differential Revision: D21983297
fbshipit-source-id: 72981ea596e62dced3918f2fc6272953ab3d84cd
Summary:
Treemanifest was recently enabled in this test, but it breaks the Windows
tests. For now, let's just go back to the previous state and we can investigate
later.
Differential Revision: D21989349
fbshipit-source-id: fb3d3d1315400f56a6166a01112affc0a4d96da6
Summary:
Somehow clone-dst isn't empty which makes the second clone to fail due to it,
let's just clone it elsewhere.
Differential Revision: D21989189
fbshipit-source-id: 125f33f3f498d0c089785018e39ee7c5b795b472
Summary: Those tests only need minor changes to be compatible with mutation.
Reviewed By: sfilipco
Differential Revision: D21913606
fbshipit-source-id: 0a2b0e08cb558960a7d0f938300f375c34457a62
Summary:
Incompatible tests are blacklisted.
The affects non-tests environment: `proxy-obsstore` is turned on.
That is in theory not very different from having an empty obsstore.
If it becomes an issue we can easily turn it off.
Reviewed By: sfilipco
Differential Revision: D21899139
fbshipit-source-id: e4a87ecced0542e7852b48b1a807775e523b3cc9
Summary: Those tests can be migrated with minor changes like rev number fixups.
Reviewed By: sfilipco
Differential Revision: D21913589
fbshipit-source-id: 0ee080ea9d70ded74c07ce6a93d309437b992bd6
Summary: Some `x` commits become `o` because they don't have successors.
Reviewed By: sfilipco
Differential Revision: D21913601
fbshipit-source-id: 8d23edd4daa948433969460be9e6488865be7fb0
Summary:
I found myself friendly editing the output of run-tests.py:
Failed test-failure.t: output changed
Failed test-failure-unicode.t: output changed
to extract file names:
test-failure-unicode.t
test-failure.t
Then use the file names as parameters to other commands, like `run-tests.py`
again, or `hg revert`, or `hg shelve`.
Change the output format to print the filenames directly to make it easier to
use.
Reviewed By: sfilipco
Differential Revision: D21913609
fbshipit-source-id: 4b1d2265f384ad9afbe478b7ff0d6e543eff00df
Summary: We're getting rid of obsstore. The extension is no longer needed.
Reviewed By: sfilipco
Differential Revision: D21899142
fbshipit-source-id: 6e4dd2b2ddfa9325c19d6d1d9951099748260904
Summary:
The feature is off in production.
It calls the "scan changelog" code path in pullcreatemarkers.py, which
is also unused in production.
This allows us to remove a test that no longer passes with the mutation config.
Remove it.
Reviewed By: sfilipco
Differential Revision: D21913600
fbshipit-source-id: 23850fca2d1dc3d135d25954f00a08dc6d13972e
Summary:
The test uses `obsolete.revive`, which is no longer supported.
The alternative is to use unhide / undo which revives commits by changing
visibleheads directly.
Reviewed By: sfilipco
Differential Revision: D21913591
fbshipit-source-id: fea8c00692ba84c3a079012176970efb5aa29fae
Summary: We're migrating off obsstore. The test is no longer relevant.
Reviewed By: sfilipco
Differential Revision: D21913592
fbshipit-source-id: 8a721a9eea19ad04027f014727cdf0e6fb9a6b60
Summary:
The test is mostly about obsmarkers and is not that valuable with mutation.
Delete it since we do have mutation + histedit test coverage.
Reviewed By: sfilipco
Differential Revision: D21913594
fbshipit-source-id: 8c785a698fcf7c0d9f947e961c66960c13ea1111
Summary:
The old code only compares successors without checking predecessors, and skips
creating markers even if predecessors do not match. Update it to also check
predecessors. This is needed to maintain the `x` commits in
test-fb-hgext-pull-createmarkers-hide-later.t when switched to use mutation.
Reviewed By: markbt
Differential Revision: D21913610
fbshipit-source-id: 1fec2f96544236dd675ceb8c1a399bcef6cd0ede
Summary:
The test is coupled with legacy logic (obsmarker, phase exchange, over
ssh/http, bundle1, etc.), is flaky (6%-10% flaky score), and hard to maintain.
It provides little value since we probably want Thrift / Serde / HTTP protocols
instead of improving bundle2. Therefore remove it.
Reviewed By: sfilipco
Differential Revision: D21913603
fbshipit-source-id: 9d3d7410e60b67aadf816c917949772f991d2e65
Summary:
They are no-op with mutation, which is enabled in production. Therefore remove
the template keywords and tests.
Reviewed By: sfilipco
Differential Revision: D21913602
fbshipit-source-id: 40ef3a915d591d6bcae2ca7931e388b5d5e00c25
Summary:
Pruned commits are handled by visibility. Ignore them when calculating
`obsolete()`.
This change makes transition to mutation smoother. The latter does not allow
obsoleting commits without successors.
Practically, this means some `x` commits will show as `o`s.
Reviewed By: sfilipco
Differential Revision: D21913596
fbshipit-source-id: 8a6495d040595ead1f6f08cf81e095e25cd8ff1f
Summary: This makes transition to mutation smoother.
Reviewed By: sfilipco
Differential Revision: D21913599
fbshipit-source-id: 74cc0b7f0608a952eded4a40291b56b291a221e5
Summary:
The code path printing that message is rarely used in production setup. This
makes transition to mutation easier.
Reviewed By: sfilipco
Differential Revision: D21913595
fbshipit-source-id: 3aa22575c11e38dbb7d4c4cbc7aad8dde2e755ed
Summary: This makes transition to mutation smoother.
Reviewed By: sfilipco
Differential Revision: D21913608
fbshipit-source-id: b9ba470572ed68dff75104bba9dbb5c8622dcfb0
Summary:
The summary callback is only used to display "obsoleted n changesets".
Remove it to make transition to mutation smoother.
Reviewed By: sfilipco
Differential Revision: D21913597
fbshipit-source-id: e47fa9df0033d1302ffc8dfa3b9885eb75861f4b
Summary: This is needed to enable mutation and disable the old obsstore in more places.
Reviewed By: sfilipco
Differential Revision: D21913607
fbshipit-source-id: 2e1ef1e7df58fb134d541f550b6fb90508b9289a
Summary:
This makes code relying on `repo.obsstore` work with the new mutation backend.
Namely, commands like `debugobsolete` will work in tests.
Reviewed By: sfilipco
Differential Revision: D21899149
fbshipit-source-id: b4906ededdc32cfd9c9b7efbf2c85600a0c4a8f4
Summary:
These particular unit tests essentially just verify that these types (which are trivial wrapper structs around `Vec`s) implement `IntoIterator`. As such, they don't really provide value (the `IntoIterator` implementations are trivial*) but require updates whenever the format of the items produced by the iterators changes. This adds a maintenance burden with little value to show for it, so let's just get rid of these.
*The only thing non-trivial is that for `HistoryResponse`, the iterator does convert the entries from `WireHistory` to `HistoryEntry`, but the actual logic of this conversion is tested separately in `api.rs`, so testing it here is redundant.
Reviewed By: quark-zju
Differential Revision: D21983471
fbshipit-source-id: beebad53abb9ce5801b1738af92f250405c50990
Summary:
This diff fixes `eden rage` command on Windows.
`eden doctor` and `eden redirect` related rage items are disabled as they are not implemented on Windows yet.
Reviewed By: chadaustin
Differential Revision: D21962673
fbshipit-source-id: 14399a89c26eead8fe5114e0a3764644ee86a1ed
Summary:
Some Thrift clients require that values of type `string` must contain
valid UTF-8, and fail to decode otherwise. Ensure that EdenError's
message is always valid UTF-8.
Reviewed By: fanzeyi
Differential Revision: D21890761
fbshipit-source-id: 93b554e0bc6648c685f5fa54612576bd95d95ae5
Summary:
Add some functions for validating and producing valid UTF-8 to be used
in an upcoming diff.
Reviewed By: fanzeyi
Differential Revision: D21890510
fbshipit-source-id: b25144a34f1df91c72e8ed776b1ee7c1d68344c8
Summary: `getpack` requires list and will crash on set.
Reviewed By: xavierd
Differential Revision: D21968128
fbshipit-source-id: 75956f5adfcdb8082608e65f041007e58fc36c5c
Summary:
The documentation for PRJ_NOTIFICATION_FILE_RENAMED points out that the
filenames can be empty strings to represent files moved to and from the
repository itself. By not special casing this we were trying to create and/or
remove an empty file at the root of the repo which would lead to an overlay
corruption.
Reviewed By: genevievehelsel
Differential Revision: D21961579
fbshipit-source-id: fbfd872ca0377cb90224eba5505ff406812d51a7
Summary:
Reading the 00changelog.i entirely can be very expensive both in time and
memory. Since the 00changelog.i is now copied, then truncated and copied back
in place, we no longer have the limitation that prevents Windows from using
mmap for it, so let's do it.
Reviewed By: quark-zju
Differential Revision: D21968805
fbshipit-source-id: c0d19593e1f565c93b9243a63c87d03cfbc821d8
Summary: This diff fixes `'__isset' is deprecated` problem when building `//eden/fs/service:edenfs`
Reviewed By: simpkins
Differential Revision: D21966922
fbshipit-source-id: c3b12134f3f84db0a97f3975dae5d64a2c98641a
Summary: It was used by APIServer only, now when it's decomissioned we can delete this.
Reviewed By: HarveyHunt
Differential Revision: D21954670
fbshipit-source-id: a16bbbbb9dfe17eda5ff1ee6267251708ee4d562
Summary:
Enables treemanifest for about 100 more tests. To make them pass I had
to expose the gettreepack capability from peers, which just works, and enable
treeonly and sendtrees by default in the tests, which I probably should've done
before anyway.
Reviewed By: quark-zju
Differential Revision: D21894955
fbshipit-source-id: 7a6a4d453824fb8c81a797a5487bf2ecc2b67761
Summary: This diff resolves TODO to make a helper function a crate private function as the only dependency (APIServer) is now gone.
Reviewed By: StanislavGlebik
Differential Revision: D21954238
fbshipit-source-id: 7cfc49534b7ab8af6035f7a87fdc763127dce318
Summary:
APIServer is deprecated, tw jobs stopped and deleted.
This diffs removes the source code.
Reviewed By: krallin
Differential Revision: D21839442
fbshipit-source-id: 5a4089d9205d8b0061c8aa01dcd74674fe9baca8
Summary:
On Windows, if a file is memory mapped it cannot be truncated, unfortunately,
with the 00changelog.i being almost 500MB, this means that every time Mercurial
starts, the file needs to be read entirely. With the code being moved to Rust,
this read is done twice, resulting in a memory consumption of over 900MB.
Instead of always trying truncate the file in place, let's just try to truncate
it first, and then copy it, truncate it and then move it back in place. This
will allow us to memory map the 00changelog.i on Windows, which should
significantly reduce the memory consumption of Mercurial.
Reviewed By: DurhamG
Differential Revision: D21953553
fbshipit-source-id: e0f447fea017a34c8da8cdbbc6914cee7b5b2bee
Summary:
We need to support pushing legacy flat manifets commits without push
rebase. Currently we rebuild the tree on the server and recompute it's hash,
which can result in a tree hash instead of a flat hash for the pushed commit.
Since we're provided with the correct hash via the push, we should just use that
directly.
Since the hash isn't validated, it opens some opportunities for receiving
incorrect data from the server, but we can disable this functionality at some
point, once all legacy repos have been merged in. Then we can get rid of the
need for flat manifest hashes entirely once we switch to a new hash scheme.
Reviewed By: quark-zju
Differential Revision: D21962513
fbshipit-source-id: 8cdaf3a4e2aba73d5fc97fcb78821d2c88f5284f
Summary: D21880220 renamed the `depth` parameter in Mononoke's history fetching functions to be `length`. This diff makes the same change for EdenAPI's `HistoryRequest` struct.
Reviewed By: StanislavGlebik
Differential Revision: D21948599
fbshipit-source-id: fe8649a5789f07d8262ad3d5e2f477a8b50f2c6f
Summary:
Add a new `subtree` endpoint to the EdenAPI server, which fetches trees using the underlying server-side logic for the `gettreepack` wire protocol command. This is intended for use in situations where compatibility with `gettreepack` is desired when using HTTP for tree fetching.
The name of the endpoint is up for bikeshedding. It seemed weird to name the endpoint `gettreepack` since that name is a verb and refers to a "pack" which is not relevant in this context (there are no wirepacks or packfiles involved). I chose the name `subtree` since the endpoint logically returns all of the nodes underneath a given node in the tree (though in the most frequent case, the node will be the root node and therefore the subtree will be the entire tree).
In practice, this initial implementation is not ideal because it buffers the trees in memory, which is problematic because `gettreepack` requests are likely to produce a very large number of trees. Later in this stack, the endpoint will be updated to produce a streaming response instead.
Reviewed By: StanislavGlebik
Differential Revision: D21782764
fbshipit-source-id: 726925858352c33c923da1979da9d20fbcf930f6
Summary:
Several of the structs used by EdenAPI were previously defined in Mercurial's
`types` crate. This is not ideal since these structs are used for data interchange
between the client and server, and changes to them can break Mononoke, Mercurial,
or both. As such, they are more fragile than other types and their use should be
limited to the EdenAPI server and client to avoid the need for extensive breaking
changes or costly refactors down the line.
I'm about to make a series of breaking changes to these types as part of the
migration to the new EdenAPI server, so this seemed like an ideal time to split
these types into their own crate.
Reviewed By: krallin
Differential Revision: D21857425
fbshipit-source-id: 82dedc4a2d597532e58072267d8a3368c3d5c9e7
Summary: There's over 150 lines of code here, or around 1/3rd of lib.rs. Move to a separate module, so that growth of tests won't make lib.rs too long
Reviewed By: ahornby
Differential Revision: D21952798
fbshipit-source-id: 4e4edf4fde5a6626e2c33c8ea30ec85e6f3c6bf1
Summary: These were minimal conversions of old-style tests - use `.compat()` so that we have "true" async tests.
Reviewed By: ahornby
Differential Revision: D21950533
fbshipit-source-id: 71608201c4fe475d07b9ff3fafa138036f3dad5b
Summary: We used to inline smaller blobs into the data table, and only chunk bigger blobs. Performance tests show that we don't have significant issues with round trips to our MySQL infrastructure, so let's simplify the problems of deduplication and hard linking by always storing the data in chunks, and having the keys directly return significant data.
Reviewed By: ahornby
Differential Revision: D21939500
fbshipit-source-id: 6ad73b25ac859fb1b4f067a3016516b713e8b2f5
Summary: [Folly] Cut the FOR_EACH_KV macro, which may be replaced by a combination of range-for and structured bindings.
Differential Revision: D21826182
fbshipit-source-id: ce4712afd3d0d7806eb1fca8c97009da117f982e
Summary:
Add a disabled test that illustrates that binary data fails to
serialize between edenfs and the CLI with the current Thrift
implementation.
Reviewed By: fanzeyi
Differential Revision: D21890379
fbshipit-source-id: 725e99751e1d62c39f4059ec0f1197857c47e3b4
Summary:
The Python 2-and-3 Thrift API is sort of deprecated and does not
handle binary data in `binary` fields. In advance of migrating to the
modern Python 3 API, remane eden.thrift to eden.thrift.legacy.
Reviewed By: fanzeyi
Differential Revision: D21889697
fbshipit-source-id: a745ee8977999acbfb383a4edebe81d8deb1734e
Summary: Now that the blobstore_sync_queue has no unique key we do not expect duplicate key errors, and would like to know of any other problems occuring on insert.
Reviewed By: StanislavGlebik
Differential Revision: D21618062
fbshipit-source-id: 85991184234a807553388b456ad76971104000a3
Summary: I made a change before landing my last diff and didn't retest.
Reviewed By: xavierd
Differential Revision: D21944268
fbshipit-source-id: a5c820cef5b0fc019c7be5d096db9d0e7b3e5a44
Summary:
Introduced a new local commit metadata storage system, smallcommitmetadata, which stores a simple local JSON file for a mapping of (node, category) -> value.
This data can be manipulated with the debugsmallcommitmetadata command, and can be consumed in the smartlog using the smallcommitmeta template func.
Total number of entries can be limited with smallcommitmetadata.entrylimit configuration option, which defaults to 100.
Reviewed By: DurhamG
Differential Revision: D21673611
fbshipit-source-id: 2239a47867118dd86b15944058505ddf67548549
Summary:
Let's record the caller of the hg command that's using deprecated
logic. In most cases the caller will be the unix username, but we'd like to get
services to set an environment variable with their oncall so we could associate
hg usage with specific oncalls and eventually automatically file tasks for using
deprecated functionality.
Reviewed By: markbt
Differential Revision: D21891753
fbshipit-source-id: a4de0d348c583777cc3eeb9be26510e6af8b3a27
Summary:
Marks a number of commands as deprecated. The default ui.deprecate()
behavior is to just log the usage to scuba, so this diff doesn't actually change
any behavior.
Reviewed By: markbt
Differential Revision: D21891192
fbshipit-source-id: 4611e31cf630eee4f07443fe242c759c3cb3fe31
Summary:
Adds a simple ui.deprecate() function for deprecating code paths. It
has several levels of deprecation and can optionally automatically escalate
through those levels across a specified time frame.
The levels are:
- Log - just logs the usage to scuba
- Warn - same as Log but also prints a warning to the user
- Slow - same as Warn but also sleeps for 2 seconds
- OptIn - throws an exception, but hints that the exception can be bypassed by
setting 'deprecated.bypass-XXXX=True'
- Block - throws an exception that cannot be bypassed
I also call it from a few locations that are expected to be deprecated already.
Reviewed By: markbt
Differential Revision: D21890496
fbshipit-source-id: faddb301888ef75cc71d46ffb7374f3fe3d044bd
Summary:
`chg` is now integrated as `#chg-compatible` by the test framework. The helper
script no longer works.
Reviewed By: markbt
Differential Revision: D21895009
fbshipit-source-id: 257ed081fbd1a6acc97afbfa2934134e389ab400
Summary:
This requires `--hidden` for `hg tip` to show the pending commits in hooks.
The real fix would be handling pending visibility changes in metalog layer.
Reviewed By: markbt
Differential Revision: D21894321
fbshipit-source-id: 3b2296c9c38b37aee2c57eea0ad03c3acaab7507
Summary:
The test depends on lots of implementation details of obsmakers (ex. prune
markers, etc). In production we use mutation templates tested by
test-mutation.t. Therefore remove the test.
Reviewed By: markbt
Differential Revision: D21894319
fbshipit-source-id: 549423b1f5f45cf567b3c95e5fe3ccfa30b6695a
Summary:
The mount path is required in 2 places: at mount time, and when deleting files.
It turns out that the second one doesn't require it as PrjDeleteFile specify
the filename to be: "A null-terminated Unicode string specifying the path,
relative to the virtualization root, to the file or directory to be deleted."
Thus, we only need it during mount time, so let's pass it to the start method.
Reviewed By: fanzeyi
Differential Revision: D21797080
fbshipit-source-id: 27f5d18841ac077c81ad3f6b3686e20917505c41
Summary:
This will simplify some later diffs where the channel construction doesn't
change any state.
Reviewed By: fanzeyi
Differential Revision: D21797077
fbshipit-source-id: 67d9ee693b067a08986e9bd33f9d91f9a1dadac1
Summary: This is already created, no need for re-recreating it.
Reviewed By: fanzeyi
Differential Revision: D21676314
fbshipit-source-id: 8e3eecfb6069f9bed5bfec7ad6631ed8830d796d
Summary:
These are pure implementation details of ProjectedFs, and don't need to be in
the class, let's move them onto an anonymous namespace in the cpp file.
Reviewed By: chadaustin
Differential Revision: D21676315
fbshipit-source-id: 4fa7088a10256581d7eed2c7cbd90c30a7fef3d2
Summary:
I find this approach to be slighly less error prone due to not having
to manually specify the array size in advance. We lose the field names,
but they should be pretty self explanatory.
Reviewed By: chadaustin
Differential Revision: D21676319
fbshipit-source-id: db4dc993d3dd2a502ae188b08b8d4230939c0415
Summary:
This moves the edenToWinPath inside Windows specific files and makes
the interface clear about the kind of path that is expected.
Reviewed By: fanzeyi
Differential Revision: D21676318
fbshipit-source-id: d153bb543da5f481e94f83d52e45c6cc871d1f54
Summary:
The only reason tests need to set an FsChannel is for calling
cleanupPrjfsCache. For consistency sake, let's do the same as what we do on
unices: verify if we have a valid channel first.
Reviewed By: wez
Differential Revision: D21676316
fbshipit-source-id: 022ca91d32a7ab047a79acccd7d17dfd7906d527
Summary:
The only missing piece on Windows is the bind mount/redirection due to
folly::Subprocess not being present on Windows.
Reviewed By: wez
Differential Revision: D21676313
fbshipit-source-id: a5ba09be04c94b66edf9d40884753afa3865def2
Summary:
We want to use these on Windows too, so let's rename them to not be FUSE
specific.
Reviewed By: wez
Differential Revision: D21676317
fbshipit-source-id: 9f1bde9e0467d8828d4168a69581c8fea54669cd
Summary:
This moves it out of EdenServer and into EdenMount directly. This is closer
to what is happening in the Linux code and thus better.
Reviewed By: fanzeyi
Differential Revision: D21663501
fbshipit-source-id: 84420fcb37586cc2695568f0167b39f91c9bfb63
Summary:
Instead of having 2 different functions for Windows and other unices, let's
only have one. This allows more of the mount code to be shared between both.
Reviewed By: wez
Differential Revision: D21517477
fbshipit-source-id: 185ce864aecdf9a7880a9c705df444ec1e39319a
Summary:
Add a permission checker that verifies that the user is permitted to make
requests on behalf of a service. This uses a different ACL than the usual repo
ACL.
Reviewed By: mitrandir77
Differential Revision: D21886531
fbshipit-source-id: bceb63aed12c11ebe2a3d65a2560c014267b0e20
Summary:
Regexes stored in config need to be comparable so that config is comparable.
Normally regexes are not comparable, so wrap them in a newtype wrapper that
implements comparison, rather than manually implemented PartialEq and Eq for
anything that contains them.
Reviewed By: mitrandir77
Differential Revision: D21886533
fbshipit-source-id: 1fb39c0874daed383624eeda61c903a4731b9cb8
Summary:
The repository write methods are likely to get fairly large. Move
`create_changeset` to its own module so that we can add future methods as
separate modules.
Reviewed By: mitrandir77
Differential Revision: D21886532
fbshipit-source-id: 08de34a400d056bc97cf6e54856f2a2fd185814c
Summary:
batch_derive() is a dangerous function to use. I'd love to delete it but this
function is very useful for backfilling, so unfortunately I can't.
The problem arises when one tries to backfill blame and unodes simultaneously
(or just derive blame which in turn derives unodes). While batch_derive()
tries to be careful with inserting "outer" derived data's mappings (i.e. blame
mapping), it doesn't do it for inner derived data mappings (i.e. unodes). So we
might end up in the situation where we insert unodes mapping before we inserted
all the manifests for it. If this thing fails in the middle of derivation then
we have a corruption.
Let's do not use it in blobimport. It will make derivation slower, but I'd
rather make it slower than incorrect.
Reviewed By: farnz
Differential Revision: D21905619
fbshipit-source-id: c0227df195a8cf4482b2452ca928acbc5750b3e5
Summary: This will be very useful when we migrate to Mononoke.
Reviewed By: StanislavGlebik
Differential Revision: D21912115
fbshipit-source-id: 1ed2204f17f90152a67fd4e10b0ae61aacfc41d7
Summary:
There are people that are hurt by usage of these terms, this should be more
then enough reason to replace these. Newly chosen terms are more
self-explanatory as well.
This doesn't yet touch the actualy config files, as that requires a bit more
effort than 1 diff and will require more coordination.
Reviewed By: krallin
Differential Revision: D21924440
fbshipit-source-id: e24fc638dc8c9d6d20b6f3fa5f0d0bbc91bbf77b
Summary:
This test checks that we can start Mononoke and serve pull/push/update with
filenodes
Reviewed By: ahornby
Differential Revision: D21904753
fbshipit-source-id: 86690c5ed5ce7d022844809b09beb25c7961cac8
Summary: Let's log to ods so that we can check what's hapenning with filenodes.
Reviewed By: ahornby
Differential Revision: D21904400
fbshipit-source-id: e602dfc338c02252cad286176a1965bdc7043d7f
Summary:
This diff migrates add_filenodes method to return FilenodeResult.
That means that all filenodes methods now return FilenodeResult and it's time
now to remove TODOs from derived_data filenodes.
Note that I had to change the test "derive_disabled_filenodes" a bit.
Previously FilenodesOnlyPublic::mapping::get() method immediately returned
FilenodesOnlyPublic::Disabled, while now it returns None if hg changeset is not
derived. This is an expected change in behaviour, so I just updated the test to
try to derive FilenodesOnlyPublic first, which in turns triggers generation of hg changeset.
Reviewed By: ahornby
Differential Revision: D21904401
fbshipit-source-id: f6f4cd14e6cdce5a4b95d8f3f9acff305ae6fa88
Summary:
Similar to get_all_filenodes_maybe_stale() make this method return
FilenodeResult if filenodes are disabled.
Note: this diff adds one TODO in fetch_root_filenode, which will be removed
together with other TODOs in the next diff.
Reviewed By: ahornby
Differential Revision: D21904399
fbshipit-source-id: 1569579699c02eb07021f8143aa652aa192d23bc
Summary:
With the internal streampager, progress bars must be sent on a separate stream so that
streampager can render them correctly.
Reviewed By: quark-zju
Differential Revision: D21906173
fbshipit-source-id: eb41b0bf22807d9cae518b3f676996ab1c642c6e
Summary: Upgrade the internal streampager to the latest version, which includes line wrapping.
Reviewed By: quark-zju
Differential Revision: D21906172
fbshipit-source-id: 1bc63723f55ac115090e7ae0a2541158863056b9
Summary: The test is changed because an extra file is copied during local-repo clone.
Reviewed By: markbt
Differential Revision: D21894318
fbshipit-source-id: 4417fb5473dc2cb98eda4dcc4d0484cc3b31fae2
Summary:
`index2` is the Rust revlog index implementation that has extra requirements,
namely, the revlog cannot be inlined. We have code to make sure changelog is
not inlined, but manifest or filelog can still be inlined.
`index2` is only needed for changelog for phase and visibility calculations.
Therefore, disable it for manifest and filelog.
Reviewed By: singhsrb
Differential Revision: D21917447
fbshipit-source-id: c45f955175c623092e6f728042dbfd3b942fe911
Summary: This diff updated `eden du` to display a summary and display warnings and cleaning information with colors.
Reviewed By: kmancini
Differential Revision: D21885051
fbshipit-source-id: be127b81c92bea1051a80715682cdbccf22f22e3
Summary: This diff updated `eden du` to display an aggregated result of all mounts instead of showing all details for each mount, as users generally just want to reduce disk usage but don't really care about details.
Reviewed By: genevievehelsel
Differential Revision: D21877178
fbshipit-source-id: dde43e51e96a5c2569c9fe21ab06cc7ea4295866
Summary:
NOTE: This stack works together, they are separated for easier reviewing. Check D21723465 if you want to read this stack in one place.
This diff finally connects EdenAPI with EdenFS. Previously we only had basic EdenAPI implemented but the performance was not acceptable. This is largely due to overheads in talking with the remote server. Now EdenFS is able to import multiple files at once, we efficiently use EdenAPI to import these data now.
Reviewed By: chadaustin
Differential Revision: D21576569
fbshipit-source-id: a45e832ec63d057730138551393ff7547fa2c22f
Summary:
NOTE: This stack works together, they are separated for easier reviewing. Check D21723465 if you want to read this stack in one place.
This diff encapsulates the newly added batch blob import API in backingstore crate, and expose it to EdenFS. It converts incoming complex data structures down to primitive data types that can go through FFI boundary.
Reviewed By: xavierd
Differential Revision: D21697578
fbshipit-source-id: 8f5505ef4cab342dfa710798a04df4e0e94055d9
Summary:
NOTE: This stack works together, they are separated for easier reviewing. Check D21723465 if you want to read this stack in one place.
This diff implements getBlobBatch in `backingstore` crate that is able to process multiple blob import request at once. This will help EdenFS to process blob import request efficiently. See following diffs in the stack for usage.
To import a list of import requests, the function first check if the file exists locally. When it is already in local store, the function will call the provided `resolve` function to send the blob back. Then it proceeds to send **ONE** request to remote store (in our case, EdenAPI) for all these missing files if this import permits remote importing.
We use the callback style `resolve` parameter because we want to awake waiting threads as soon as the blob is ready, so we don't waste time on unnecessary waiting.
Reviewed By: xavierd
Differential Revision: D21697580
fbshipit-source-id: b550accf6f6163cf6f2e9be6b628e46f44076c37
Summary:
It was turned on in production (facebook.rc) except for hg servers.
This mainly affects tests.
Reviewed By: markbt
Differential Revision: D21894313
fbshipit-source-id: 5f12620cfc11bb243e96fba12a07cbf0241c738b
Summary:
This is a followup from s195788 - this little check should be enough to prevent
us from having the paths in our DB being truncated on insert.
Reviewed By: markbt
Differential Revision: D21904281
fbshipit-source-id: c9306e0cca4a2e009bb50bc287190d8f7ad12be4
Summary:
I am planning to start migrating Eden's CLI to the new Python 3 Thrift
implementation. In preparation, slightly clean up the interface and
implementation of our Python 2 Thrift wrapper.
Reviewed By: genevievehelsel
Differential Revision: D21854539
fbshipit-source-id: d398dd3f324c12288871cf0c9db41e64ed4cf7ed
Summary:
Visibleheads (and even narrow-heads) are enabled by default except for hg
servers. No need to log them specially.
Reviewed By: markbt
Differential Revision: D21894308
fbshipit-source-id: 7ffe0977e51c17e743b62da0e2bfbe7618c64610
Summary: This is required to make some tests pass.
Reviewed By: markbt
Differential Revision: D21894312
fbshipit-source-id: 481bcc42f8f7fbb5c1707e66da31aee29932484c
Summary: Make `debugobsolete` also update visibility. This affects many tests.
Reviewed By: markbt
Differential Revision: D21894310
fbshipit-source-id: eba6ecb07fdd8f5a13085434b303581ec5acf463
Summary: This makes tests more stable when changed.
Reviewed By: markbt
Differential Revision: D21894314
fbshipit-source-id: 73c36f79763643ccbe5dd52522fa8f22fa17a08a
Summary: We don't use obsmarker but mutation records. Therefore, remove the test.
Reviewed By: markbt
Differential Revision: D21894315
fbshipit-source-id: a1ccd8f78bea8adcba419b0d8ec99d52cd14d4e4
Summary:
We don't use obsmarker exchange but mutation record exchange.
Therefore, remove the test.
Reviewed By: markbt
Differential Revision: D21894311
fbshipit-source-id: 73d1f8031b2f601d133a98ffaa92a890bb59a656
Summary: Let's log the name as well - it will help with investigation.
Reviewed By: farnz
Differential Revision: D21906595
fbshipit-source-id: 51eb49354017c17ba3304f0a66c95dfc3c695e6a
Summary: The replacement will make lfs_server, mononoke_api, edenapi_server and walker OSS buildable.
Reviewed By: krallin
Differential Revision: D21884324
fbshipit-source-id: e6cdf8356b13056a9944ab9da18dc15977b6a2ec
Summary:
This adds changes, introduced by me in D21841293.
For my stuff: I will only land this once the configerator diff is reviewed.
Reviewed By: farnz
Differential Revision: D21862276
fbshipit-source-id: d4f5ccdd4ef9d62bb1e394d4db36930413633fa6
Summary:
The CreateProcess API allows inheritable handles to be available in the spawned
process by passing a flag to it. What the documentation for this API leave to
the imagination is what happens when processes are spawned concurrently in
multiple threads and handles are opened and made inheritable while doing this.
The answer is obvious but the consequences aren't: they are inherited.
When anonymous Pipes are used, one end needs to be inheritable for the spawned
process to being able use it, but if one is created concurrently with spawning
a process, that other process may have an open handle to that unrelated pipe.
And since to detect that a pipe is closed, all handles to the other end needs
to be closed, this can lead to never being able to detect that it is closed...
This scenario is exactly what is happening in eden when spawning the Mercurial
process, and when one of these processes would die, eden would just hang trying
to write to the pipe, not knowing that the process was already gone. To unblock
eden, all the hg.real processes had to be killed, as this would close all the
pipe handles, and then eden would detect that the pipe was closed and re-spawn
Mercurial (only for the exact same thing to happen).
In order to fix this, we need to tell CreateProcess to only inherit the pipes
handles, and nothing else.
Reviewed By: wez
Differential Revision: D21895537
fbshipit-source-id: 3c84a1d0316b50b5750f554fa20f72f59a718882
Summary:
The old `py` Thrift language support doesn't correctly handle string
vs. bytes, which causes an exception to be thrown when deserializing
paths or blobs that aren't UTF-8.
We will eventually want to migrate to the py3 language implementation,
which supports streaming.
Reviewed By: genevievehelsel
Differential Revision: D21693082
fbshipit-source-id: 0ea10fd3960f5acba353bccb83b5cf539e7eeffb
Summary:
It looks like a refactoring broke the EDEN_TEST_NO_CLEANUP feature
which was useful when trying to debug the eden state directory after a
failure.
While fixing it, I went ahead and added support for only saving the
state directory upon test failure.
Reviewed By: simpkins
Differential Revision: D21780122
fbshipit-source-id: 0cd6e9f274601eebd9b4a6978c0cf61fb1b85545
Summary:
Let's return FilenodeResult from get_all_filenodes_maybe_stale and change
callers to deal with that.
The change is straightforward with the exception of `file_history.rs`.
get_all_filenodes_maybe_stale() is used here to prefetch a lot filenodes in one
go. This diff changes it to return an empty vec in case filenodes are disabled.
Unfortunately this is not a great solution - since prefetched files are empty
get_file_history_using_prefetched() falls back to fetching filenodes
sequentially from the blobstore. that might be too slow, and the next diffs in
the stack will address this problem.
Reviewed By: krallin
Differential Revision: D21881082
fbshipit-source-id: a86dfd48a92182381ab56994f6b0f4b14651ea14
Summary:
In order to start EdenFS automatically at boot, a template service was used
previously, but due to several issues, we decided to move away from it.
Thankfully microsoft supports several other ways of starting tasks at startup,
one of which is the "Task Scheduler" itself.
One of the weird part of the task scheduler is that there isn't a good way
to tell it to not show a console for a non-graphical application, and thus
plainly executing edenfsctl start in it would create a cmd window, which
would then disappear a couple of seconds later. To avoid this, a "graphical"
version of Python is used (pythonw.exe) to start edenfsctl.
Reviewed By: fanzeyi
Differential Revision: D21732281
fbshipit-source-id: 87ef3a2d5569302392bd30a4b9e7fc48807ee316
Summary:
There's an issue where hg foo could spawn hg dynamicconfig which could
spawn hg version (via the telemetry wrapper) which could spawn hg dynamicconfig.
If the config generation fails then this could enter an infiniteloop, since the
mtime of the file would not be updated appropriately, so we will always think we
need to regenerate.
Let's prevent this by setting an environment variable when spawning the
background process.
Reviewed By: quark-zju
Differential Revision: D21885363
fbshipit-source-id: ba4938c926d1219985813a521fec412097df4b4a
Summary:
I observed that for whatever reason our setting of `use_try_shorthand = true` in rustfmt.toml was causing entire functions to not get processed by rustfmt. Even files that contain neither `try` nor `?`. Remove it and reformat fbsource.
Documentation of that config:
- https://github.com/rust-lang/rustfmt/blob/master/Configurations.md#use_try_shorthand
We don't particularly care about the value anymore because nobody writes `r#try!(...)` in 2018 edition code.
Minimized:
```
fn f() {
g(
)
// ...
.h
}
```
This function gets formatted only if use_try_shorthand is not set.
The bug is fixed in the rustfmt 2.0 release candidate.
Reviewed By: jsgf
Differential Revision: D21878162
fbshipit-source-id: b028673c0eb703984d24bf0d2983453fc2a8c212
Summary:
When I was debugging the multithreaded bug issue it came up that the issue would
only affect certain versions of macos. There may be other bugs that come up
affecting only certain os versions. Having the os version in eden rage could be
helpful to identify such issues.
Reviewed By: chadaustin
Differential Revision: D21776154
fbshipit-source-id: a493e7da1823075ca4a845bd73b21716ce884911
Summary: This will allow me to test it easily.
Reviewed By: StanislavGlebik
Differential Revision: D21840079
fbshipit-source-id: 1b3743da9c7908eb0dedd665aa24a4bf7aabd94f
Summary: We need to thread it through this layer to get it to SCS
Reviewed By: StanislavGlebik
Differential Revision: D21840078
fbshipit-source-id: d3c16a4afd87aa844628dd0aab7f85b8d38dadf8
Summary:
We need a lowest common ancestor operation for `merge_base` call
implementation.
The `lca` function returns only one of many possible ancestors in some
situation (multiple merges of the same branches). This is done to:
simplify the implementation and hide the complexity from our API users
(people don't really expect to receive more than one node from such APIs).
Mercurial arbitrarily chooses the commit with smallest hash in those situations.
Reviewed By: StanislavGlebik
Differential Revision: D21840081
fbshipit-source-id: 2dfc95a4cf549d8941fc5166e878bfee4b6b2ece
Summary: This methods will replace the fbcode_build provided functions. Also add Cargo files for crates that are now thanks to that buildable in OSS.
Reviewed By: farnz
Differential Revision: D21549860
fbshipit-source-id: 69f4179aa7a9081d33fac1f2f88aa3b32cd31e17/
Summary: I will modify in the next diffs, it's nice to have some test coverage
Reviewed By: krallin
Differential Revision: D21880996
fbshipit-source-id: 266622982e5d7d6d19ab3ac921ddc51079e51457
Summary:
The max_depth parameter was confusing - one might have thought that it returns
all entries that are `max_depth` hops away from `startnode`. This is not the
case - in reality it just limits what's the total number of entries can be
returned.
In fact, I think it can be replaced with Stream::take() method, however that's
a larger refactoring, and I'm not going to do it now.
It doesn't matter much because usually we have linear histories, but I think
it's still worth to make it clearer.
Reviewed By: krallin
Differential Revision: D21880220
fbshipit-source-id: d209e1e129383f9181ae11b489992dc4c3ce2d5c
Summary:
In the hg sync job, we need to load up the ancestors for all bookmarks known to
the server we are pushhing to, and for e.g. fbsource that might be > 10K
bookmarks. If we fetch those 1 by 1 (because e.g. cold cache), that will take a
very long time.
Unfortunately, we don't currently have a way of buffering access to changesets,
so for now let's mitigate by buffering.
Reviewed By: ikostia, HarveyHunt
Differential Revision: D21860228
fbshipit-source-id: 90977a9e00689c1df5ae53d149c267de9b2f973e
Summary:
When you call poll() from within an async context (as opposed to calling
await), awkward things happen, like this:
```
thread 'validation::test::slow_ready_validates' panicked at 'no Task is currently running',
```
So, let's stop doing that by instead checking that our stream eventually
yields. We can't count the polls this way, but that feels a bit immaterial so
that's probably OK. Also, let's clean up the code a little bit by removing a
bunch of conditional compilation.
Reviewed By: ahornby
Differential Revision: D21862830
fbshipit-source-id: 3eb49575d940ca85f59c49295dd6b0dcfb2e5d15
Summary:
We are going to start using tunables in Mononoke in the next diffs, and the
name clash between "tunables" and "newfilenodes::tunables" makes it confusing.
Let's rename newfilenodes::tunables to sql_timeout_knobs
Reviewed By: krallin
Differential Revision: D21879093
fbshipit-source-id: ab0bae4be3c319dcb6afeecdd1c13df395e79e3b
Summary: The thrift team is looking to retire `stopWorkers()` and `stopListening()`, so this refactors our code to stop using both of those methods.
Differential Revision: D21851442
fbshipit-source-id: 6d63d435cbd4a51c855bf71a3b2165e35431a4d7
Summary: we need an option to rebuild skiplists from scratch in case of corruptions.
Reviewed By: farnz
Differential Revision: D21863639
fbshipit-source-id: 56d8360a6c2c38aeb35f534758f5cde410fef421
Summary: This makes the API more flexible.
Reviewed By: sfilipco
Differential Revision: D21626196
fbshipit-source-id: d0d28ba075ee3321a1a757f848fb72592827d75d
Summary: This allows DAG algorithms to be executed on revlog.
Reviewed By: sfilipco
Differential Revision: D21626217
fbshipit-source-id: e9fd61ad62f95be7b055a0ef8879c59cbeeb60b9
Summary:
Mostly copy-paste from code added in D19503373 and D19511574. Adjusted to match
the revlog index interface.
Reviewed By: sfilipco
Differential Revision: D21626201
fbshipit-source-id: 05d160e4c03d7e2482b6a4f2d68c3688ad78f568
Summary: The trait converts NameSet to IdSet. It'll be used by the revlog index.
Reviewed By: sfilipco
Differential Revision: D21795869
fbshipit-source-id: 55f7a238158442db9d8bdfe84e64438be504f618
Summary: Add a way to inverse the DAG (swap parent / children relations).
Reviewed By: sfilipco
Differential Revision: D21795870
fbshipit-source-id: 2d076f4ae491141aa758faa5f5f303c97f7e56dc
Summary:
Similar to LazySet, but the iterator is using Ids. This will be useful for
lazy calculations that are cheaper with Ids.
Reviewed By: sfilipco
Differential Revision: D21626208
fbshipit-source-id: 9a34fbf18f0039caeb4f6e698294c4d335354093
Summary:
The NameSet is not really about Dag. It is about using Id and is static.
Rename it to clarify. In an upcoming change we'll have IdLazySet.
Reviewed By: sfilipco
Differential Revision: D21626204
fbshipit-source-id: 84f25008f7032f6e26a26fc656ccbcd2a5880ecf
Summary:
This makes it possible to use NameIter without manually specifying out iterator
types, which might be quite long.
Reviewed By: sfilipco
Differential Revision: D21626202
fbshipit-source-id: 67b338765c09629645794cf73a9b496271524f9d
Summary: Take advantage of Hints and add fast paths.
Reviewed By: sfilipco
Differential Revision: D21626216
fbshipit-source-id: 6d43666bd6cdec7ff4b93032c1064cafd8de85cf
Summary: Update hints if they are easy to obtain or calculate.
Reviewed By: sfilipco
Differential Revision: D21626206
fbshipit-source-id: 453b7db2444406ce51d574c688fe536316fb9b0f
Summary: The `secure_utils` crate from common/rust/secure_utils was moved to rust-shed, the remaining crates in that folder are being refactored here into a single crate `identity_ext` for clarity.
Reviewed By: StanislavGlebik
Differential Revision: D21549861
fbshipit-source-id: 4da6566a09ba7a772e8062632f9d7520af2e09e6
Summary:
In some rare cases, we would have hgcache that contains broader permission than we are expecting. We shouldn't be fixing it if that's the case.
We also might be in situations where hgcache directory isn't entirely created by Mercurial, and the owner of the directory will be different than the process. This will cause the `chmod` call to fail with permission error. In that case, this will cause EdenFS to panic. This is undesirable. We should be handling this case more gracefully and let the original error populate.
Reviewed By: xavierd
Differential Revision: D21854542
fbshipit-source-id: e9d11399aeb40b375725b49f4bcd54050afdcbad
Summary:
In the next diffs we'll make it possible to disable filenodes in Mononoke. See
D21787848 and attached task for more details, but TL;DR is that if xdb is down
we still want to serve "hg update" traffic.
If filenodes are disabled we obviously can't generate filenodes for new
commits. So one option would be to just return an error from
FilenodesOnlyPublic::derive(...) call. But that would mean that any attempt to
call derivation would fail, and e.g. Mononoke servers won't be able to start up
- (see https://fburl.com/diffusion/roau028d). We could change callers to always
process errors from FilenodesOnlyPublic, but I think it would be harder to
enforce and easier to forget.
So this diff changes FilenodesOnlyPublic to be an enum, and
FilenodesOnlyPublic::Disabled is returned immediately if filenodes are
disabled. For callers it means that they can't rely on filenodes being present
in db even after FilenodesOnlyPublic were derived. That's the whole of the
stack, and the next diffs will update the callers to properly deal with missing
filenodes.
One caveat is that when we re-enable filenodes back we might need to derive
them for a lot of commits.
I don't expect it to happen often (i.e. if xdb is down then we probably can't
commit anyway), but if somehow it happened, then we should be a bit more
careful with re-enabling them after the problem was fixed. For example, we can
first derive all the filenodes locally by e.g. running backfill_derived_data,
and only after that has finished successfully we can re-enable them.
Reviewed By: krallin
Differential Revision: D21840328
fbshipit-source-id: ce9594d4a21110a5cb392c3049ccaede064c1e66
Summary: This diff made `eden du` able to run under any directory and gives statistics for every EdenFS mount on disk by default.
Reviewed By: fanzeyi
Differential Revision: D21846894
fbshipit-source-id: 2d421db8a4a0202419aa7a41e620d92f6ebdea2e
Summary:
Instead of always building from scratch, continue assiging Vertexes and
Segments from the last commit that was processed.
Reviewed By: StanislavGlebik
Differential Revision: D21634699
fbshipit-source-id: 9f8b890dcf65c59a66651343f0ccc1487efc2394
Summary:
Somehow, on Windows, the socket file that was removed during `prepare` may be
back, but not bound, preventing Thrift from binding to it, let's remove it
again since it's not supposed to be there.
Note that this happens 100% of the time when starting EdenFS via the Task
Scheduler at log on, I couldn't reproduce when triggering the task manually,
or when starting edenfs by hand.
Reviewed By: simpkins
Differential Revision: D21755498
fbshipit-source-id: 32a343d9de531b90417dac0aa382a07c85cdee5a
Summary:
The CREATE_NO_WINDOW is documented as:
The process is a console application that is being run without a console
window. Therefore, the console handle for the application is not set.
The last part is what matters here, by default stdin, stdout and stderr are
inherited from the parent process, with this flag, they won't as these 3 will
simply not be set. Removing it allows Mercurial's output to be sent directly to
the edenfs log which will greatly improve our debuggability.
Reviewed By: akrieger
Differential Revision: D21820195
fbshipit-source-id: 63496a1ad28ecf5440c0846d80e1fbda5756b971
Summary:
Previously, the NameSet has properties like "is_all", "is_topo_sorted", etc.
To make lazy sets efficient, it's important to have hints about min / max Ids
and maybe some other information.
Add a dedicated Hints structure for that.
Reviewed By: sfilipco
Differential Revision: D21626219
fbshipit-source-id: 845e88d3333f0f48f60f2739adae3dccc4a2dfc4
Summary:
Implement a small subset of DagAlgorithm by default. This makes
other implementations of DagAlgorithm slightly easier.
Reviewed By: sfilipco
Differential Revision: D21626199
fbshipit-source-id: ac6dfb5c22bf1da44f521fc9e76d59bfb95063c7
Summary:
D21479023 broke it. It should convert to Id, and check Id against the SpanSet,
instead of just checking the IdMap ignoring the SpanSet.
Reviewed By: sfilipco
Differential Revision: D21626193
fbshipit-source-id: 6daf86f292a7acfd3688893a55e2a794cfe068fe
Summary: This makes the next change easier to implement.
Reviewed By: sfilipco
Differential Revision: D21626198
fbshipit-source-id: 57ab69cba7f43350767e5d0d52ebfe66764895ca
Summary:
Implements part of the dag IdMap related traits.
It does not get used yet, but eventually I'd like `pydag` to be able to work
with an abstracted dag including RevlogIndex.
Reviewed By: sfilipco
Differential Revision: D21626210
fbshipit-source-id: 53f19622f03fd71b76073dccf8dcc9b4778b40ca
Summary:
This will allow RevLogIndex to answer node -> rev and hex lookup queries.
Also change RevlogIndex::new to take file names so it can write back the
nodemap index when the index is lagging. That part of logic currently exists in
pyindexes + clindex.pyx, which are going to be replaced by revlogindex.
Practically, this will generate a `00changelog.nodemap` file in svfs, which is
temporarily unused, but will be used once clindex.pyx gets replaced.
Reviewed By: sfilipco
Differential Revision: D21626209
fbshipit-source-id: 297d9eff26a73c26558708f7a2290d4d8ba1e777
Summary: Previously, `read_res` was called `data_util` and only dealt with EdenAPI data responses. Support for history responses was added later as a `history` subcommand. For consistency, let's move the top-level commands for data responses underneath a new `data` subcommand. When support for addition response types is added in the future, those can also go under their own subcommands.
Reviewed By: quark-zju
Differential Revision: D21825197
fbshipit-source-id: f5cb759a68324e7d0f98e3448bd5d1cba6417bad
Summary: Give this tool a more descriptive name. (It reads EdenAPI responses, so `read_res` seemed fitting.)
Reviewed By: quark-zju
Differential Revision: D21796964
fbshipit-source-id: 8a4ee365aa3bcf115fc7a3452406ed96b4a25edc
Summary: In line with other crates that contain utility binaries alongside the crate, rename the `edenapi/utils` directory to `edenapi/tools`.
Reviewed By: quark-zju
Differential Revision: D21796899
fbshipit-source-id: 058319e2756b1d596f06d6e57d17a6c07a7f1c9c
Summary:
Clean up some of the conversion functions by renaming variables that are
keywords in other languages, and simplifying error handling code.
Differential Revision: D21839019
fbshipit-source-id: d8945a14a230caa744040e134203a908ad9cef20
Summary: `ErrorKind` is not meaningful, and is an artifact of older-style error handling crates. A better name is `ConfigurationError`.
Reviewed By: krallin
Differential Revision: D21837271
fbshipit-source-id: 709d9e2ab7f18dd2f7cb2489f24e91612bc378db
Summary:
Replace the use of `RepoConfigs::read*` associated functions with free
functions. These didn't really need to be associated functions (and in the
case of the common and storage configs, really didn't belong there either).
Reviewed By: krallin
Differential Revision: D21837270
fbshipit-source-id: 2dc73a880ed66e11ea484b88b749582ebdf8a73f
Summary:
Refactor parsing of repo config using a new `Convert` trait to allow
definition of each part of parsing separately.
The wireproto logging args require access to the storage definitions, so need
to be parsed by their own special function for now.
Differential Revision: D21837269
fbshipit-source-id: 7ab0e3f4b3b8549aaefb45201388c3dfc7633ef7
Summary:
Refactor parsing of storage config using a new `Convert` trait to allow
definition of each part of parsing separately.
Differential Revision: D21766761
fbshipit-source-id: 7e224e9d322a3a16a64f5ebba2243bbe6341c8f0
