Summary: Prep for 1.44 but also general cleanups.
Reviewed By: dtolnay
Differential Revision: D22024428
fbshipit-source-id: 8e1d39a1e78289129b38554674d3dbf80681f4c3
Summary:
Python 3.8 provides the (poorly named)
unittest.IsolatedAsyncioTestCase base class that gives each test an
asyncio event loop and runs it. This allows us to write async
integration tests, required to use modern Thrift Python.
Reviewed By: genevievehelsel
Differential Revision: D21955584
fbshipit-source-id: 8366dc7c72b31e1667e72fe1e359587b74e2c614
Summary:
In text mode, Python will use '\r\n' as line endings instead of '\n' on
Windows. This has the negative side effect of making some tests fails due to
the file content not being exactly what is being written.
Reviewed By: fanzeyi
Differential Revision: D21944456
fbshipit-source-id: f7f271e2ef9e2f21bd78c76233e644ce411b63df
Summary: Replaces usages of whitelist/blacklist with include/exclude/allow. These terms are more descriptive and less likely to contribute to racial stereotyping. More context: https://fb.workplace.com/groups/sourcecontrolteam/permalink/2926049127516414/
Reviewed By: farnz
Differential Revision: D22039299
fbshipit-source-id: d5b4cdeca681e1f4c992f7ef0b9f772b2ae1abd3
Summary:
Add a `--record` flag, which uses `git` to track changes in `$TESTTMP`.
I use it to track changes with C revlog vs the Rust revlog.
Reviewed By: farnz
Differential Revision: D22007501
fbshipit-source-id: a8027aa4b2b3b97c3cf5f642434451826fcbded9
Summary:
Interactive smartlog history attempts to access `sltimestamp` as a field of
`slinfo`. The field is actually called `timestamp`.
Reviewed By: quark-zju
Differential Revision: D22042316
fbshipit-source-id: 7c1d53dfc1c8d661a248d8412f7d780c83723721
Summary:
Rename the `subtree` endpoint on the EdenAPI server to `complete_trees` to better express what it does (namely, fetching complete trees, in contrast to the lighter weight `/trees` endpoint that serves individual tree nodes). This endpoint is not used by anything yet, so there isn't much risk in renaming it at this stage.
In addition to renaming the endpoint, the relevant request struct has been renamed to `CompleteTreeRequest` to better evoke its purpose, and the relevant client and test code has been updated accordingly. Notably, now that the API server is gone, we can remove the usage of this type from Mononoke's `hgproto` crate, thereby cleaning up our dependency graph a bit.
Reviewed By: krallin
Differential Revision: D22033356
fbshipit-source-id: 87bf6afbeb5e0054896a39577bf701f67a3edfec
Summary: Replace usages of whitelist/blacklist with include/exclude/filter/allow. These terms are more descriptive and less likely to contribute to racial stereotyping. More context: https://fb.workplace.com/groups/sourcecontrolteam/permalink/2926049127516414/
Reviewed By: kulshrax
Differential Revision: D22039298
fbshipit-source-id: 255c7389ee5ce5e54bbccdfb05ffa4cafc6958e5
Summary: Replaces usages of whitelist/blacklist with include/exclude/allowed. These terms are more descriptive and less likely to contribute to racial stereotyping. More context: https://fb.workplace.com/groups/sourcecontrolteam/permalink/2926049127516414/
Reviewed By: kulshrax
Differential Revision: D22039297
fbshipit-source-id: ab6e55889e6cf42aed35c856475b2ff92f4cb802
Summary: Enable the most common autocomplete use cases in zsh hg histedit autocomplete
Reviewed By: fanzeyi
Differential Revision: D22032612
fbshipit-source-id: 1fd47d30564c51a010785d7d0553e3a5d16f163a
Summary:
This diff updated the `eden du` command. Instead of printing
```Reclaim space from the LFS cache directory by running:
hg -R /data/users/akushner/eden-repos/opsfiles gc
Reclaim space from the LFS cache directory by running:
hg -R /data/users/akushner/eden-repos/megarepo_test gc
Reclaim space from the LFS cache directory by running:
hg -R /data/users/akushner/eden-repos/instagram-server_test gc
```
We now print "Reclaim.." only once and put available commands together
```
LFS cache detected in backing repo. To reclaim space from the LFS cache directory, run:
hg -R /data/users/akushner/eden-repos/opsfiles gc
hg -R /data/users/akushner/eden-repos/megarepo_test gc
hg -R /data/users/akushner/eden-repos/instagram-server_test gc
```
and did the same for working copy under backing repositories.
Reviewed By: fanzeyi
Differential Revision: D22038555
fbshipit-source-id: c19869db6f91e90a0627b3bb4b4f7e95142ae198
Summary: Replaces usages of whitelist/blacklist with includelist/excludelist/incompatiblelist/. These terms are more descriptive and less likely to contribute to racial stereotyping. More context: https://fb.workplace.com/groups/sourcecontrolteam/permalink/2926049127516414/
Reviewed By: kulshrax
Differential Revision: D22039296
fbshipit-source-id: d4c1fd09e40d5ebeaab99515ed19aa5df6c2ecc4
Summary: This diff fixed small problems in previous `eden du` summary printing and added a reminder of option `--clean` to users.
Reviewed By: fanzeyi
Differential Revision: D22038552
fbshipit-source-id: 7dbac858b74833e59d8d3f1c6fb1f5d8944935d0
Summary:
Those tests show up as "skipped". I removed some rarely used ones:
- test-check-jshint: we seldom write javascript.
- test-check-shbang: a slow test that does not do much.
- test-chg: chg is being tested in other tests by default.
- test-docker-packaging: probalby already broken, we don't use docker.
- test-verify-repo-operations: a slow test that is too expensive to run with
limited value.
Reviewed By: markbt
Differential Revision: D21989103
fbshipit-source-id: 8080ad68dc75b59f87336b94af38c8231f4f10f4
Summary:
After being trimmed down to store only a single edge the skiplist update
algoritm was working incorrectly. This diff introduces a way of fixing things
up.
Reviewed By: StanislavGlebik
Differential Revision: D21976995
fbshipit-source-id: 3bc4678011b332419d9f6f7b983a33fa066674bf
Summary: Make most of the handlers on the EdenAPI server return streaming responses.
Reviewed By: krallin
Differential Revision: D22014652
fbshipit-source-id: 177e540e1372e7dfcba73c594614f0225da3a10f
Summary:
In production we trim skiplists to hold a single entry which causes
the update algorithm to behave incorrectly
Reviewed By: StanislavGlebik
Differential Revision: D21976996
fbshipit-source-id: 4946be11512ac9577bdd3320913fe092d513ecdd
Summary:
Support for this was added in D20763778, however they were reverted later in
D21762641. Turned out that all cancelled requests were counting as still
"active" even though they were cancelled a long time ago. It accumulated over
time, and at some point (to be precise, at ~65K of "active" connections) thrift
server started to reject new connections.
The problem was that original diff D20763778 didn't delete ReplyState
structure, which had pointers to a few other C++ structures whose destructor
decremented the number of active connections. To be precise:
1) https://fburl.com/diffusion/qxsu4dqe - ReplyState is allocated here, and was
previously deallocated here - https://fburl.com/diffusion/9fj9ksgx
2) ReplyState has a pointer to ResponseChannelRequest
(https://fburl.com/diffusion/6au1duav), which is in our case is an instance of
Cpp2Request, whose destructor decrements the number of active requests
(https://fburl.com/diffusion/ylp2108m).
This diff actually does two things. It adds back the changes that were added in
D20763778, and also adds a fix that explicitly deletes ReplyState structure.
For convinience, you can see just the fix in D22017238.
Previously we were deleting ReplyState in send_reply function, however now
freeReplyState function should be explicitly called from the Rust code, which
should also fix a memory leak we had when `processor.call()` method returned an error
(https://fburl.com/diffusion/ymnwc3jx).
This is quite unfortunate that we have to do a manual actions to clean
up the memory - I'll try to improve it in the next diff.
Reviewed By: markbt
Differential Revision: D22018113
fbshipit-source-id: 2e9762bf7300129ad9e58e39fb93fca6f9505159
Summary:
Add logging of infinitepush (draft) commits to a separate scribe category.
The logging will also include the username and hostname of the pusher. Since
this code is shared with the public commits scribe logging, that logging will
also gain this information.
Reviewed By: farnz
Differential Revision: D21742656
fbshipit-source-id: bdbfd14db9e8aae190c634ac4bfff35b3f62bbe4
Summary: Found this wasn't used while looking at prefixblob usages
Reviewed By: farnz
Differential Revision: D22034647
fbshipit-source-id: a0d7dc02dbee9fdf685df464539323d4c200a818
Summary: incremental step on the path to modernization of this module
Reviewed By: StanislavGlebik
Differential Revision: D21937310
fbshipit-source-id: 6ae5f9eb43355efee35705cce96736bfcb19c8ca
Summary:
NOTE: this diff doesn't change the public API of the trait - that's why we do
`compat()` and `boxify()`. This change will follow later in the stack.
Reviewed By: StanislavGlebik
Differential Revision: D21937308
fbshipit-source-id: 11bef4c5087a9ac06e209685d3281483b804c394
Summary:
I'm not sure why but D14405696 added this `is_ancestor` method with a
duplicate ancestry check implementation which:
* has different behaviour when `ancestor == descendant`
* doesn't log to ODS
Let's just settle on using one implementation now.
Reviewed By: StanislavGlebik
Differential Revision: D21937312
fbshipit-source-id: 0f4e2390cb0b984f92cd4fe645d9313fa4bba70e
Summary: incremental step on the path to modernization of this module
Reviewed By: StanislavGlebik
Differential Revision: D21937311
fbshipit-source-id: 9db84c7cb166084edfc5a0638c1c884964f9984c
Summary: incremental step on the path to modernization of this module
Reviewed By: StanislavGlebik
Differential Revision: D21937309
fbshipit-source-id: 61b1b4302782cea35d3f65e6fe982f530293dd81
Summary:
This will let us serialize and deserialize thrift structures from
non-thrift sources. In my particular case I'll be using it to deserialize json
into hg client config thrift structures that are shared with the backend.
Reviewed By: quark-zju
Differential Revision: D21875371
fbshipit-source-id: 6a85f28e69f2eeb375f6639ed1dcd3c1cd957d99
Summary: The intention of this parameter was to produce human-readable JSON for tests and for manual debugging. In practice, given that the JSON response here is just a (typically short) list, there isn't really much value in prettifying. We weren't using this anyway, so let's remove it to simplify the code.
Reviewed By: xavierd
Differential Revision: D22020784
fbshipit-source-id: 09290cc667cfdb194693e6c05858af2f8905f88e
Summary: Add wrapper functions around `data::data` for files and trees. The intention is to make the code more easily searchable, and also to establish a consistent naming pattern across all the handler functions.
Reviewed By: xavierd
Differential Revision: D22020783
fbshipit-source-id: 5e7c63ff8f366b0c6aa4412d5d04d17ca6cbc6b9
Summary: Reduce the amount of repeated code in the handlers module by defining a macro to wrap async handler functions.
Reviewed By: krallin
Differential Revision: D22015557
fbshipit-source-id: 276dad4112e7019f8d5f5fa74243747a0104e1ca
Summary:
The anyhow error contains the context for the error which will lead to better
error message for the user (and us). What the code previously did was simply
using the Debug trait to print the error, and thus was missing context.
Reviewed By: DurhamG
Differential Revision: D21985745
fbshipit-source-id: 31c603d7f42e79a360541f39e4aaf0fcfbb9a14f
Summary: This diff passes a callback function `progressCallback` from `EdenServer` to make `OverlayChekcer` print fsck messages to stdout.
Reviewed By: fanzeyi
Differential Revision: D21966060
fbshipit-source-id: 317b2c8954c718b51b5295c9f8d7698eb203906e
Summary: Add an `errors` module containing an `ErrorKind` implemented via `thiserror`. All errors in the server's HTTP handles been annotated with an appropriate `ErrorKind` via `anyhow::Context`. Notably, there is specialized handling of `MononokeError`s from the `mononoke_api` crate via a new `MononokeErrorExt` trait.
Reviewed By: krallin
Differential Revision: D22012237
fbshipit-source-id: 5f6948696e4741816636f1c6fee12069ee1a4b32
Summary:
Add trait bounds to ensure that implementors of these trait can be
safely used across threads. This is particularly useful when using these traits
in generic async code -- async code typically requires these trait bounds on
data captured in combinator closures, etc.
Reviewed By: krallin
Differential Revision: D22008466
fbshipit-source-id: 48eac59ca62debc9bd293261f6079a4fa134f331
Summary: Move the `util` module out of `handlers` and split it up into multiple submodules.
Reviewed By: krallin
Differential Revision: D22008148
fbshipit-source-id: 11b286fdef8f9d93fcc10a283ec1b2f42e9c8c04
Summary:
Note: Although this diff looks big, it's all just cutting and pasting code.
Restructure the crate so that the types relevant to each EdenAPI endpoint are grouped in their own module. This means that authors of new endpoints can just add a new module and put their types there, rather than having to make edits throughout the crate.
Reviewed By: quark-zju
Differential Revision: D21983470
fbshipit-source-id: 4901662dc6b5f0a1feca1ab8d914f394114ef233
Summary: Change `HistoryResponse` so that instead of repeatedly sending the file path alongside every history entry, we group the history entries by file and send the path once per group. This will prevent the server from transmitting redundant information.
Reviewed By: quark-zju
Differential Revision: D21982558
fbshipit-source-id: f1c5d2573c97940c7bf1645ed7fef6e1887c0d42
Summary:
This is consistent with what is being done for now in hg for tests that haven't been migrated
to modern configurations yet, and ensures we get stable commit hashes in our tests: D21899139. It's already explicitly turned on on tests that want it.
In the future, this should probably be updated to use "modern configs" like the
Mercurial tests do.
Reviewed By: ikostia
Differential Revision: D22016705
fbshipit-source-id: b27f6423bf4ec5244ef3ce2e7676306165a331a8
Summary: This message is gone we shouldn't expect it anymore: D21913608
Reviewed By: ikostia
Differential Revision: D22016684
fbshipit-source-id: 97d86e9750e775c1bb3a1e75939f506cd35851c0
Summary:
This test broke when this got turned on for all tests (D21899139). It's not
enabled for other commit cloud tests there, so let's be consistent.
Reviewed By: ikostia
Differential Revision: D22016686
fbshipit-source-id: 5f4385b60fd31c89e335e971f262da1226f32254
Summary:
In the Mononoke integration tests (which are .t tests), we've started seeing
this waring show up.
We probably shouldn't have mutation enabled for most of those tests anyway
(save for the tests that actually exercise mutation), so I'm turning that off.
That said, we'd still see the warning. So, this diff refactors mutation a
little bit to only warn if we do have something to send to the server.
Reviewed By: ikostia
Differential Revision: D22016685
fbshipit-source-id: e9e8dbcc44a0fa048dca3ca7ce0961631058a2d6
Summary: Last usage was removed in D20342689.
Reviewed By: xavierd
Differential Revision: D22011195
fbshipit-source-id: 58455ad3dc77438704e80c69eb728a319ee71a39
Summary: In its current state, FORCE is a restart mode that cannot be combined with other modes (specifically `graceful`). Removing this coupling from `force` will allow us to at first attempt a graceful restart, but if the daemon is not healthy, then do a force restart, all within one call.
Reviewed By: wez
Differential Revision: D21873166
fbshipit-source-id: 079a777ff6e7b219198cbef15335244b9865f5f3
Summary:
Move the LFS server's `StreamBody` into `gotham_ext`, along with some changes to decouple it from the LFS server.
In particular, the `Content-Length` header and post-request `Sender` have been made optional fields that can be set via a builder-style interface. The LFS server's `StreamBody` has been renamed to `LfsStreamBody` and is now a thin wrapper around `gotham_ext`'s `StreamBody` that preserves the old behavior.
Reviewed By: krallin
Differential Revision: D21988855
fbshipit-source-id: a9bf9c04bb791388d761fc705ebc38472a713b65
Summary: I would like to use these utilities when building segmented changelog.
Reviewed By: krallin
Differential Revision: D21876432
fbshipit-source-id: 9022627e224bfcb155b47d696371d24e538e6f39
Summary:
Useful for keeping the insert logic working nicely even when we double insert
or have other inconsistencies.
Reviewed By: krallin
Differential Revision: D21853048
fbshipit-source-id: 958cbc0435f330749c5aae1ea0f41ecd03b01468
Summary:
If the caller provides a service identity in `repo_create_commit`, use that to obtain
write access to the repository.
Reviewed By: StanislavGlebik
Differential Revision: D21995724
fbshipit-source-id: ba218e392b979938d598a37494b0ffe496f28722
Summary: Now that apiserver is gone, we can remove the legacy parts of mononoke_api.
Reviewed By: krallin
Differential Revision: D21976615
fbshipit-source-id: 139d7ebd4a7185693f6a4fe4e5c234c143f18498
Summary:
Add the `move_bookmark` method to `mononoke_api`.
This attempts to re-use code from `repo_client`. This code isn't really designed to be called from this API, so the fit is very poor. In a future diff we will fix this up.
The `repo_client` code for force-moving a bookmark does not support running hooks for that bookmark. For now we will prevent API users from moving bookmarks that have hooks configured. This will also be addressed in a future diff.
Reviewed By: krallin
Differential Revision: D21904979
fbshipit-source-id: 42bf840489e5b04f463c69c752bcaa5174630c21
Summary:
There is no need to migrate if the obsstore is a proxy to the mutation store.
This avoids a crash reading the attribute `_all`.
Reviewed By: DurhamG
Differential Revision: D21999245
fbshipit-source-id: bfa6439982834c6835412fc175132526525af548
Summary: The test breaks, let's not enable mutation for it.
Reviewed By: DurhamG
Differential Revision: D21998151
fbshipit-source-id: 9c06485287d906837444f39faf90886933decc9b
Summary:
Some of the hg changeset structures use `comments` as the name for commit
message. But this is confusing - let's rename
Reviewed By: HarveyHunt
Differential Revision: D21974571
fbshipit-source-id: e7c5c5ad8db9b2f1343abe9101fc56a6d4287548
Summary:
Unawaited coroutines are pernicious and not errors by default. In
advance of enabling asynchronous tests, fail our tests when Python
warnings are shown.
We record warnings and report them after the test body instead of
enabling an "error" warning filter because exceptions thrown by
finalizers are silently swallowed, and most important warnings occur
during finalization.
Reviewed By: genevievehelsel
Differential Revision: D21955210
fbshipit-source-id: b1fc52ddfa9f9befbde6800f85f090862af0ef15
Summary:
The remotenames extension was enabled everywhere in production. It is
unsupported if remotenames is disabled.
Ideally the cpp tests should use the in-repo configuration instead of
maintaining its own hardcoded hgrc. For now I just did the minimal change to
unblock tests.
Reviewed By: fanzeyi
Differential Revision: D21986274
fbshipit-source-id: b632da074baf4026f331381ae8f2f8479a34e1b3
Summary:
When kicking off the background process, we were depending on the
current working directory being inside the repo. This is not always true, like
when the current command was invoked like `hg -R xxx log -r .`. Let's invoke the
debugdynamicconfig command with the explicit path.
Reviewed By: quark-zju
Differential Revision: D21983297
fbshipit-source-id: 72981ea596e62dced3918f2fc6272953ab3d84cd
Summary:
Treemanifest was recently enabled in this test, but it breaks the Windows
tests. For now, let's just go back to the previous state and we can investigate
later.
Differential Revision: D21989349
fbshipit-source-id: fb3d3d1315400f56a6166a01112affc0a4d96da6
Summary:
Somehow clone-dst isn't empty which makes the second clone to fail due to it,
let's just clone it elsewhere.
Differential Revision: D21989189
fbshipit-source-id: 125f33f3f498d0c089785018e39ee7c5b795b472
Summary: Those tests only need minor changes to be compatible with mutation.
Reviewed By: sfilipco
Differential Revision: D21913606
fbshipit-source-id: 0a2b0e08cb558960a7d0f938300f375c34457a62
Summary:
Incompatible tests are blacklisted.
The affects non-tests environment: `proxy-obsstore` is turned on.
That is in theory not very different from having an empty obsstore.
If it becomes an issue we can easily turn it off.
Reviewed By: sfilipco
Differential Revision: D21899139
fbshipit-source-id: e4a87ecced0542e7852b48b1a807775e523b3cc9
Summary: Those tests can be migrated with minor changes like rev number fixups.
Reviewed By: sfilipco
Differential Revision: D21913589
fbshipit-source-id: 0ee080ea9d70ded74c07ce6a93d309437b992bd6
Summary: Some `x` commits become `o` because they don't have successors.
Reviewed By: sfilipco
Differential Revision: D21913601
fbshipit-source-id: 8d23edd4daa948433969460be9e6488865be7fb0
Summary:
I found myself friendly editing the output of run-tests.py:
Failed test-failure.t: output changed
Failed test-failure-unicode.t: output changed
to extract file names:
test-failure-unicode.t
test-failure.t
Then use the file names as parameters to other commands, like `run-tests.py`
again, or `hg revert`, or `hg shelve`.
Change the output format to print the filenames directly to make it easier to
use.
Reviewed By: sfilipco
Differential Revision: D21913609
fbshipit-source-id: 4b1d2265f384ad9afbe478b7ff0d6e543eff00df
Summary: We're getting rid of obsstore. The extension is no longer needed.
Reviewed By: sfilipco
Differential Revision: D21899142
fbshipit-source-id: 6e4dd2b2ddfa9325c19d6d1d9951099748260904
Summary:
The feature is off in production.
It calls the "scan changelog" code path in pullcreatemarkers.py, which
is also unused in production.
This allows us to remove a test that no longer passes with the mutation config.
Remove it.
Reviewed By: sfilipco
Differential Revision: D21913600
fbshipit-source-id: 23850fca2d1dc3d135d25954f00a08dc6d13972e
Summary:
The test uses `obsolete.revive`, which is no longer supported.
The alternative is to use unhide / undo which revives commits by changing
visibleheads directly.
Reviewed By: sfilipco
Differential Revision: D21913591
fbshipit-source-id: fea8c00692ba84c3a079012176970efb5aa29fae
Summary: We're migrating off obsstore. The test is no longer relevant.
Reviewed By: sfilipco
Differential Revision: D21913592
fbshipit-source-id: 8a721a9eea19ad04027f014727cdf0e6fb9a6b60
Summary:
The test is mostly about obsmarkers and is not that valuable with mutation.
Delete it since we do have mutation + histedit test coverage.
Reviewed By: sfilipco
Differential Revision: D21913594
fbshipit-source-id: 8c785a698fcf7c0d9f947e961c66960c13ea1111
Summary:
The old code only compares successors without checking predecessors, and skips
creating markers even if predecessors do not match. Update it to also check
predecessors. This is needed to maintain the `x` commits in
test-fb-hgext-pull-createmarkers-hide-later.t when switched to use mutation.
Reviewed By: markbt
Differential Revision: D21913610
fbshipit-source-id: 1fec2f96544236dd675ceb8c1a399bcef6cd0ede
Summary:
The test is coupled with legacy logic (obsmarker, phase exchange, over
ssh/http, bundle1, etc.), is flaky (6%-10% flaky score), and hard to maintain.
It provides little value since we probably want Thrift / Serde / HTTP protocols
instead of improving bundle2. Therefore remove it.
Reviewed By: sfilipco
Differential Revision: D21913603
fbshipit-source-id: 9d3d7410e60b67aadf816c917949772f991d2e65
Summary:
They are no-op with mutation, which is enabled in production. Therefore remove
the template keywords and tests.
Reviewed By: sfilipco
Differential Revision: D21913602
fbshipit-source-id: 40ef3a915d591d6bcae2ca7931e388b5d5e00c25
Summary:
Pruned commits are handled by visibility. Ignore them when calculating
`obsolete()`.
This change makes transition to mutation smoother. The latter does not allow
obsoleting commits without successors.
Practically, this means some `x` commits will show as `o`s.
Reviewed By: sfilipco
Differential Revision: D21913596
fbshipit-source-id: 8a6495d040595ead1f6f08cf81e095e25cd8ff1f
Summary: This makes transition to mutation smoother.
Reviewed By: sfilipco
Differential Revision: D21913599
fbshipit-source-id: 74cc0b7f0608a952eded4a40291b56b291a221e5
Summary:
The code path printing that message is rarely used in production setup. This
makes transition to mutation easier.
Reviewed By: sfilipco
Differential Revision: D21913595
fbshipit-source-id: 3aa22575c11e38dbb7d4c4cbc7aad8dde2e755ed
Summary: This makes transition to mutation smoother.
Reviewed By: sfilipco
Differential Revision: D21913608
fbshipit-source-id: b9ba470572ed68dff75104bba9dbb5c8622dcfb0
Summary:
The summary callback is only used to display "obsoleted n changesets".
Remove it to make transition to mutation smoother.
Reviewed By: sfilipco
Differential Revision: D21913597
fbshipit-source-id: e47fa9df0033d1302ffc8dfa3b9885eb75861f4b
Summary: This is needed to enable mutation and disable the old obsstore in more places.
Reviewed By: sfilipco
Differential Revision: D21913607
fbshipit-source-id: 2e1ef1e7df58fb134d541f550b6fb90508b9289a
Summary:
This makes code relying on `repo.obsstore` work with the new mutation backend.
Namely, commands like `debugobsolete` will work in tests.
Reviewed By: sfilipco
Differential Revision: D21899149
fbshipit-source-id: b4906ededdc32cfd9c9b7efbf2c85600a0c4a8f4
Summary:
These particular unit tests essentially just verify that these types (which are trivial wrapper structs around `Vec`s) implement `IntoIterator`. As such, they don't really provide value (the `IntoIterator` implementations are trivial*) but require updates whenever the format of the items produced by the iterators changes. This adds a maintenance burden with little value to show for it, so let's just get rid of these.
*The only thing non-trivial is that for `HistoryResponse`, the iterator does convert the entries from `WireHistory` to `HistoryEntry`, but the actual logic of this conversion is tested separately in `api.rs`, so testing it here is redundant.
Reviewed By: quark-zju
Differential Revision: D21983471
fbshipit-source-id: beebad53abb9ce5801b1738af92f250405c50990
Summary:
This diff fixes `eden rage` command on Windows.
`eden doctor` and `eden redirect` related rage items are disabled as they are not implemented on Windows yet.
Reviewed By: chadaustin
Differential Revision: D21962673
fbshipit-source-id: 14399a89c26eead8fe5114e0a3764644ee86a1ed
Summary:
Some Thrift clients require that values of type `string` must contain
valid UTF-8, and fail to decode otherwise. Ensure that EdenError's
message is always valid UTF-8.
Reviewed By: fanzeyi
Differential Revision: D21890761
fbshipit-source-id: 93b554e0bc6648c685f5fa54612576bd95d95ae5
Summary:
Add some functions for validating and producing valid UTF-8 to be used
in an upcoming diff.
Reviewed By: fanzeyi
Differential Revision: D21890510
fbshipit-source-id: b25144a34f1df91c72e8ed776b1ee7c1d68344c8
Summary: `getpack` requires list and will crash on set.
Reviewed By: xavierd
Differential Revision: D21968128
fbshipit-source-id: 75956f5adfcdb8082608e65f041007e58fc36c5c
Summary:
The documentation for PRJ_NOTIFICATION_FILE_RENAMED points out that the
filenames can be empty strings to represent files moved to and from the
repository itself. By not special casing this we were trying to create and/or
remove an empty file at the root of the repo which would lead to an overlay
corruption.
Reviewed By: genevievehelsel
Differential Revision: D21961579
fbshipit-source-id: fbfd872ca0377cb90224eba5505ff406812d51a7
Summary:
Reading the 00changelog.i entirely can be very expensive both in time and
memory. Since the 00changelog.i is now copied, then truncated and copied back
in place, we no longer have the limitation that prevents Windows from using
mmap for it, so let's do it.
Reviewed By: quark-zju
Differential Revision: D21968805
fbshipit-source-id: c0d19593e1f565c93b9243a63c87d03cfbc821d8
Summary: This diff fixes `'__isset' is deprecated` problem when building `//eden/fs/service:edenfs`
Reviewed By: simpkins
Differential Revision: D21966922
fbshipit-source-id: c3b12134f3f84db0a97f3975dae5d64a2c98641a
Summary: It was used by APIServer only, now when it's decomissioned we can delete this.
Reviewed By: HarveyHunt
Differential Revision: D21954670
fbshipit-source-id: a16bbbbb9dfe17eda5ff1ee6267251708ee4d562
Summary:
Enables treemanifest for about 100 more tests. To make them pass I had
to expose the gettreepack capability from peers, which just works, and enable
treeonly and sendtrees by default in the tests, which I probably should've done
before anyway.
Reviewed By: quark-zju
Differential Revision: D21894955
fbshipit-source-id: 7a6a4d453824fb8c81a797a5487bf2ecc2b67761
Summary: This diff resolves TODO to make a helper function a crate private function as the only dependency (APIServer) is now gone.
Reviewed By: StanislavGlebik
Differential Revision: D21954238
fbshipit-source-id: 7cfc49534b7ab8af6035f7a87fdc763127dce318
Summary:
APIServer is deprecated, tw jobs stopped and deleted.
This diffs removes the source code.
Reviewed By: krallin
Differential Revision: D21839442
fbshipit-source-id: 5a4089d9205d8b0061c8aa01dcd74674fe9baca8
Summary:
On Windows, if a file is memory mapped it cannot be truncated, unfortunately,
with the 00changelog.i being almost 500MB, this means that every time Mercurial
starts, the file needs to be read entirely. With the code being moved to Rust,
this read is done twice, resulting in a memory consumption of over 900MB.
Instead of always trying truncate the file in place, let's just try to truncate
it first, and then copy it, truncate it and then move it back in place. This
will allow us to memory map the 00changelog.i on Windows, which should
significantly reduce the memory consumption of Mercurial.
Reviewed By: DurhamG
Differential Revision: D21953553
fbshipit-source-id: e0f447fea017a34c8da8cdbbc6914cee7b5b2bee
Summary:
We need to support pushing legacy flat manifets commits without push
rebase. Currently we rebuild the tree on the server and recompute it's hash,
which can result in a tree hash instead of a flat hash for the pushed commit.
Since we're provided with the correct hash via the push, we should just use that
directly.
Since the hash isn't validated, it opens some opportunities for receiving
incorrect data from the server, but we can disable this functionality at some
point, once all legacy repos have been merged in. Then we can get rid of the
need for flat manifest hashes entirely once we switch to a new hash scheme.
Reviewed By: quark-zju
Differential Revision: D21962513
fbshipit-source-id: 8cdaf3a4e2aba73d5fc97fcb78821d2c88f5284f
Summary: D21880220 renamed the `depth` parameter in Mononoke's history fetching functions to be `length`. This diff makes the same change for EdenAPI's `HistoryRequest` struct.
Reviewed By: StanislavGlebik
Differential Revision: D21948599
fbshipit-source-id: fe8649a5789f07d8262ad3d5e2f477a8b50f2c6f
Summary:
Add a new `subtree` endpoint to the EdenAPI server, which fetches trees using the underlying server-side logic for the `gettreepack` wire protocol command. This is intended for use in situations where compatibility with `gettreepack` is desired when using HTTP for tree fetching.
The name of the endpoint is up for bikeshedding. It seemed weird to name the endpoint `gettreepack` since that name is a verb and refers to a "pack" which is not relevant in this context (there are no wirepacks or packfiles involved). I chose the name `subtree` since the endpoint logically returns all of the nodes underneath a given node in the tree (though in the most frequent case, the node will be the root node and therefore the subtree will be the entire tree).
In practice, this initial implementation is not ideal because it buffers the trees in memory, which is problematic because `gettreepack` requests are likely to produce a very large number of trees. Later in this stack, the endpoint will be updated to produce a streaming response instead.
Reviewed By: StanislavGlebik
Differential Revision: D21782764
fbshipit-source-id: 726925858352c33c923da1979da9d20fbcf930f6
Summary:
Several of the structs used by EdenAPI were previously defined in Mercurial's
`types` crate. This is not ideal since these structs are used for data interchange
between the client and server, and changes to them can break Mononoke, Mercurial,
or both. As such, they are more fragile than other types and their use should be
limited to the EdenAPI server and client to avoid the need for extensive breaking
changes or costly refactors down the line.
I'm about to make a series of breaking changes to these types as part of the
migration to the new EdenAPI server, so this seemed like an ideal time to split
these types into their own crate.
Reviewed By: krallin
Differential Revision: D21857425
fbshipit-source-id: 82dedc4a2d597532e58072267d8a3368c3d5c9e7
Summary: There's over 150 lines of code here, or around 1/3rd of lib.rs. Move to a separate module, so that growth of tests won't make lib.rs too long
Reviewed By: ahornby
Differential Revision: D21952798
fbshipit-source-id: 4e4edf4fde5a6626e2c33c8ea30ec85e6f3c6bf1
Summary: These were minimal conversions of old-style tests - use `.compat()` so that we have "true" async tests.
Reviewed By: ahornby
Differential Revision: D21950533
fbshipit-source-id: 71608201c4fe475d07b9ff3fafa138036f3dad5b
Summary: We used to inline smaller blobs into the data table, and only chunk bigger blobs. Performance tests show that we don't have significant issues with round trips to our MySQL infrastructure, so let's simplify the problems of deduplication and hard linking by always storing the data in chunks, and having the keys directly return significant data.
Reviewed By: ahornby
Differential Revision: D21939500
fbshipit-source-id: 6ad73b25ac859fb1b4f067a3016516b713e8b2f5
Summary: [Folly] Cut the FOR_EACH_KV macro, which may be replaced by a combination of range-for and structured bindings.
Differential Revision: D21826182
fbshipit-source-id: ce4712afd3d0d7806eb1fca8c97009da117f982e
Summary:
Add a disabled test that illustrates that binary data fails to
serialize between edenfs and the CLI with the current Thrift
implementation.
Reviewed By: fanzeyi
Differential Revision: D21890379
fbshipit-source-id: 725e99751e1d62c39f4059ec0f1197857c47e3b4
Summary:
The Python 2-and-3 Thrift API is sort of deprecated and does not
handle binary data in `binary` fields. In advance of migrating to the
modern Python 3 API, remane eden.thrift to eden.thrift.legacy.
Reviewed By: fanzeyi
Differential Revision: D21889697
fbshipit-source-id: a745ee8977999acbfb383a4edebe81d8deb1734e
Summary: Now that the blobstore_sync_queue has no unique key we do not expect duplicate key errors, and would like to know of any other problems occuring on insert.
Reviewed By: StanislavGlebik
Differential Revision: D21618062
fbshipit-source-id: 85991184234a807553388b456ad76971104000a3
Summary: I made a change before landing my last diff and didn't retest.
Reviewed By: xavierd
Differential Revision: D21944268
fbshipit-source-id: a5c820cef5b0fc019c7be5d096db9d0e7b3e5a44
Summary:
Introduced a new local commit metadata storage system, smallcommitmetadata, which stores a simple local JSON file for a mapping of (node, category) -> value.
This data can be manipulated with the debugsmallcommitmetadata command, and can be consumed in the smartlog using the smallcommitmeta template func.
Total number of entries can be limited with smallcommitmetadata.entrylimit configuration option, which defaults to 100.
Reviewed By: DurhamG
Differential Revision: D21673611
fbshipit-source-id: 2239a47867118dd86b15944058505ddf67548549
Summary:
Let's record the caller of the hg command that's using deprecated
logic. In most cases the caller will be the unix username, but we'd like to get
services to set an environment variable with their oncall so we could associate
hg usage with specific oncalls and eventually automatically file tasks for using
deprecated functionality.
Reviewed By: markbt
Differential Revision: D21891753
fbshipit-source-id: a4de0d348c583777cc3eeb9be26510e6af8b3a27
Summary:
Marks a number of commands as deprecated. The default ui.deprecate()
behavior is to just log the usage to scuba, so this diff doesn't actually change
any behavior.
Reviewed By: markbt
Differential Revision: D21891192
fbshipit-source-id: 4611e31cf630eee4f07443fe242c759c3cb3fe31
Summary:
Adds a simple ui.deprecate() function for deprecating code paths. It
has several levels of deprecation and can optionally automatically escalate
through those levels across a specified time frame.
The levels are:
- Log - just logs the usage to scuba
- Warn - same as Log but also prints a warning to the user
- Slow - same as Warn but also sleeps for 2 seconds
- OptIn - throws an exception, but hints that the exception can be bypassed by
setting 'deprecated.bypass-XXXX=True'
- Block - throws an exception that cannot be bypassed
I also call it from a few locations that are expected to be deprecated already.
Reviewed By: markbt
Differential Revision: D21890496
fbshipit-source-id: faddb301888ef75cc71d46ffb7374f3fe3d044bd
Summary:
`chg` is now integrated as `#chg-compatible` by the test framework. The helper
script no longer works.
Reviewed By: markbt
Differential Revision: D21895009
fbshipit-source-id: 257ed081fbd1a6acc97afbfa2934134e389ab400
Summary:
This requires `--hidden` for `hg tip` to show the pending commits in hooks.
The real fix would be handling pending visibility changes in metalog layer.
Reviewed By: markbt
Differential Revision: D21894321
fbshipit-source-id: 3b2296c9c38b37aee2c57eea0ad03c3acaab7507
Summary:
The test depends on lots of implementation details of obsmakers (ex. prune
markers, etc). In production we use mutation templates tested by
test-mutation.t. Therefore remove the test.
Reviewed By: markbt
Differential Revision: D21894319
fbshipit-source-id: 549423b1f5f45cf567b3c95e5fe3ccfa30b6695a
Summary:
The mount path is required in 2 places: at mount time, and when deleting files.
It turns out that the second one doesn't require it as PrjDeleteFile specify
the filename to be: "A null-terminated Unicode string specifying the path,
relative to the virtualization root, to the file or directory to be deleted."
Thus, we only need it during mount time, so let's pass it to the start method.
Reviewed By: fanzeyi
Differential Revision: D21797080
fbshipit-source-id: 27f5d18841ac077c81ad3f6b3686e20917505c41
Summary:
This will simplify some later diffs where the channel construction doesn't
change any state.
Reviewed By: fanzeyi
Differential Revision: D21797077
fbshipit-source-id: 67d9ee693b067a08986e9bd33f9d91f9a1dadac1
Summary: This is already created, no need for re-recreating it.
Reviewed By: fanzeyi
Differential Revision: D21676314
fbshipit-source-id: 8e3eecfb6069f9bed5bfec7ad6631ed8830d796d
Summary:
These are pure implementation details of ProjectedFs, and don't need to be in
the class, let's move them onto an anonymous namespace in the cpp file.
Reviewed By: chadaustin
Differential Revision: D21676315
fbshipit-source-id: 4fa7088a10256581d7eed2c7cbd90c30a7fef3d2
Summary:
I find this approach to be slighly less error prone due to not having
to manually specify the array size in advance. We lose the field names,
but they should be pretty self explanatory.
Reviewed By: chadaustin
Differential Revision: D21676319
fbshipit-source-id: db4dc993d3dd2a502ae188b08b8d4230939c0415
Summary:
This moves the edenToWinPath inside Windows specific files and makes
the interface clear about the kind of path that is expected.
Reviewed By: fanzeyi
Differential Revision: D21676318
fbshipit-source-id: d153bb543da5f481e94f83d52e45c6cc871d1f54
Summary:
The only reason tests need to set an FsChannel is for calling
cleanupPrjfsCache. For consistency sake, let's do the same as what we do on
unices: verify if we have a valid channel first.
Reviewed By: wez
Differential Revision: D21676316
fbshipit-source-id: 022ca91d32a7ab047a79acccd7d17dfd7906d527
Summary:
The only missing piece on Windows is the bind mount/redirection due to
folly::Subprocess not being present on Windows.
Reviewed By: wez
Differential Revision: D21676313
fbshipit-source-id: a5ba09be04c94b66edf9d40884753afa3865def2
Summary:
We want to use these on Windows too, so let's rename them to not be FUSE
specific.
Reviewed By: wez
Differential Revision: D21676317
fbshipit-source-id: 9f1bde9e0467d8828d4168a69581c8fea54669cd
Summary:
This moves it out of EdenServer and into EdenMount directly. This is closer
to what is happening in the Linux code and thus better.
Reviewed By: fanzeyi
Differential Revision: D21663501
fbshipit-source-id: 84420fcb37586cc2695568f0167b39f91c9bfb63
Summary:
Instead of having 2 different functions for Windows and other unices, let's
only have one. This allows more of the mount code to be shared between both.
Reviewed By: wez
Differential Revision: D21517477
fbshipit-source-id: 185ce864aecdf9a7880a9c705df444ec1e39319a
Summary:
Add a permission checker that verifies that the user is permitted to make
requests on behalf of a service. This uses a different ACL than the usual repo
ACL.
Reviewed By: mitrandir77
Differential Revision: D21886531
fbshipit-source-id: bceb63aed12c11ebe2a3d65a2560c014267b0e20
