Summary: This change introduces a bail macro that allows tagging errors using the syntax `bail!(fault=Fault::Request, "my normal {}", bail_args)` or `bail!(Fault::Request, "my normal {}", bail_args)`.
Reviewed By: DurhamG
Differential Revision: D22646428
fbshipit-source-id: a6ec2940001b26db8ddc3a6d3620a1e17406c867
Summary:
lower the debug level for scs proxy hash logging so that this does not cause so
much noise in the logs of production eden
Reviewed By: chadaustin
Differential Revision: D22668574
fbshipit-source-id: 1a7c2a4706514c0ef0bb848424681eef9316d296
Summary: Fix flaky test-walker-validate.t. There can be more than one route to the bad filenode, so wildcard the src and via fields when matching the output.
Reviewed By: StanislavGlebik
Differential Revision: D22664371
fbshipit-source-id: f4d880187ec2b557fb5f69ad546c2486d150b337
Summary: Use `create_getpack_v2_blob` instead of `create_getpack_v1_blob` for fetching file content because the former also provides metadata, which is required to support LFS.
Reviewed By: quark-zju
Differential Revision: D22564950
fbshipit-source-id: 2835160a9dfd18b80cd13e4a5dbcf6f4ce2f4579
Summary:
For large lists it's much more convenient to specify them in a file - we are
not limited by cmd line size limit.
Reviewed By: krallin
Differential Revision: D22595023
fbshipit-source-id: 93035208700f981453eaf98f84341a86f2f1c04d
Summary: This is to be able to inspect `LiveCommitSyncConfig` from our admin tooling.
Reviewed By: StanislavGlebik
Differential Revision: D22497065
fbshipit-source-id: 3070890b7dc2a4075a5c15aca703494e33ee6530
Summary:
We have three different types of manifests that store file type and content -
hg manifests, fsnodes and unodes.
Let's add a command that verifies that these manifests are consistent.
There's some copy-paste in the code when listing manifests (e.g. list_fsnodes,
list_unodes etc are quite similar). There might be a way to have less
copy-paste, but given that each of the functions have some small differences it
doesn't really seem worth it.
Reviewed By: krallin
Differential Revision: D22663631
fbshipit-source-id: 487be8611df218472cec1899f34367906794484b
Summary:
We are unifying C++ APIs for accessing optional and unqualified fields:
https://fb.workplace.com/groups/1730279463893632/permalink/2541675446087359/.
This diff migrates code from accessing data members generated from unqualified
Thrift fields directly to the `field_ref` API, i.e. replacing
```
thrift_obj.field
```
with
```
*thrift_obj.field_ref()
```
The `_ref` suffixes will be removed in the future once data members are private
and names can be reclaimed.
The output of this codemod has been reviewed in D20039637.
The new API is documented in
https://our.intern.facebook.com/intern/wiki/Thrift/FieldAccess/.
drop-conflicts
Reviewed By: yfeldblum
Differential Revision: D22631599
fbshipit-source-id: 9bfcaeb636f34a32fd871c7cd6a2db4a7ace30bf
Summary: `commit_validator` cannot just use latest `CommitSyncConfig` to validate how a commit was synced between large and small repos. Instead, when querying `synced_commit_mapping`, it needs to pay attention to `version_name` used to create the mapping. Then it needs to query `LiveCommitSyncConfig` for a config of this version and use it as a validation basis.
Reviewed By: StanislavGlebik
Differential Revision: D22525606
fbshipit-source-id: 6c32063b18461d592d931316aec7fd041bcc1ae4
Summary:
Currently the repo lock is checked only once at the beginnig of unbundle future. That unbundle process take some time and during that time repo can be locked by someone.
We can reduce that possibility by creating additional future, which will check the repo in the loop and poll both futures for whoever will finish first.
Reviewed By: StanislavGlebik
Differential Revision: D22560907
fbshipit-source-id: 1cba492fa101dba988e07361e4048c6e9b778197
Summary:
This is the next step in exposing version, used to sync commits in read
queries. The previous step was to query this from DB, now let's also put it
into an enum payload. Further, I will add consumers of this in admin and
validation.
Note that ideally, `RewrittenAs` should always have a version associated with
it, but:
- right now this is not true in the DB (needs backfilling)
- even when I backfill everything, I would not like to error out just at
reading time, I would prefer the consumers to deal with the absense of a
version in rewrtitten commits.
Therefore, I decided to use `Option` here.
Reviewed By: StanislavGlebik
Differential Revision: D22476166
fbshipit-source-id: 5bc27bb21b7e59c604755ef35aa5d3d2c3df527e
Summary:
When we change `CommitSyncConfig`, we want to not have to restart `scs` servers, and instead have them pick up the new config by using `LiveCommitSyncConfig`.
This diff turned out larger than I expected, mainly due to the need to introduce various things around `TestLiveCommitSyncConfig`:
- `TestLiveCommitSyncConfig`: the trait implementer to use in `mononoke_api::Repo`
- `TestLiveCommitSyncConfigSource`: the helper struct to keep around for new values injection (very similar to how our `ConfigStore` has an inner `ConfigSource`, which can also be `TestSource`, but here injected values can be `CommitSyncConfig` instead of JSON).
- all the places in integration tests, where `setup_configerator_configs` is now needed (I plan to start setting up configerator configs always in a separate diff, as it is cheap)
Here are the explanations for a few things I think may be not immediately obvious:
- I removed the `Clone` bound from `LiveCommitSyncConfig` trait, as `Clone` traits [cannot be made into trait objects](https://doc.rust-lang.org/book/ch17-02-trait-objects.html#object-safety-is-required-for-trait-objects)
- `TestLiveCommitSyncConfigSource` vs `TestLiveCommitSyncConfigSourceInner` discrepancy is to ensure nobody should instantiate `TestLiveCommitSyncConfigSourceInner` outside of `live_commit_sync_config/src`
- I am aware of the ugly discrepancy between the main `--mononoke-config-path`, which is used to load initial configuration and can be both a file-based and a configerator-based config; and `--local-configerator-path`, used to override config sources for `Tunables` and `LiveCommitSyncConfig`. Ideally these two should just be unified somehow, but that is a little out of scope of this diff (I've already added it to the dirt doc though).
- in `mononoke_api::Repo` there are methods `new_test_xrepo` and `new_test_common`, which changed from maybe accepting just a `CommitSyncConfig` to now maybe accepting both `CommitSyncConfig` and `LiveCommitSyncConfig`. It can be made a bit cleaner: I can just query `CommitSyncConfig` from `LiveCommitSyncConfig` in `new_test_common` and avoid having two args. I was too lazy to do this, lmk if you feel strongly about it.
Reviewed By: StanislavGlebik
Differential Revision: D22443623
fbshipit-source-id: 0d6bbda2223e77b89cc59863b703db5081bcd601
Summary: After we derived the bonsaichangesets (D22455297 (c315c56c05)), we want to move a bookmark in small increments to reveal the commits to the users (https://fburl.com/wiki/zp9hgd7z step 8). This diff adds this functionality to repo_import and automate this step.
Reviewed By: StanislavGlebik
Differential Revision: D22598159
fbshipit-source-id: 01db898f07a0b7be50c3f595e78931204f33bb46
Summary: Add context to show the affected key if there are problems deserializing an alias.
Reviewed By: krallin
Differential Revision: D22629544
fbshipit-source-id: 1718d4187386e37038bb5c958db2659bd5b54cfd
Summary:
The spanset has the assumption that `0..len(repo)` are valid revs.
That's not true with segmented changelog. So reduce the dependency on the
assumption.
Reviewed By: kulshrax
Differential Revision: D22519586
fbshipit-source-id: a493d26d6d69a36966f4a037f87a03593b697cbd
Summary: Match the Rust `IdSet` name and be more different from `spanset`.
Reviewed By: kulshrax
Differential Revision: D22519590
fbshipit-source-id: 5be7990c2157dc09a37067ea7402a49da2a3127c
Summary: This makes tests depend less on revision numbers.
Reviewed By: DurhamG
Differential Revision: D22468669
fbshipit-source-id: 74a06930faa3e6ee9d246ecc718c2a3740f57a54
Summary: The `__len__` operation returns the number of commits.
Reviewed By: DurhamG
Differential Revision: D22402200
fbshipit-source-id: 3084dc02dc1e9999f839f22e314eb09e430cd6d2
Summary:
It turns out the Python world needs the integer range API in many places.
Deprecating them is non-trivial. Therefore expose the API.
Reviewed By: DurhamG
Differential Revision: D22402201
fbshipit-source-id: de31d15c18e5f4e0f8826f71315b98ad58b1764e
Summary:
The Rust strip logic works for both revlog and segmented changelog. Use it in
tests for easier segmented changelog migration.
The old revlog strip has a lot of complexities on linkrev. Since this is only
used for testing, I avoided the complexities by just saying all linkrevs are
invalidated so we can skip the complexities of dealing with filelog or
manifest, or generating a temporary bundle and unbundling them later.
Most test changes are because we no longer strip filelogs. So unbundling
them will not add new file revisions.
`test-repair-strip.t` is removed because it is only about "Permission Denied"
cases. With the new code path file logs are not stripped and the test is not
relevant.
This does not affect hgsql as it has the Rust logic totally disabled.
Reviewed By: DurhamG
Differential Revision: D22402198
fbshipit-source-id: 1c207393f59a8a3a601cfbdf7f07534b8acf46bf
Summary: Add a new config option (`remotefilelog.http`) to make Mercurial use an EdenAPI-backed remote store in place of the SSH remote store.
Reviewed By: quark-zju
Differential Revision: D22492536
fbshipit-source-id: efcc780b65d4fd64b70728d056eadff59ed78128
Summary: It has been on beta for a while.
Reviewed By: DurhamG
Differential Revision: D22519355
fbshipit-source-id: 805341d6a8bf13b6847830aee55e2cf19940a626
Summary:
Extend `_log_exception` to inspect exceptions for metadata, and log any that is found to Scuba. Eventually I'd like to merge at least some of this information from the `perfpipe_hg_errors` table into the `perfpipe_dev_command_timers`.
Introduce `Tagged` mixin and `Fault` enum Python types for tagging Python exceptions.
Introduce `debugthrowexception` for testing Python error handling flow.
Reviewed By: DurhamG
Differential Revision: D22585764
fbshipit-source-id: f6300ae25db25d5e75fab9c2a03fb4f9beb06c1f
Summary:
Smartlog commands with `--commit-info` additionally print out the changes in
the current commit, in the same format as `hg status`. Add labels so that
the output is correctly colorized.
Reviewed By: farnz
Differential Revision: D22625583
fbshipit-source-id: e4658ecabbf8bbd2121bdf284eae9766ac98641e
Summary:
This allows clients to get more commits in the followup query, also it lets the
clients know that the limit was reached.
Reviewed By: markbt
Differential Revision: D22576875
fbshipit-source-id: 93de20b1033cd5d0cdf902a418d7b727b03d2d08
Summary:
We already traverse in useful order, let's preserve this order as it's natural
for clients to expect some kind of topological ordering.
Reviewed By: markbt
Differential Revision: D22576873
fbshipit-source-id: 32a6f0de1ba9cc473e57b5a69fde538dfe8a3d75
Summary: This prevents us from abuse in form of crazy big queries.
Reviewed By: markbt
Differential Revision: D22576876
fbshipit-source-id: 0e407d79ba367f1b42faa82e4757053e43001e50
Summary:
The customers need our queries to return a predictable amounts of output
whilist limiting the depth doesn't really do it: the output may be
theoretically exponential.
Reviewed By: markbt
Differential Revision: D22576874
fbshipit-source-id: 65c793b229889c0cf26af693537ed6dbc0d33ebd
