Summary:
Logs the python version string to scuba. This is the full string, like
'2.7.5 (default, Apr 2 2020, 13:16:51) \n[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]'
so it will require contains "2.7" or contains "3.8" type checks. I figured
having the full python information would be useful, instead of just logging the
major and minor versoin.
Reviewed By: singhsrb
Differential Revision: D22650538
fbshipit-source-id: 9bdbd5f71c2063ecdd06ed4994e00d8f9c9e0698
Summary:
In Windows 10 2004, the behavior of AF_UNIX socket changed slightly, a socket
on which a server is listening to now appears to be shown as a "reparse point"
on disk. This appears to confuse Python as `os.path.exists(socketpath)` no
longer returns `True` for a listened to socket, which means that edenfsctl
would not be able to communicate with EdenFS, even though it is up and running.
Windows still has this weird behavior where the socket is created on disk
during connect, but EdenFS knows about this and already retries several times
(D21755498 (2c5768404d)) to avoid the issue, it's therefore fine to remove this code.
Reviewed By: wez
Differential Revision: D22666081
fbshipit-source-id: e2eea3fdd7628354b8b187b76899be4e9e2fa17f
Summary: This change introduces a bail macro that allows tagging errors using the syntax `bail!(fault=Fault::Request, "my normal {}", bail_args)` or `bail!(Fault::Request, "my normal {}", bail_args)`.
Reviewed By: DurhamG
Differential Revision: D22646428
fbshipit-source-id: a6ec2940001b26db8ddc3a6d3620a1e17406c867
Summary:
lower the debug level for scs proxy hash logging so that this does not cause so
much noise in the logs of production eden
Reviewed By: chadaustin
Differential Revision: D22668574
fbshipit-source-id: 1a7c2a4706514c0ef0bb848424681eef9316d296
Summary: Fix flaky test-walker-validate.t. There can be more than one route to the bad filenode, so wildcard the src and via fields when matching the output.
Reviewed By: StanislavGlebik
Differential Revision: D22664371
fbshipit-source-id: f4d880187ec2b557fb5f69ad546c2486d150b337
Summary: Use `create_getpack_v2_blob` instead of `create_getpack_v1_blob` for fetching file content because the former also provides metadata, which is required to support LFS.
Reviewed By: quark-zju
Differential Revision: D22564950
fbshipit-source-id: 2835160a9dfd18b80cd13e4a5dbcf6f4ce2f4579
Summary:
For large lists it's much more convenient to specify them in a file - we are
not limited by cmd line size limit.
Reviewed By: krallin
Differential Revision: D22595023
fbshipit-source-id: 93035208700f981453eaf98f84341a86f2f1c04d
Summary: This is to be able to inspect `LiveCommitSyncConfig` from our admin tooling.
Reviewed By: StanislavGlebik
Differential Revision: D22497065
fbshipit-source-id: 3070890b7dc2a4075a5c15aca703494e33ee6530
Summary:
We have three different types of manifests that store file type and content -
hg manifests, fsnodes and unodes.
Let's add a command that verifies that these manifests are consistent.
There's some copy-paste in the code when listing manifests (e.g. list_fsnodes,
list_unodes etc are quite similar). There might be a way to have less
copy-paste, but given that each of the functions have some small differences it
doesn't really seem worth it.
Reviewed By: krallin
Differential Revision: D22663631
fbshipit-source-id: 487be8611df218472cec1899f34367906794484b
Summary:
We are unifying C++ APIs for accessing optional and unqualified fields:
https://fb.workplace.com/groups/1730279463893632/permalink/2541675446087359/.
This diff migrates code from accessing data members generated from unqualified
Thrift fields directly to the `field_ref` API, i.e. replacing
```
thrift_obj.field
```
with
```
*thrift_obj.field_ref()
```
The `_ref` suffixes will be removed in the future once data members are private
and names can be reclaimed.
The output of this codemod has been reviewed in D20039637.
The new API is documented in
https://our.intern.facebook.com/intern/wiki/Thrift/FieldAccess/.
drop-conflicts
Reviewed By: yfeldblum
Differential Revision: D22631599
fbshipit-source-id: 9bfcaeb636f34a32fd871c7cd6a2db4a7ace30bf
Summary: `commit_validator` cannot just use latest `CommitSyncConfig` to validate how a commit was synced between large and small repos. Instead, when querying `synced_commit_mapping`, it needs to pay attention to `version_name` used to create the mapping. Then it needs to query `LiveCommitSyncConfig` for a config of this version and use it as a validation basis.
Reviewed By: StanislavGlebik
Differential Revision: D22525606
fbshipit-source-id: 6c32063b18461d592d931316aec7fd041bcc1ae4
Summary:
Currently the repo lock is checked only once at the beginnig of unbundle future. That unbundle process take some time and during that time repo can be locked by someone.
We can reduce that possibility by creating additional future, which will check the repo in the loop and poll both futures for whoever will finish first.
Reviewed By: StanislavGlebik
Differential Revision: D22560907
fbshipit-source-id: 1cba492fa101dba988e07361e4048c6e9b778197
Summary:
This is the next step in exposing version, used to sync commits in read
queries. The previous step was to query this from DB, now let's also put it
into an enum payload. Further, I will add consumers of this in admin and
validation.
Note that ideally, `RewrittenAs` should always have a version associated with
it, but:
- right now this is not true in the DB (needs backfilling)
- even when I backfill everything, I would not like to error out just at
reading time, I would prefer the consumers to deal with the absense of a
version in rewrtitten commits.
Therefore, I decided to use `Option` here.
Reviewed By: StanislavGlebik
Differential Revision: D22476166
fbshipit-source-id: 5bc27bb21b7e59c604755ef35aa5d3d2c3df527e
Summary:
When we change `CommitSyncConfig`, we want to not have to restart `scs` servers, and instead have them pick up the new config by using `LiveCommitSyncConfig`.
This diff turned out larger than I expected, mainly due to the need to introduce various things around `TestLiveCommitSyncConfig`:
- `TestLiveCommitSyncConfig`: the trait implementer to use in `mononoke_api::Repo`
- `TestLiveCommitSyncConfigSource`: the helper struct to keep around for new values injection (very similar to how our `ConfigStore` has an inner `ConfigSource`, which can also be `TestSource`, but here injected values can be `CommitSyncConfig` instead of JSON).
- all the places in integration tests, where `setup_configerator_configs` is now needed (I plan to start setting up configerator configs always in a separate diff, as it is cheap)
Here are the explanations for a few things I think may be not immediately obvious:
- I removed the `Clone` bound from `LiveCommitSyncConfig` trait, as `Clone` traits [cannot be made into trait objects](https://doc.rust-lang.org/book/ch17-02-trait-objects.html#object-safety-is-required-for-trait-objects)
- `TestLiveCommitSyncConfigSource` vs `TestLiveCommitSyncConfigSourceInner` discrepancy is to ensure nobody should instantiate `TestLiveCommitSyncConfigSourceInner` outside of `live_commit_sync_config/src`
- I am aware of the ugly discrepancy between the main `--mononoke-config-path`, which is used to load initial configuration and can be both a file-based and a configerator-based config; and `--local-configerator-path`, used to override config sources for `Tunables` and `LiveCommitSyncConfig`. Ideally these two should just be unified somehow, but that is a little out of scope of this diff (I've already added it to the dirt doc though).
- in `mononoke_api::Repo` there are methods `new_test_xrepo` and `new_test_common`, which changed from maybe accepting just a `CommitSyncConfig` to now maybe accepting both `CommitSyncConfig` and `LiveCommitSyncConfig`. It can be made a bit cleaner: I can just query `CommitSyncConfig` from `LiveCommitSyncConfig` in `new_test_common` and avoid having two args. I was too lazy to do this, lmk if you feel strongly about it.
Reviewed By: StanislavGlebik
Differential Revision: D22443623
fbshipit-source-id: 0d6bbda2223e77b89cc59863b703db5081bcd601
Summary: After we derived the bonsaichangesets (D22455297 (c315c56c05)), we want to move a bookmark in small increments to reveal the commits to the users (https://fburl.com/wiki/zp9hgd7z step 8). This diff adds this functionality to repo_import and automate this step.
Reviewed By: StanislavGlebik
Differential Revision: D22598159
fbshipit-source-id: 01db898f07a0b7be50c3f595e78931204f33bb46
Summary: Add context to show the affected key if there are problems deserializing an alias.
Reviewed By: krallin
Differential Revision: D22629544
fbshipit-source-id: 1718d4187386e37038bb5c958db2659bd5b54cfd
Summary:
The spanset has the assumption that `0..len(repo)` are valid revs.
That's not true with segmented changelog. So reduce the dependency on the
assumption.
Reviewed By: kulshrax
Differential Revision: D22519586
fbshipit-source-id: a493d26d6d69a36966f4a037f87a03593b697cbd
Summary: Match the Rust `IdSet` name and be more different from `spanset`.
Reviewed By: kulshrax
Differential Revision: D22519590
fbshipit-source-id: 5be7990c2157dc09a37067ea7402a49da2a3127c
Summary: This makes tests depend less on revision numbers.
Reviewed By: DurhamG
Differential Revision: D22468669
fbshipit-source-id: 74a06930faa3e6ee9d246ecc718c2a3740f57a54
