Summary: See title. Starting to go through and manually enable autodeps for these targets.
Reviewed By: stanleyz510
Differential Revision: D56361731
fbshipit-source-id: 95f5f004d6d9e86e3e98b9c1cd6b6007deb91fd1
Summary:
From D56593397:
'channel' used after it was moved
fbcode/eden/fs/inodes/EdenMount.cpp:2046:43: move occurred here
```
channel_2 = std::move(channel)](
^
```
fbcode/eden/fs/inodes/EdenMount.cpp:2040:25: the use and move are unsequenced, i.e. there is no guarantee about the order in which they are evaluated
```
channel->getAddr(),
^
```
Reviewed By: jdelliot
Differential Revision: D56645977
fbshipit-source-id: b12ff23e0eca8e5d6092e871a2cbcd1b3689d66c
Summary:
# Context
FilteredFS mounts use FilteredBackingStores and therefore they use FilteredObjectIDs instead of underlying ObjectIDs. That means the getAttributesFromFiles endpoint will spit out FilteredObjectIDs instead of ObjectIDs. This test previously checked only for ObjectIDs. For FilteredFS mixins of this test we should expect to see FOIDs instead of OIDs.
# This diff
Adds a utility to translate underlying ObjectIDs into FilteredObjectIDs based on the type of object that's passed in. This causes all the re-enabled tests to pass.
Reviewed By: jdelliot
Differential Revision: D56593399
fbshipit-source-id: a428782c05617d8ad34f87d8c29796ed36bc5946
Summary:
# Context
The scm_type of FilteredFS mounts isn't "hg", instead it's "filteredhg" to indicate a FilteredBackingStore is being used instead of just the underlying HgQueuedBackingStore.
This caused Doctor to skip some health checks because the hg-specific logic was gated on `CheckoutConfig.scm_type == "hg"`
# This diff
This diff adds "filteredhg" to the list of scm_type that will trigger hg-specific doctor checks.
Reviewed By: genevievehelsel
Differential Revision: D56593396
fbshipit-source-id: c12b7d9a605a268d95dfeca5bdbff50e47d4cd25
Summary: Some tests are now passing, so we can reenable them. Other tests are still failing. I'm documenting why they're failing in this diff.
Reviewed By: genevievehelsel
Differential Revision: D56593395
fbshipit-source-id: cb2fc18221adb2a4dd091002b39aeceb08824c72
Summary:
# Context
The Eden thrift error on commit/parent mismatch should look like this:
```
error.EdenError: error computing status: requested parent commit is out-of-date: requested 681e0c9978e24003b6432cf86f8a5e9b51d0f4af, but current parent commit is 49aacbce01b5b7729b5a7ad6d551d7202d8b8c04.
```
On FilteredFS repos, it looks like this (notice the extra `(` and `null` at the beginning and end):
```
error.EdenError: error computing status: requested parent commit is out-of-date: requested (681e0c9978e24003b6432cf86f8a5e9b51d0f4afnull, but current parent commit is (49aacbce01b5b7729b5a7ad6d551d7202d8b8c04null.
```
The extra parts are added because we naively print out FilteredRootIDs as hex (see https://www.internalfb.com/code/fbsource/[8a227dc30524]/fbcode/eden/fs/inodes/EdenMount.cpp?lines=1781-1787). FilteredRootIDs have extra bits on either side (the length at the beginning and the filter at the end), so those parts of included in the error message.
# A potential solution:
For FilteredFS repos, I want to render only the underlying RootID (so the `(` and `null` don’t get printed). We have renderRootId() that kind of does this, but it renders the underlying RootID as binary. See:
```
error.EdenError: error computing status: requested parent commit is out-of-date: requested h�x�@�C,�o�^�Q���, but current parent commit is I�����r�Zz��Q� -��.
Try running `eden doctor` to remediate
```
I want it to be rendered as hex. If I round trip the conversion by going from HexFilteredRootId -> BinaryUnderlyingRootId -> HexUnderlyingRootId, then everything works as expected on both FilteredFS and Vanilla EdenFS repos. The extra round trip is basically a no-op for Vanilla EdenFS repos.
# This diff
This diff implements the round-trip so that FilteredFS repos print out correctly formatted underlying ObjectIDs.
Reviewed By: jdelliot
Differential Revision: D56593397
fbshipit-source-id: eb41f5fad5dff0289ed4ddad4270b91a3aa3c3d4
Summary:
# Context
If the user supplies a backing-repo path that points to an existing Eden repo, then clone will try to copy that existing repo's config. This works in most cases, but it will override user supplied options in some cases. One such case was when the user supplied a --backing-repo option. This option was entirely ignored in favor of whatever value was contained in the existing repo's checkout config.
# Solution
We should teach clone to respect the user-supplied option or default to "hg" instead.
# This diff
Fixes a test that was originally broken by this bug.
Reviewed By: genevievehelsel
Differential Revision: D56593402
fbshipit-source-id: 2cb47a177669137728172e007980b53a2a03d44e
Summary:
# Context
If the user supplies a backing-repo path that points to an existing Eden repo, then clone will try to copy that existing repo's CheckoutConfig. This works in most cases, but it will override user supplied options in other cases. One such case was when the user supplied a --backing-repo option. This option was entirely ignored in favor of whatever value was contained in the existing repo's checkout config.
# Solution
We should teach clone to respect the user-supplied option or default to "hg" if no option is supplied.
# This diff
Adds a check to ensure the backing store types match. If not, it replace the backing store field with the option supplied by the user (or "hg" by default)
Reviewed By: genevievehelsel
Differential Revision: D56593400
fbshipit-source-id: 608e26a6556cd6f0df6ed06a963ea1bfbfc0ea77
Summary: The help text is wrong in a few ways. Let's fix it so that it prints a more correct/helpful output.
Reviewed By: genevievehelsel
Differential Revision: D56593398
fbshipit-source-id: bf4845aa3c1a1b67ffb2e376cfaa36b21b01fa85
Summary:
# Context
We confirmed that these tests weren't causing hangs on macOS a while ago. It's worth turning these back on so that we have sufficient coverage for FilteredFS repos.
Some additional tests have been broken since we disabled them, and some have been fixed. The rest of this stack cleans up, fixes, and reenables some tests.
# This diff
This diff re-enables running FilteredFS mixins of Hg tests.
Reviewed By: genevievehelsel
Differential Revision: D56445381
fbshipit-source-id: 78a6603609fcfa1e1465e1687d6ed90581ad18d7
Summary:
# Context
An XFN partner reported a bug that occurs when a user modifies the filter file that's currently active. In other words, commiting changes to the active filter file leads to unexpected behavior.
**Expectation**
Committing changes to a filter file will cause those changes to be immediately reflected in the repo working copy.
**Actual behavior**
Committing changes to a filter file doesn't actually update the working copy accordingly. Instead, the changes (especially those that cause a directory to go from filtered -> unfiltered) will cause the newly unfiltered files to show up as "missing" in `sl status`.
# Fix
We need to fix the `sl commit` hook to update the working copy when filter changes are commited.
# This diff
Fixes the behavior of commiting a filter change so that the filter change is immediately reflected after the commit is made.
Reviewed By: quark-zju
Differential Revision: D56445400
fbshipit-source-id: a79296cd285ea21b577f608361378cbf8a36ad2c
Summary:
# Context
An XFN partner reported a bug that occurs when a user modifies the filter file that's currently active. In other words, commiting changes to the active filter file leads to unexpected behavior.
**Expectation**
Committing changes to a filter file will cause those changes to be immediately reflected in the repo working copy.
**Actual behavior**
Committing changes to a filter file doesn't actually update the working copy accordingly. Instead, the changes (especially those that cause a directory to go from filtered -> unfiltered) will cause the newly unfiltered files to show up as "missing" in `sl status`.
# Fix
We need to fix the `sl commit` hook to update the working copy when filter changes are commited.
# This diff
Adds a test that showcases the broken behavior. This test will be used to verify the bug fix works as expected.
Reviewed By: sggutier
Differential Revision: D56445402
fbshipit-source-id: 5fd0fb66a3721d0a0696ba741a8a23f737cebde4
Summary:
Part of a D56490217 split.
This diff uses the new handler.exe output parser, and changes the logic in that if a process can't be killed it will continue trying to kill whatever can be killed (instead of giving up). The idea here is that if the user wanted us to kill all processes but we can't, he/she will have to do it anyway, so we might as well get as much of the work done.
The kill logic is smarter, skipping the process own tree completely.
Process are order in priority order, starting with Hubhub.exe, then dotnet, then the rest.
Reviewed By: MichaelCuevas
Differential Revision: D56731085
fbshipit-source-id: d5bbc4b4bcf46f8bddf2bfd5500f72bd622603d2
Summary: This allows reposetup functions to detect if a repo is a submodule or not.
Reviewed By: sggutier
Differential Revision: D56481294
fbshipit-source-id: ef090233211bb86692b474f411b544d830927669
Summary:
We need to derive fsnodes for the commits as they're imported, because otherwise we'll be rederiving it for all ancestors of the commits being processed every time, leading to O(n2) complexity.
This is because without this change we request derivation using the large repo `InMemoryRepo`, which doesn't persist the data to the production blobstore afterwards.
I created T187426389 to go further and automatically derive everything for the commits as they go.
Differential Revision: D56706360
fbshipit-source-id: d107cb32673a8259e445d4c71b2d24715b6944a8
Summary:
I noticed that import was going very slow, but that was being spent on validation of submodules that weren't even being modified by a changeset...
So I looked into it and it turns out we were always deriving the fsnodes for the large bonsai, even when we didn't need to.
This doesn't really solve the problem of slow derivation, but saves some unnecessary delays for most changesets.
Differential Revision: D56701028
fbshipit-source-id: 158c3992b2102a4642930329064732b48d4b32a5
Summary:
Introducing a whole new exception so we can configure different behavior in ServiceRouter, as well as keeping separate metrics.
This is a crude mechanism for now, but it should be enough to distribute the load better when getting overloaded by a few calls. A followup diff will make the behavior configurable by method and repository.
Reviewed By: clara-9
Differential Revision: D56715749
fbshipit-source-id: 775f52d716c5984132a02c799f1e501440887e85
Summary:
When eden rm fails due to resources being locked a usual reason is adb server.
We can't kill adb because it runs as admin, but luckily we can ask it to stop.
Since it's started automatically when needed, this is harmless.
If it wasn't running, it's just no op.
Same with buck2.
Also - fix a missing arg in an instantiation of WinFileHandlerReleaser :-(
This diff is part of a split of D56490217
Reviewed By: MichaelCuevas
Differential Revision: D56590257
fbshipit-source-id: 156d3ed72039cc8783a719640551f114abc39671
Summary: new directory metadata fields to cover augmented trees
Differential Revision: D56521895
fbshipit-source-id: 03280aa5cd0e73ccbb876e7d479268844e120b4f
Summary: I am initially adding this instrumentation to just one method to get some experience in practical use. I chose this method because it's pretty resource-intensive. The qps is low enough that I'm not worried about the overhead, and the JK allows us to turn this off if it gets in the way at all. It might be worth memoizing `memory::get_stats` with a short TTL (well under a second though, or it wouldn't be useful with sudden spikes).
Reviewed By: clara-9
Differential Revision: D56701239
fbshipit-source-id: 3f2e525df983e5e8414da45c6de885c268684b7f
Summary: Along with its tests, move this utility struct to its own module.
Differential Revision: D56569170
fbshipit-source-id: ba319a49824f4977b4263affcca00f2492d93678
Summary:
These tests use a different manifest type than the other tests, as well as a custom store. This is error-prone, for example `test_unfold_dir_from_dirs` has a cyclic manifest at `2`, which should be impossible. It's also an unnecessary duplication of code.
Switch it to use the same test manifest as the rest of the tests.
Differential Revision: D56569171
fbshipit-source-id: ea69f3016ed0272540dd04f9317bd3750a18ccbb
Summary: We want to use these test manifests to interact with bonsai diff which requires the file type, so add it to the manifest in the same way that we do for other manifests.
Differential Revision: D56569173
fbshipit-source-id: f74fc4a1f52a46d845a985f4920975dcd3e0cf71
Summary: Start cleaning up the manifest tests by extracting its private test manifest type to its own module file.
Differential Revision: D56569172
fbshipit-source-id: 8ec8392bc26701b87e4b84c8b39f6b01ffeff4ce
Summary: Fixes the issue described in D56633699.
Reviewed By: mitrandir77
Differential Revision: D56690832
fbshipit-source-id: ed61affed8c4d3d6c96ff64071b67c2c793cb8b3
Summary:
# Context
Suffix queries are a type of query that tend to take a very long time. We would like to determine how long this type of glob queries takes in order to determine impact of improvements to this type of query.
# This Diff
This diff adds telemetry to findGlobs that adds additional logging if a suffix glob query is present.
# Technical Details
This telemetry is done via creating a SuffixLogRequestScope that creates the log when it gets cleaned up.
Logging is done to XLOG via a string output, and Scuba via the SuffixGlob object
Reviewed By: jdelliot
Differential Revision: D55259063
fbshipit-source-id: 1d802832395089b35139cc2c00fafd1b2420fc26
Summary: BlobCache also stores an `EdenStats` object, this mirrors that setup, but also takes the `EdenStats` object into the base class rather than the derived classes
Differential Revision: D56369797
fbshipit-source-id: 609ca5db298bc83d946f3ae48eeba30c34f7abf3
Summary: Templatizes ObjectCache with a generic `ObjectCacheStats` object so `TreeCache` and `BlobCache` can be used separately but can still share the same base class. `FakeStats` was added in tests for compilation reasons, which shares the same member variables as `TreeCacheStats` and `BlobCacheStats`
Reviewed By: jdelliot
Differential Revision: D56369795
fbshipit-source-id: 326e4ffccf70a36db6aae457a966b2710927ae29
Summary:
Prior to {D54500949}, the generated `:foo-rust` target used to depend on `:foo-rust-services` and needed to refer to its contents.
After that diff, there is no longer any generated code that depends on the `-services` crate, so we no longer need to pass the Thrift compiler a crate name by which to refer to the contents of the `-services` crate. This diff cleans that up.
Reviewed By: shayne-fletcher
Differential Revision: D56726365
fbshipit-source-id: d0e1cc1d302a6e848699c0e2e78692c9c6f9af5f
Summary: This simply call all the previously implemented update functions of every data structure
Reviewed By: mitrandir77
Differential Revision: D56691652
fbshipit-source-id: 7c16b4bade415403dc09aa6ad582aeb9ab20d331
Summary: The way commit cloud does update right now is deleting old records and inserting new ones, let's add a function into all data structures that supports that using the primitive sql ops.
Reviewed By: mitrandir77
Differential Revision: D56691625
fbshipit-source-id: c6ba941c6f455471a811248eeaf7df449ec3266f
Summary: Most of removed bookmarks/heads/snapshots come as strings, let's add some functions to handle these operations
Reviewed By: mitrandir77
Differential Revision: D56691591
fbshipit-source-id: bccb41a6c9909d0c74e0dfd99fb176849d213c90
Summary:
This is a basic impl, not ready for testing cause we don't correctly have the heads dates since we need to retrieve them from SCS.
This diff only contains the commit cloud service part, edenapi declaration and implementation will come in further diffs.
This diff also adds some data structures like ReferencesData that will be declared in edenapi later and used here. This is just me tryin to keep diffs as "small" as possible.
Reviewed By: mitrandir77
Differential Revision: D56647152
fbshipit-source-id: 0d294cbf5f470f1c2406e8f9fda383fd327a7342
Summary: Same as the sqlite queries but these use "node" instead of "commit". Also, in bookmarks they use "bookmarks" instead of "workspacebookmarks"
Differential Revision: D56637389
fbshipit-source-id: 42a446d2587bf93659932cbf114b92b47d7b2dfe
Summary:
So for commit cloud there's two DBs, `xdb.commit_cloud`(prod) and a sqlite one we use for test and that we're aiming to move to as we move more things over to mononoke. I'm trying to make these queries as deduped as possible so there's a seamless db usage instead of having to define everything twice.
One of the main differences is that we store hex-encoded commits in `xdb.commit_cloud`, so, the previous diff added functions that encode changesets depending on the db type, and here we just use them in the existing data structures. Technically we could still simply use `HgChangesetId`for sqlite and bytes for mysql. But again, I'm trying to make this as seamless and possible and abstract all of the minor type casting within the db and avoid duplication.
Differential Revision: D56597877
fbshipit-source-id: a02ea5ee0f55e06baaae9b45532e2da2251a565b
Summary:
There are some cases when we want tests to work both with EdenFS and non-EdenFS. Ideally we should not specify anything, so no references to `#require eden` / `#if eden`, etc. should exist.
Before this diff newly added tests only would work on non-EdenFS if it was explicitly required or if there was something that tested against it, similar to what was done in D56220001.
The reason just checking for EdenFS would make EdenFS available in tests' PATHs was that there is some code that eventually updates the list of binaries that should be available in the path:
https://www.internalfb.com/code/fbsource/fbcode/eden/scm/sapling/testing/t/runner.py?lines=349-356
And in turn this is later used to create the wrappers / symlinks that call to the original binaries:
https://www.internalfb.com/code/fbsource/[765627dc802b6cad309e1763479c66ca3cb1fa46]/fbcode/eden/scm/sapling/testing/t/runner.py?lines=396-397
On EdenFS .t tests we are expecting the EdenFS CLI to always be available, which this diff makes so.
Reviewed By: quark-zju
Differential Revision: D56660786
fbshipit-source-id: 33066d4e07b56a7952c61f5b06944a3c6a518231
Summary: This will be used by the next change.
Reviewed By: sggutier
Differential Revision: D56481293
fbshipit-source-id: c9ae7db76780be13eba0e447827115a2e553716c
Summary: This will be used by the next change.
Reviewed By: sggutier
Differential Revision: D56481292
fbshipit-source-id: e5492db0a4547461ef7cf434385fc5c4a1727ea4
Summary:
Test Cases for D55096200
Extends glob_test to also apply for prefetch
Changes:
prefetch always retrieves dotfiles
Reviewed By: kmancini
Differential Revision: D55323841
fbshipit-source-id: 4ec8c3bb068534d2bf885c1cb6761d810eef435d
Summary: Provide the ConfigSet content as a layer so it can be inspected.
Reviewed By: muirdm
Differential Revision: D56592490
fbshipit-source-id: d443f85c477568e6bf5210e9dc8a96bd2ed6a0c4
Summary: Make it easier to figure out what those ConfigSets are.
Reviewed By: muirdm
Differential Revision: D56592495
fbshipit-source-id: 2a09c8651fb71fc2d93bcf655953abf5be230c0e
Summary: This makes it easier to use.
Reviewed By: muirdm
Differential Revision: D56592500
fbshipit-source-id: 6979dd85e505b3b5b397784066ce32326d3f5e17
Summary:
The config validation logic was migrated to Chef in D56651559.
Remove this function to unblock config layout refactoring.
Reviewed By: muirdm
Differential Revision: D56592494
fbshipit-source-id: 04a6a6f5bf4da7131f8aab45909467c8d74b5d1c
Summary:
# Context
This Diff separates prefetchFiles from globFiles. The intention behind this change is to allow us to identify when globFiles is intentionally being called vs as part of a prefetcing operation.
# This Diff
- Adds new thrift endpoing prefetchFilesV2
- Switches over the usage of globFiles in prefetchCli to use prefetchFilesV2
- adds debugPrint flag to PrefetchParam. The intended use case is to allow the user to control when they want a a prefetch to print out the list of prefetched files.
# Technical Details
- prefecthFilesV2 is basically the same as the existing globFiles except it returns a PrefetchResult. This is an object containing an optional glob. The glob will be present if debugPrint is true
Reviewed By: genevievehelsel
Differential Revision: D55096200
fbshipit-source-id: e689ca2d2d818bfd3a33e49a1e734f059b9e0a13
Summary: This diff adds a static counter for number of items in the cache. A later diff adds time-based `ObjectCache` counters
Reviewed By: MichaelCuevas
Differential Revision: D56369793
fbshipit-source-id: 39d5576f1c9d3d614a13e0fa034fad03a4b4860b
Summary: Moving these to their repective classes as it will make a followup diff to register more counters a lot cleaner
Reviewed By: MichaelCuevas
Differential Revision: D56319373
fbshipit-source-id: 8a4cb9203af0801634feed33e1c64853bc3f1b10
