Summary:
add test for git rename, confirm it does not detect renames. After we
complete the feature, this test should fail and need to update.
Reviewed By: quark-zju
Differential Revision: D42787246
fbshipit-source-id: b65c454b3abf1cf7b25e5a501e3b4c652940a834
Summary:
Context: https://fb.workplace.com/groups/rust.foundation.team/posts/1557305454784096.
Artifact dependencies allow a Cargo.toml to declare a dependency on a binary-only crate.
```
[dependencies]
rustfilt = { version = "0.2", artifact = "bin" }
```
Previously in order to import such crates, we would need to fork the crate and put in an empty lib.rs so that the dependency would become accepted by Cargo.
Dealing with forks is substantially more burdensome than straightforward crates.io dependencies so I am excited to move away from needing them.
Reviewed By: zertosh
Differential Revision: D42398657
fbshipit-source-id: 50c2fb7138c216552a37d5231062cedf94ce60fc
Summary: Generated by `common/rust/tools/reindeer/vendor`, separated out manually from {D42808024} so that diff remains just the consequence of the 1.68.0-beta Cargo update.
Reviewed By: zertosh
Differential Revision: D42808575
fbshipit-source-id: 78e4c5279acf45de182fabe47deb6c1a7eb18337
Summary:
# Problem
In macOS Ventura, copying a file into an EdenFS NFS mount in Finder only creates an empty file, producing the error message:
> The operation can’t be completed because an unexpected error occurred (error code 100072)
This error code indicates a bad RPC struct, and it happens because we're sending a malformed response to some SETATTR requests.
# Analysis
In D40798840 (34884008f5) we added code to ignore two types of setattr requests:
- On NFS, requests that are no-op aside from updating an `atime`.
- Everywhere, requests that are entirely no-op.
Contrary to the description of D40798840 (34884008f5), it's the former type of request that can be created by `mmap`-ing a file over NFS. The latter appears to be generated at various points by macOS, including when copying a file into an NFS share with Finder.
We currently return an empty response body to that second type of request, without even a response status. As of macOS Ventura this produces an error message.
# Fix
Here we modify the `is_nop` check at the NFS level so that SETATTRs that only update `atime` become fully no-op, falling through to the FileInode check (which doesn't ignore `atime`) which then considers the request no-op. This returns up through the NFS stack to produce a well-formed NFS response.
This leaves in place the `atime`-sensitive behavior at the FileInode level.
This does not affect the invalidation behavior introduced in D35435764 (8f3f873874), which sets `mode`.
Reviewed By: xavierd
Differential Revision: D42785781
fbshipit-source-id: af174abd61b4927040f481d0a177443eb64ae8da
Summary:
oss: make 'make tests' run Python tests
'make check' tries to run 'make tests', but despite being documented, the
'tests' target does not exist in the Makefile.
Create a 'tests' target which runs the Python test suite.
Pull Request resolved: https://github.com/facebook/sapling/pull/458
Test Plan:
$ make oss
$ make check
[bunch of failures]
$ (cd tests && python run-tests.py test-debugcheckcasecollisions.t)
----------------------------------------------------------------------
# Ran 1 tests, 0 skipped, 0 failed.
---
Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/458).
* https://github.com/facebook/sapling/issues/459
* __->__ https://github.com/facebook/sapling/issues/458
Reviewed By: muirdm
Differential Revision: D42742607
Pulled By: zzl0
fbshipit-source-id: 790624f28da629fe25846d1dde1fad2aa3382911
Summary:
Like it says in the title. Sometimes boxing fields is good to ensure structs
aren't stupidly large (in particular because of rare-but-large variants).
It also allows having Prost deserialize things directly into the heap, which
makes it cheaper to move across the `Stream` you get back from Prost (since its
just a pointer now).
Reviewed By: lmvasquezg
Differential Revision: D42776199
fbshipit-source-id: 96fce99cc1802e42d2b85cfd67af38655b3b93d8
Summary:
If you ran autocargo right now on master it would fail because of this. I need
to run it later in this stack, so let's fix it.
Reviewed By: lmvasquezg
Differential Revision: D42776200
fbshipit-source-id: c95a8f0e5d34a99b11a069752fbfbde568fbecac
Summary:
Current usages favor hashmap for some reason. I don't like
lack of order stability with hashmaps so I want to be able to use the
algo with BTreeMap.
Reviewed By: markbt
Differential Revision: D42709713
fbshipit-source-id: 8a7aa923f7097c99dc17c0fe7ac251a207acda13
Summary:
There are many valid topological orders for non-linear DAGs. When there are
merges the starting point for the traversal matters a lot. If we start from one
of the heads the merge branches would be continous parts of the sorted list.
If we start from the leaves we might end up interleaving merges.
This diff allows the topo sort caller to express preference about starting
point of the traversal. All nodes are still guaranteed to be traversed.
Reviewed By: markbt
Differential Revision: D42709711
fbshipit-source-id: ced83b813dc87a4157ed433cc0cddbed952819e0
Summary:
There's no need to add each node parents to the start points for DAG traversal
as they will be definitely visited when their parent is visited.
Now because we iterate only the keys list from dict there's no need to use
hashset for dedup. But.. even if there were duplicated on that list they'd be
harmless as they'd be skipped as "visited"
Reviewed By: markbt
Differential Revision: D42709712
fbshipit-source-id: 90ffdbf934e12d0b6be210be5ac2fb411c903adb
Summary: Add some instructions about debugging stack traces which use minified .js line numbers on the entry point file into their original .ts files with proper line numbers.
Reviewed By: bolinfest
Differential Revision: D42789908
fbshipit-source-id: 9f55794c9b31fc8df22bb4c680ba57704d87e4cb
Summary:
This application has been shown to crawl the repository, to prevent future
issues, let's prevent it from doing so.
Reviewed By: chadaustin
Differential Revision: D42779963
fbshipit-source-id: 1f47ca43ca6913b13eb36f87adc728327eb00bb6
Summary: In case we want a different value, we can use this config option.
Reviewed By: muirdm
Differential Revision: D42783555
fbshipit-source-id: d3d7e1036db4635be45ecc61b0f521552dca6f75
Summary:
While D42626678 provides a way to set the max chain length per index, it's
annoying to update all users of indexedlog. Add a global option so we can
just update once for all.
Reviewed By: muirdm
Differential Revision: D42783554
fbshipit-source-id: 8bffe6de4225560c59da207f321fdaa2b65e25a1
Summary: Move global configuration to a module to make it clear that those are configs.
Reviewed By: muirdm
Differential Revision: D42783055
fbshipit-source-id: d725946f1198199c9a40c612c435a60d62838e0d
Summary:
See the previous diff for context. This diff relaxes the metadata check to only
check the log length without checking indexes. This should resolve the race
condition.
The index uses persistent data structure with explicit pointers (offsets) so
data appended by another process would simply be ignored without causing
issues.
Reviewed By: muirdm
Differential Revision: D42627387
fbshipit-source-id: 5e776a8c6b34eeb6eefbb78f249fd99c7bd05b02
Summary:
The checksum entry could form a long linked list that takes time to read from
disk. Add a way to "flatten" the chain so the linked list length could be reset
to 1 from time to time for optimized load time.
Reviewed By: muirdm
Differential Revision: D42626678
fbshipit-source-id: 57f13c235939ac416c6871eae7ec4e72b2c72f8e
Summary:
Rust makes a pretty good portability layer for efficiently doing IO
across platforms. To get accurate IO benchmark numbers on Windows, add
a 4K random writes benchmark in Rust. Because Rust calls NtWriteFile
directly, it's much faster than our C++ benchmarks.
Reviewed By: mshroyer
Differential Revision: D42624363
fbshipit-source-id: ee0f011e8006440c5dbc0b5b33f015db610fed1f
Summary:
On Windows, `os.times()` does not provide a meaningful `elapsed`:
In [1]: os.times()
Out[1]: nt.times_result(user=1.921875, system=1.3125, children_user=0.0, children_system=0.0, elapsed=0.0)
Replace it with `time.monotonic()` so the wall clock time works on Windows.
Reviewed By: muirdm
Differential Revision: D42718496
fbshipit-source-id: dd1bfc313201cc6d321ee23397168d3981d03ffb
Summary:
I need to write a custom backend for the Session middleware. Gotham in 0.6.0 used synchronous methods in the session `Backend` trait. In 0.7.0 they modified [the trait](https://github.com/gotham-rs/gotham/blob/main/gotham/src/middleware/session/backend/mod.rs#L29) to support async definitions for `persist_session`, `read_session`, and `drop_session`.
This commit also migrates a few callsites in mononoke (edenserver and lfs server). Unit tests are passing, so probably we're good?
Reviewed By: zertosh
Differential Revision:
D42745808
Privacy Context Container: L1153662
fbshipit-source-id: 8b04c8fd55c411eb003264a71136b55007e81517
Summary:
Remove `BlobRepo::get_blobstore`, which returns a clone of the repo blobstore, and replace it with the equivalent facet methods.
In some cases, the obtained clone of the blobstore was just immediately followed by a call to `.boxed()`, which constructs a new `Arc` around the clone of the blobstore. This is a double-indirection, and we can avoid this by using the `RepoBlobstoreArc` trait instead.
Differential Revision: D42747658
fbshipit-source-id: 4941068c7ccb3e4a627f4a26e0d2905e0b54d3a7
Summary: This method returns a reference to the `RepoBlobstore`. Replace with the corresponding facet trait.
Differential Revision: D42745917
fbshipit-source-id: 2202083cc2e4ee5ede9bf2de9d7d80ab926a3cff
Summary:
Teach Repo::file_store() and tree_store() to use the gitstore for git repos.
I also had to avoid a git store tree query for empty repos since git doesn't like our null ID.
Reviewed By: quark-zju
Differential Revision: D42711084
fbshipit-source-id: 399431d1478e9bcb1ffadb08e6bca0ed54f863a3
Summary:
The built-in git config file has an %include for /etc/mercurial/git_overrides.rc. Test were sensitive to this file, which is undesirable.
Fix in the easiest way possible by truncating the static config in tests starting from the first "%include".
I first tried to fix in a general way by refsuing to load any config files in tests unless they are under $TESTTMP. That didn't pan out due to various places that do exactly that.
Reviewed By: quark-zju
Differential Revision: D42711085
fbshipit-source-id: cd36fa439dd08d20ffb362c23a6296ff662b48dc
Summary:
The mononoke_hg_sync can't deal with commits will null manifests because it
tries to fetch them and they don't exist. Let's fix it by skipping over them
Reviewed By: yancouto
Differential Revision: D42753347
fbshipit-source-id: e6f3e05a7f1e0cf9253c22c6614fd941a7c8c202
Summary:
Sapling when trying to push a commit with null manifest tries to pack it first and fails miserably to obtain it. This diffs makes it skip null manifests as those don't need to be transferred.
NOTE: I've also included comment about more general problem that I've noticed. In `shallowbundle.py` we skip over all manifests that don't exist by catching exception but this exception had changed due to rust migration. **I didn't fix that problem.**
Reviewed By: muirdm
Differential Revision: D42753348
fbshipit-source-id: 9a1ae168d09d89bd0316991af88c6801e4c2f41b
Summary: This diff fixes up few places in Mononoke that fail when hitting null manifest. This fix is by no means complete but it's all that's needed to unblock pushes of commits with null manifests.
Differential Revision: D42753351
fbshipit-source-id: 7b0df1f78e5be66e3fd89b929197597331d38e98
Summary:
If the commit without parents is also one of the roots of the repo it has null hg manifest. Many things in our infra are struggling with such commits, most recently monononoke_hg_sync job. This diff shows that the issue exists but doesn't event get to syncing commits because it fails to push those commits in the first place.
I'll use this test to show the impact of next few diffs in this stack.
Reviewed By: yancouto
Differential Revision: D42753350
fbshipit-source-id: 2995fb95b28343b85c8d08a3b7c14b2fcfa56c47
Summary:
This stack adds an API for analytics tracking in ISL.
In open source builds, this will be completely disabled. It's only internally that we want to be able to track usage to understand how ISL is used.
Reviewed By: bolinfest
Differential Revision: D42466433
fbshipit-source-id: cd5e08f37be541b62b29603581880771b5af9c20
Summary:
On ProjectedFS, concurrent invalidation with placeholders being laid on disk is
racy, and the race is unfortunately not fixable from EdenFS. The way this would
manifest is that IO done on files being invalidated would randomly fail with
"Internal Error", or "File not found" while EdenFS would succeed all the IO.
The only way to prevent it is to make sure that GC always considers the atime
of a file prior to invalidating it, and since on Windows the atime is only
updated at most once an hour, files that have been accessed less than an hour
ago must never be invalidated.
Reviewed By: kmancini
Differential Revision: D42694759
fbshipit-source-id: 5ec880b2df8dc96f05b5a060a3e15950f296ffe3
Summary:
In some cases, we want to be able to express configs that must be constrained
between a lower and higher duration. This adds support for such configs.
Reviewed By: kmancini
Differential Revision: D42694760
fbshipit-source-id: b8c48a4e5cb1a09ce4a217194f3d3ef5fd8d4662
Summary:
The isSafeForInodeAccess is used to know when the root inode is initialized and
valid, but this is used in racy context. For instance, the getMountPoints
collects all the mounts whose root inode is initialized, but then releases the
lock on the mountPoints_ list, subsequently, the EdenMount's root Inode is
accessed without checking. If the EdenMount is shutdown prior to the root Inode
being copied, the root Inode might be a nullptr, causing a NULL-dereference.
This pattern can be found in all of the Thrift entry points that play with
mounts and thus are all potentially sources of EdenFS crashes. To solve this,
we need to guarantee 2 things:
- isSafeForInodeAccess must only change state while the mountPoints_ lock is
held, specifically at shutdown time,
- The root inode must be copied while the mountPoints_ lock is held.
This diff does both. For the second one in particular EdenServer::getMount is
modified to also return the root Inode. This is done instead of introducing a new
function to prevent future bugs from seeping in that may be missed at review time.
Reviewed By: chadaustin
Differential Revision: D42563083
fbshipit-source-id: c0267277a54c425f330bbd58d1dc86ec3746502d
