Summary: In the process of factoring out generally useful parts of the LFS server to `gotham_ext`, it seemed like the `lfs_protocol` crate was the logical place for the Git LFS MIME type constant to live. In addition to moving it, this diff swaps out the deprecated `lazy_static` crate with `once_cell`.
Reviewed By: krallin
Differential Revision: D21193938
fbshipit-source-id: 81dc23e8f37a6c0a45ae44443807e5e21214bcd5
Summary: Move the various body types out of the LFS server into `gotham_ext`. `StreamBody` was intentionally left behind for now since it contains some LFS-specific logic that would need to be factored out before it can be moved.
Reviewed By: krallin
Differential Revision: D21193941
fbshipit-source-id: 638b9e93e9dc7385f7fde9dbb3a2392ad0e18385
Summary: Move `SignalStream` out of the LFS server into `gotham_ext`. This is a step towards extracting all of the functionality needed to support streaming bodies in `gotham_ext`.
Reviewed By: krallin
Differential Revision: D21193940
fbshipit-source-id: 832a5254c80e4ee085ece371b45b38a4519403f3
Summary: Move the `HttpError` type out of the LFS server into `gotham_ext` so it can be used by the EdenAPI server too.
Reviewed By: krallin
Differential Revision: D21193937
fbshipit-source-id: dff59e3ae995fe5771db47174a96e31b2c9f4c73
Summary:
The new merged EdenMount uses the sqlite overlay to track the status. Removing the calls to CurrentStatus from EdenMount.
This change is required here before we can switch to the merged EdenMount. Later in this stack we will add calls to the Inode APIs to report the FS changes.
Reviewed By: simpkins
Differential Revision: D20895074
fbshipit-source-id: 5e04684203f5735ff9068d3609b256991d1569f4
Summary: This diff is merging the Windows version of MountTest with the POSIX version. The merged MountTest is build and tested later in this stack with the Merged EdenMount.
Reviewed By: simpkins
Differential Revision: D20480864
fbshipit-source-id: 65e9402f1b03c81166835a6a605053a1bf011ddc
Summary: The diff merge the Windows version of EdenMount into POSIX version. We don't start using the new all platform version of EdenMount in this diff. This switch needs few more changes and is done later in this diff stack.
Reviewed By: simpkins
Differential Revision: D20480878
fbshipit-source-id: 42bfaee8769beb7a2ac45cfcff5dda2a52a7dcb0
Summary: This diff implements DirList for Windows and use it for readdir implemetation. On Windows readdir will return all the entries in one single call.
Reviewed By: simpkins
Differential Revision: D20480871
fbshipit-source-id: 15abb337c55c5016debeb0680a1a3a7063b341c3
Summary:
This diff introduces a sqlite based overlay, which stores the directory inode and its entries in the sqlite database. This is similar to FsOverlay but doesn't support all the functionality. Sqlite overlay in this diff is only designed to handles the requirement for Eden on Windows.
We did not build sqlite overlay as a virtual class because as of this diff the interface and functionality of this is different from fsOverlay.
Reviewed By: simpkins
Differential Revision: D20480870
fbshipit-source-id: c87cb2ffd11c6c0c7a10bc8dfaf6164e0e442b2b
Summary:
Add two utility programs for ad-hoc debugging of EdenAPI. EdenAPI requests and responses are encoded as CBOR, which is not easy to work with manually on the command line. In order to allow debugging the HTTP API using tools like `curl`, we need tools that can generate raw request payloads and interpret CBOR responses.
The utility programs included in this diff are:
- `make_req` - Can construct EdenAPI request payloads from a human-editable JSON representation.
- `data_util` - Can list, validate, and extract the contents of an EdenAPI data response.
These tools can be used by themselves or as part of a pipeline. See test plan for examples.
Reviewed By: xavierd
Differential Revision: D21136575
fbshipit-source-id: d1ac8d92964614005078a6ac76dd0835c29a80a5
Summary:
Tl;DR Reduces costs of fuse request mertics by reducing lock contention.
D20922194 adds tracking for FUSE request metrics, this makes tracking those
metrics more efficient. Since every user request goes through the FUSE channel,
we want to reduce the cost of these metrics as much as possible (originally
mentioned in a comment D20922194).
The synchronization used to track the metrics is costly especially
when the lock is contended.
To reduce the cost, each FuseChannel will have a thread local copy of metrics.
Each will still be synchronized to allow for reading the metrics and for
Requests moved to other threads that will need to access the metrics. However,
the lock should be contended less often since only requests from a single fuse
channel thread will access it.
Reviewed By: chadaustin
Differential Revision: D21043792
fbshipit-source-id: ce58a0cbce334095976233bfac7578d39c81bb55
Summary:
`scmutil.revsingle` resolves the commit using the revset language layer,
which will trigger auto pull logic. The import helper does not want any
kind of auto pull logic. Therefore use `repo[rev]` instead, which resolves
the commit hash directly and bypasses the revset / auto pull logic.
Reviewed By: singhsrb
Differential Revision: D21196495
fbshipit-source-id: 5a51057a731523bbb643c7e264d6902dcfbb9059
Summary: Move the MutationEntry type to the Mercurial types crate. This will allow us to use it from Mononoke.
Reviewed By: quark-zju
Differential Revision: D20871338
fbshipit-source-id: 8de3bb8a2673673bc4c8a6cc7578a0a76358c14a
Summary:
Add debug logging and perf counters for the number of mutation entries stored
during `add_entries`, and the number of mutation entries fetched during
`all_predecessors`.
Reviewed By: StanislavGlebik
Differential Revision: D21065934
fbshipit-source-id: 9b2ff9720116e6a168706f994655daffb18d0ffc
Summary:
This commit adds the `add_entries` method to the mutation store, which
allows Mononoke to add new entries to the store for a given set of
commits.
It is expected that the client will provide all of the mutation entries related
to the commits it is sending. This may be too much information, as in normal
operation we will already know about the predecessors. In that case we can
avoid additional work by just storing the entries directly related to the new
commits.
If the client has made commits while offline, then some mutation entries may
refer to predecessors that are not known locally. In this case we must search
through the mutation history looking for the last commits we did know about,
and add all the in-between entries, too.
Reviewed By: StanislavGlebik
Differential Revision: D20287383
fbshipit-source-id: e5fb42bc4da7873c3a5aafd83684d374d9155bca
Summary:
Add the Mononoke Mercurial mutation store. This stores mutation information
for draft commits so that it can be shared between clients. The mutation
entries themselves are stored in a database, and the mutation store provides
abstractions for adding and querying them.
This commit adds the `all_predecessors` method to the mutation store, which
allows Mononoke to fetch all predecessors for a given set of commits. This
will be used to serve mutation information to clients who are pulling draft
commits.
Reviewed By: krallin
Differential Revision: D20287381
fbshipit-source-id: b4455514cb8a22bef2b9bf0229db87c2a0404448
Summary:
Make the default output for `scsc log` shorter by only including the first line of the commit message, and omitting less interesting fields like commit extras.
The full details are hidden behind a `--verbose` flag, similar to `hg log`.
Reviewed By: mitrandir77
Differential Revision: D21202318
fbshipit-source-id: f15a0f8737f17e3189ea1bbe282d78a9c7199dd9
Summary:
Multiple functions in cmdlib were looking for
`"mononoke-config-path"`. Make it into a constant and provide a helper function
to reduce duplication.
Further, update `read_common_config` to accept `impl AsRef<Path>` to make
calling the function easier.
Reviewed By: farnz
Differential Revision: D21202528
fbshipit-source-id: 96cad817ed47be0f207965ad2bc33af13ca8b5fd
Summary:
Add the `commit_history` method to the source control service.
This methods returns the history of the commit, directly from the changelog.
This differs from `commit_path_history` in that it includes all commits, whereas `commit_path_history` will skip commits that do not contain changes that affect the path.
Differential Revision: D21201705
fbshipit-source-id: dbf1f446c106620620343122176eccd5d809779c
Summary:
Add a method to get the history of a changeset. This differs from the history
of a changeset path, even the history of the root directory, in that all
changesets are included, even empty ones.
Differential Revision: D21179877
fbshipit-source-id: e19aac75fc40d8e9a3beb134e16a8cdfe882b791
Summary: Add a new unit test for `commit_path_history`. We will use this test to contrast against `commit_history`.
Differential Revision: D21179878
fbshipit-source-id: 318aa34d8d80f61c1e52d4053d5aead5a71e864c
Summary: The new MembershipChecker and PermissionChecker traits will generalize access to various permission/acl systems (like LDAP) and leave the implementation details hidden behind an object trait.
Reviewed By: StanislavGlebik
Differential Revision: D21067811
fbshipit-source-id: 3bccd931f8acdb6c1e0cff4cb71917c9711b590b
Summary: Make it possible to traverse fsnodes in walker.
Reviewed By: ahornby
Differential Revision: D21153883
fbshipit-source-id: 047ab73466f48048a34cb52e7e0f6d04cda3143b
Summary: File is getting too large - let's split it
Reviewed By: farnz
Differential Revision: D21180807
fbshipit-source-id: 43f0af8e17ed9354a575b8f4dac6a9fe888e8b6f
Summary:
The part of status that lists what files have changed is called
PendingChanges. This diff introduces the initial stub for PendingChanges. The
pending changes algorithm involves three parts:
1. Looking at files on the filesystem for changes.
2. Looking at files in the dirstate map for changes.
3. Looking at the content for any files that we were unsure of during steps 1
and 2.
This diff puts the basic state machine in place, and accepts the basic
information about the working copy (the root and what type of filesystem it is).
In the future we might have it detect what type of filesystem it is, but for now
this makes it easy.
Reviewed By: xavierd
Differential Revision: D20546898
fbshipit-source-id: a3030b7c846b3cb2fcba805b7fe4744df7c5764e
Summary:
treestate.get_filtered_keys passes directory paths to the filter
function and returns directory matches with a trailing '/' on the end. This
makes it difficult to act as a path normalization function when the caller
doesn't know if the path is a file or directory.
It seems like we can just strip the trailing '/' before exposing the strings to
the caller (both as filter inputs and as get_filtered_keys outputs).
This is useful in the following diff that adds a case normalization crate.
Reviewed By: xavierd
Differential Revision: D20880881
fbshipit-source-id: 6e9f419178b4e278844244bd6aff2fc10e09d2cd
Summary:
This logic will be used in a variety of places (update workers, status,
etc). Let's move it somewhere common.
Reviewed By: xavierd
Differential Revision: D20771623
fbshipit-source-id: b4de7c1d20055a10bbc1143d44c55ea1045ec62a
Summary:
In a later diff we'll need to be able to hand a reference to the
TreeState to the pending changes iterator. We'd like to be able to hand a
Rc<RefCell<TreeState>> but cpython requires that its fields implement Send. The
simplest solution is to use Arc<Mutex<_>>. Once we finish Rustifying all of this
code we can drop the cpython requirement that this work across threads and
downgrade this to a Rc<RefCell<_>>.
Reviewed By: xavierd
Differential Revision: D20546904
fbshipit-source-id: a4a1ce6973f53b3bb95f227616149f98fcd780e0
Summary:
PathAuditor will be needed for native status soon. Let's move it into
the workingcopy crate.
Reviewed By: xavierd
Differential Revision: D20546906
fbshipit-source-id: ef69f88ee828a72e82b5e944cc7913f391bd8a2f
Summary: Update the test to match the latest output.
Reviewed By: xavierd
Differential Revision: D21181146
fbshipit-source-id: 3b6190c9ae8e73309c0b7cb3c48d8a6646bc7fb4
Summary: This will help us debug slow commands
Reviewed By: xavierd
Differential Revision: D21075895
fbshipit-source-id: 3e7667bb0e4426d743841d8fda00fa4a315f0120
Summary:
Several of the tests started edenfs of fake_edenfs processes and didn't kill
them at the end of the test. This adds proper cleanup functions to try and
kill these processes when the test exits.
This also eliminates using pexpect in these tests, since it isn't actually
necessary here.
Reviewed By: wez
Differential Revision: D21084096
fbshipit-source-id: 4e92a99a5c398d4a78830ac51507ba34d7a6c0b1
Summary:
This updates the various systemd-related tests to use EdenTestCaseBase,
and allows us to delete the EnvironmentVariableMixin and
SystemdUserServiceManagerMixin classes.
The main goal of this clean-up is to make it easier to consolidate most of the
systemd-related code into just a few locations, so that we can more easily
disable the systemd-related tests in build environments where systemd is not
supported.
Reviewed By: wez
Differential Revision: D21084098
fbshipit-source-id: d5e05254c689c28751fe48d2dc38d722c7e77ed3
Summary:
This adds a new `EdenTestCaseBase` class to serve as the base class across a
number of our integration tests and some of our CLI tests.
The main goal of this is to allow eliminating many of the annoying `*Mixin`
classes used in a lot of our integration tests. These mixin classes are
annoying since they result in complicated multiple inheritance, and it can be
tricky to ensure that the method resolution order behaves the way you want.
The systemd tests in particular use a lot of mixins, which gets complicated.
These mixin classes are also awkward from a Python typing perspective, and the
systemd tests end up resorting to just declaring different APIs in several
places when `typing.TYPE_CHECKING` is True.
The fact that `EdenTestCaseBase` has a `contextlib.ExitStack` member variable
should make it easier for us to eliminate these mixins moving forward: rather
than using mixins that use inheritance and assume a `self.cleanUp()` method
exists, we can transition this code to standalone functions or context
managers, and they can take the `ExitStack` variable as an argument if
necessary.
Reviewed By: wez
Differential Revision: D21084097
fbshipit-source-id: 77ee457b7debe6f584f630e3e30f79fe634a2026
Summary:
If a commit operation results in a commit that is already present in the
repository, we must revive the commit, rather than create a new one. In this
case, if obsmarkers are disabled, we can't create a revive marker.
Reviewed By: quark-zju
Differential Revision: D21179879
fbshipit-source-id: 320c70b0ca1e0f384733e09e0e1dd77ac3a255f9
Summary:
Update the integration tests to only run git integration tests if EdenFS was
built with git support.
Reviewed By: genevievehelsel
Differential Revision: D21000162
fbshipit-source-id: 0a015ae93a9420f040be702096e0b378fe138e4c
Summary:
Update the code to work around the fact that Pyre refuses to unwrap Optional
member variables. Store the data in local variables so that Pyre will allow
unwrapping these objects after we check against None.
Reviewed By: genevievehelsel
Differential Revision: D21000165
fbshipit-source-id: 59e8430de30ec35213a53b06670b16c34be85593
Summary:
Update some of the eden integration tests to use the `eden.config` module
instead of trying to import the `__manifest__` module, which is only present
in Buck-based builds.
Reviewed By: wez
Differential Revision: D21000167
fbshipit-source-id: 347082a32e842752e5539a8e6c8addde97a16eee
Summary:
Update most locations in edenfsctl to report the version number that was built
into the edenfsctl binary at build time, rather than querying the RPM database
for the installed RPM version. The RPM behavior only works on to RedHat-based
Linux systems, and the currently running process doesn't necessarily have to
have come from the RPM.
The one place where we do still attempt to print the RPM version is in the
`edenfsctl rage` report, when running on Linux.
Reviewed By: wez
Differential Revision: D21000168
fbshipit-source-id: 0fb747e71b6950d74f22c458efa0dfcbd45270bd
Summary:
This updates the top-level CMakeLists.txt file to compute package version
information, and expose this to C++ code in `eden-config.h`, and to Python
code in a new `eden/config.py` module.
Previously we exposed an `EDEN_VERSION` macro for the C++ code in
`eden-config.h`, but this was not initialized or used anywhere. Now the
top-level CMakeLists.txt file computes appropriate version information and
exposes the package name, version, release, commit ID, and build time in these
configuration files.
The version selection logic in CMakeLists.txt based largely on the code that
wez wrote for watchman in D20636833.
Reviewed By: wez
Differential Revision: D21000164
fbshipit-source-id: db1a1035f1eefec058bbad558d35e113005e454e
Summary:
This exposes metrics for the live FUSE requests (the duration
of the longest outstanding request and the number of outstanding
requests).
Because FUSE is the interface through which the user mostly interacts
with the file system they provide good metrics to judge if the perfomance
of eden is normal, or there may be an issue.
Exposing these counters this way will send them to ods, so it will not only
allow for debuging current issues, but can be used to look back at previous
problems. This data could also be used for alerting or more proactive
remediation.
Metrics are exposed per checkout to allow seeing which checkout was
having issues. This data will aggregated in `eden top` to be used as
an overall health indicator, but should more information be needed it
will be logged in ods.
Reviewed By: chadaustin
Differential Revision: D20922194
fbshipit-source-id: 16208883417acb77b62bf712cfdd9068c5420303
Summary:
Without this change the following code will fail surprisingly:
In [1]: def f():
...: return m
...:
In [2]: f()
NameError: global name 'm' is not defined
Reviewed By: singhsrb
Differential Revision: D21140299
fbshipit-source-id: f0a6509badcd9314a33a5ca0c78a60bb847d63c7
Summary:
Make `ml`, `serv`, `bin`, `hex`, `util` available as they are handy.
Remove less useful names like `e`.
Reviewed By: singhsrb
Differential Revision: D21140300
fbshipit-source-id: ca012da6e33fad64b352c117595538e10d1fad83
