Summary:
Instead of overloading the FileMetadata struct that is used for directory
listing, let's use one tailored for our use. This will enable more flexibility
when deciding what to provide ProjectedFS. For instance, we could send the
InodeNumber so we don't need to do expensive path resolution for every notification.
Reviewed By: wez
Differential Revision: D22361751
fbshipit-source-id: 4801be45d8afc3af51e0a9564d9acb0a8e32255a
Summary:
Futures are intended to be chained together and not synchronously waited one
after the other. While this may be one of the goal, it also paves the way to
enable ProjectedFS asynchronous notification handling.
While doing this, a bunch of code was moved from EdenMount.cpp to the
dispatcher itself, the rationale behind this is to follow what the unix
EdenDispacher with the long term plan to merge the 2 as much as possible.
Reviewed By: wez
Differential Revision: D22361750
fbshipit-source-id: fa679a8b94ff6f8b5a33782fdb6b129ab066c4d8
Summary:
Whenever EdenFS reads a file from the overlay, there is a risk of this
triggering a prjfs notification, especially when there is a discrepancy between
the actual state of the overlay, and what EdenFS thinks the overlay should look
like.
A simple example of this includes removing a materialized file in the overlay
when EdenFS isn't running, and running `hg status` when it's back up. The
status call will try to compute the sha1 of the file, and thus will call into
the `CreateFile` API, which will then trigger an EdenFS callback to be called.
This callback will then try to recursively acquire the inode lock, which is
already held by the sha1 computation code. And that's a textbook example of a
deadlock.
To avoid this in the first place, let's simply refuse recursive callbacks. The
sha1 code will either have to be modified to handle errors, or we need to scan
the overlay when mounting the repo.
Reviewed By: fanzeyi
Differential Revision: D22288431
fbshipit-source-id: 2b4b31eddf7debcde55c22dd704d198ec44e59b4
Summary:
At mount time, EdenFS will try to create the .eden/config file which will
indicate to the client that this is an EdenFS repo, this config also contains
the location of the socket to talk to EdenFS.
On unices, and while the .eden directory is slightly different, the content of
this directory is written onto the overlay, but on Windows, the overlay is the
repository itself. What this means is that creating this config file will end
up triggering a ProjFS callback, which can potentially lead to deadlocks if
we're not careful.
A future change will simply prevent these recursive callbacks from happening
and thus a solution needs to be found for the .eden/config file itself. Since
the file itself is tiny (about 400B), and should only be accessed once[0], the
simple approach to keep it in memory and special case it works perfectly.
[0]: Once a file was read fully by ProjFS, it's present in the overlay and
EdenFS will no longer be requested to provide it.
Reviewed By: chadaustin
Differential Revision: D22310734
fbshipit-source-id: 6b2dba2164496ebd251104d7875b51569be2471f
Summary:
Files need to be decoded before they're passed to Mercurial
prefetching.
Reviewed By: singhsrb
Differential Revision: D22736198
fbshipit-source-id: f27a85442755d9fcf6c3a572c02a23d59f314d20
Summary: I spent a good chunk of time installing eden on a fresh ubuntu vm from github, so I kept track of all of the packages that I needed to end up installing. users can install these before attempting to run the `build.sh` script in order to make sure all dependencies are there. Some of these manifested in non-obvious error messages. I'm sure as time goes on these dependencies might change a bit, but I think its a good starting point for anyone and should save a few hours of headache.
Reviewed By: chadaustin
Differential Revision: D22716112
fbshipit-source-id: 49232232a8707ef59c47487fdcc0cb41afc702e8
Summary:
When using LFS, it's possible that a pointer may be present in the local
LfsStore, but the blob would only be in the shared one. Such scenario can
happen after an upload, when the blob is moved to the shared store for
instance. In this case, during a `get` call, the local LFS store won't be able
to find the blob and thus would return Ok(None), the shared LFS store woud not
be able to find the pointer itself and would thus return Ok(None) too. If the
server is not aware of the file node itself, the `ContentStore::get` would also
return Ok(None), even though all the information is present locally.
The main reason why this is happening is due to the `get` call operating
primarily on file node based keys, and for content-based stores (like LFS),
this means that the translation layer needs to be present in the same store,
which in some case may not be the case. By allowing stores to return a
`StoreKey` when progress was made in finding the key we can effectively solve
the problem described above, the local store would translate the file node key
onto a content key, and the shared store would read the blob properly.
Reviewed By: DurhamG
Differential Revision: D22565607
fbshipit-source-id: 94dd74a462526778f7a7e232a97b21211f95239f
Summary:
In some rare situation, it is possible to have an LFS pointer in both the
packfile and the LFS store. In this case, the flags for this filenode may or
may not be empty, making the assert too strong.
Reviewed By: DurhamG
Differential Revision: D22565605
fbshipit-source-id: b82282b3f47af2a9e607f09a7a7d271ecc4e521a
Summary:
Disregard two default systemd services on Fedora 32 that cause our
systemd tests to fail.
I believe genevievehelsel is planning on removing this code soon anyway.
Reviewed By: genevievehelsel
Differential Revision: D22713393
fbshipit-source-id: b703b23a3158cb007dc2e1cb53fae36be7282719
Summary:
Pull Request resolved: https://github.com/facebookexperimental/eden/pull/32
This parsing uses the standard "subject name" field of a x509 certificate to create MononokeIdentity.
Reviewed By: farnz
Differential Revision: D22627150
fbshipit-source-id: 7f4bfc87dc2088bed44f95dd224ea8cdecc61886
Summary: If fsnodes point to non-existent content we should be able to detect that.
Reviewed By: farnz
Differential Revision: D22723866
fbshipit-source-id: 31510aada5e21109b498a26e28e0f6f3b7358ec4
Summary: Previously, `BackingStore` and all its sub-classes' `getBlob` and `getTree` methods accepted both `ObjectFetchContext` and `ImportPriority` as arguments. Now, `ImportPriority` is removed because we can get the priority from `ObjectFetchContext `
Reviewed By: kmancini
Differential Revision: D22650629
fbshipit-source-id: e1b0c57a059f11504b28b2c17d698bb58f51e1ee
Summary:
check fetch count before `getPriority()` is used. If fetch count has exceeded `fetchThreshold_`, lower the priority by 1.
Note: this diff only guarantees that the `getPriority()` function is returning the lowered priority. How the returned value is used for scheduling is handled by `HgQueuedBackingStore`
Reviewed By: kmancini
Differential Revision: D22550640
fbshipit-source-id: c032f8f72ca658618ac118dfb3ad3dcae61e9735
Summary: Previously `getPriority()` always returned a fixed priority. Now that we want `ObjectFetchContext` to lower its priority, it is better to make it have its own copy of `ImportPriority`.
Reviewed By: kmancini
Differential Revision: D22550587
fbshipit-source-id: 029c797def477ae4533f66cfba146a3972cfb65d
Summary: This diff ensures that the value of ImportPriority is always positive and changes offset from 0 to half of the maximum offset to allow lowering offset in the future.
Reviewed By: kmancini
Differential Revision: D22550462
fbshipit-source-id: 69f45369359c7b2c39a0c6831e9b33982e26a16a
Summary:
I thought I had removed that code a while back, it turns out I didn't, so let's
do it.
Reviewed By: singhsrb
Differential Revision: D22583556
fbshipit-source-id: b92644195994e0a83bdbcd8019253ea217474486
Summary:
within() schedules work on the singleton timekeeper thread which
sometimes scheduled a timer callback on the ServerExecutor while
TestMount was shutting down, causing an assertion failure. Since
TestMount's executor is deterministic, just make the entire test
deterministic.
Reviewed By: genevievehelsel
Differential Revision: D22709150
fbshipit-source-id: 0a1621df0a9934bb7944fc259f7f83d110b35e93
Summary:
I'm not sure how these landed, but they need fixes to work correctly in
python.
Reviewed By: sfilipco
Differential Revision: D22699596
fbshipit-source-id: fb6d237f4de92c5efa3b422ddb86117fc256460f
Summary: A few projects out of sync between TARGETS and Cargo.toml.
Reviewed By: dtolnay
Differential Revision: D22704460
fbshipit-source-id: 3d809292d50cc42cfbc4973f7b26af38d931121f
Summary: It's nice to have this flag available
Reviewed By: krallin
Differential Revision: D22693732
fbshipit-source-id: 9d0d8f44cb0f5f7263a33e86e9c5b8a9927c0c85
Summary: If the user specifies a mount, only clear for that mount, otherwise clear for all mounts.
Reviewed By: fanzeyi
Differential Revision: D22674210
fbshipit-source-id: 832c4fd37a63fdb44cb12844378b61d113731016
Summary: add a thrift call to clear `pidFetchCount_` in `ObjectStore` and call it in `eden debug gc_process_fetch`. Users might want this command to start a new recording of process fetch counts.
Reviewed By: kmancini
Differential Revision: D22583430
fbshipit-source-id: eba7d63b08da5134fd09b7512895aba06f6a7ca5
Summary: now the terminator argument is unused - we can get rid of it.
Differential Revision: D22502594
fbshipit-source-id: e8ecec01002421baee38be0c7e048d08068f2d74
Summary:
`until_timestamp` will benefit from checking each node - this will allow for
less filtering on the caller side.
Differential Revision: D22502595
fbshipit-source-id: 23c574b0a4eeb700cf7ea8c1ea29e3a6520097a9
Summary:
This new trait is going to replace the `Terminator` argument to fastlog
traversal function. Insted of deciding if we should fetch or/not given fastlog
batch this trait allows us to make decisions based on each visited changeset.
Differential Revision: D22502590
fbshipit-source-id: 19f9218958604b2bcb68203c9646b3c9b433541d
Summary:
The function for finding the commit where the file was deleted
in the fastlog module doesn't depend on fastlog at all.
It also seems generic enough to be a good public API for deleted files
manifests module.
Differential Revision: D22502596
fbshipit-source-id: 2e226bf14339da886668ee8e3402a08e8120266e
Summary:
Let's centralize the logic that adds new nodes to BFS queue during fastlog
traversal, this will allow me to hook into it in the next diffs.
Differential Revision: D22502593
fbshipit-source-id: 63f4e7adb3a7e11b4a2b2dcc65cab3bb4bf6f015
Summary:
This new skiplist feature allows to find merges between any two related
commits.
Reviewed By: StanislavGlebik
Differential Revision: D22457894
fbshipit-source-id: 203d43588040759b89a895395058a21c9b5ca43d
Summary: I'm planning to use it in my next diff to power `find_merges` functionality.
Reviewed By: StanislavGlebik
Differential Revision: D22457898
fbshipit-source-id: 76c3f107fd8b5bbef96e978037be31efca0f9841
Summary:
The `process_frontier` function is THE function that powers skiplist traversal
and it's quite complex. To make the core more readable I'm moving part of the
code into separate functions.
Differential Revision: D22457896
fbshipit-source-id: e3521855ae7ab889c21d7aff0204e27dc23cf906
