Summary: It is helpful in CI to be able to remove all eden managed volumes at the end of a job, so that is what this command does.
Reviewed By: simpkins
Differential Revision: D19794482
fbshipit-source-id: d832d093d0a6369a4b533d66afbdce626fa78e2c
Summary:
In the initial iteration of eden_apfs_mount_helper I parsed the human
readable version of the `diskutil apfs list` output to get all of the useful
information from it. Later, I switched to using `diskutil apfs list -plist`
because it simplified the parsing dramatically.
However, it was overlooked that the `mount_point` field was not present in
the plist output. That was usually fine, except in the one case that I
didn't test after changing the parser: after reboot macOS picks a volume
name to remount these volumes. The lack of the mount_point field meant
that we would simply skip the logic that deals with unmounting and remounting
the volume in the correct place.
In order to address this gap we now need to parse the mount table to determine
where the device is mounted, so that is what this diff does.
Reviewed By: simpkins
Differential Revision: D19794478
fbshipit-source-id: ace2df145a46aad7df78c3f4b15fb2198aef3e6f
Summary:
This is done on a sort of best effort basis; the problem
we're aiming to solve (or at least avoid in the majority of cases)
is in these scenarios:
* `buck clean` gives up cleaning as soon as it finds a directory that
it doesn't have permissions to remove.
* `yarn` tries to look for node modules inside the `.Trashes` dir,
which it doesn't have permissions to access
macOS doesn't give us a solid way to indicate that these things should
be disabled at the time that we mount the volume; the various docs
and suggestions online all involve creating marker files to block the
system processes from performing their usual actions, and we could
set those up here, but if we create them with root permissions we'll
trigger the bad behavior in buck and yarn. If we create them with
regular user permissions then eg: running buck clean would remove them
and allow the system to recreate them.
The system will recreate the `.Trashes` directory when the user sends
something to the trash via Finder. Similarly, macOS will recreate
the fsevents directories when an application establishes a watch
on the volume using fsevents.
So we can't permanently prevent this problem, but by deleting these
things at mount time we should at least avoid it bubbling up for
the majority of users, and if those things come back and get in the
way, running `eden redirect unmount ; eden redirect fixup` should
remount and re-remove the problematic things.
Reviewed By: fanzeyi
Differential Revision: D19841514
fbshipit-source-id: f530ab3d68edfa643096bd27efae71c80b505184
Summary: Later in this stack, we're going to have to introduce a few more context types, including Mononoke's per-session `CoreContext` as well as an LFS-server inspired per-request `RequestContext`. To make the scope of the `EdenApiContext` more clear (namely, that unlike the other contexts, this one persists for the entirety of the server's execution), let's rename this to `EdenApiServerContext`. This mirrors the naming convention for these contexts used in the LFS server.
Reviewed By: xavierd
Differential Revision: D19863296
fbshipit-source-id: 8ad785070328523d0beaf824c86c7350ff6a2697
Summary: Use `ServerIdentityMiddleware` from `gotham_ext` in the EdenAPI server to provide server information in the HTTP response headers returned by the server. This is useful for debugging.
Reviewed By: xavierd
Differential Revision: D19863297
fbshipit-source-id: 6ed8bac35e05af580e062423e6f6389a18d17c2a
Summary:
This makes it possible to use `Bytes` for mmap buffers.
The changes are because `minibytes::Bytes` does not implement `From<&[u8]>`
with the intention to make slice copy explicit.
Reviewed By: xavierd
Differential Revision: D19818719
fbshipit-source-id: c34ee451bfd2dc7bcbbcebd52a76444b6c236849
Summary:
EdenFS will now be able to fetch blobs directly from memcache. This won't have
any big benefits as no blobs are in memcache right now, but over time, this
will significantly reduce the cost of fetching blobs.
Reviewed By: fanzeyi
Differential Revision: D19861643
fbshipit-source-id: c2e9d317bd30d4656bf0b3f8897794161697761a
Summary:
This improves scs client ux as you can see in the test.
Cross scheme collisions have been already supported in the code, and there will
be a proper message.
For the ambiguity within a single scheme, I added a message.
Reviewed By: mitrandir77
Differential Revision: D19834142
fbshipit-source-id: a2cff44f6ef031b1224ebf13d38d76c66c9ee4b9
Summary: The exact format of the GPL license header changed when the Mononoke directory moved from `/scm/mononoke` to `/eden/mononoke`. This file ended up with the old header somehow (was created prior to the rename but landed after), so let's fix it to make the linter happy.
Reviewed By: farnz
Differential Revision: D19848640
fbshipit-source-id: 39c5b49850e5a3cba1951bf4e5b813cd08940f01
Summary: Move the generally-useful ServerIdentityMiddleware into gotham_ext so we can use it in the EdenAPI server.
Reviewed By: xavierd
Differential Revision: D19845282
fbshipit-source-id: 3a01b15dc64cee99cefafcdac229c0b70a4db683
Summary:
These tracing points will help us understand the memcache hit rate as well as
the fetching speed.
Reviewed By: quark-zju
Differential Revision: D19836499
fbshipit-source-id: 1936c44efc3e7715069e6a959f5331139d591d5c
Summary:
Everytime a cache miss is seen, the data fetched from the server will be sent
directly to memcache for future use. Unfortunately, doing so in a blocking
manner severely impact the overall fetching speed from the server. Since
memcache is purely an optimization, we can afford to send data to it
asynchronously.
Let's move as much as possible of the code to a background thread to reduce the
overhead of memcache.
Reviewed By: DurhamG
Differential Revision: D19836011
fbshipit-source-id: 68e506ef7464d6e99d98457d0d37178f514be1a9
Summary:
Instead of fetching data one-by-one, let's prefetch data concurrently by using
the new get_iter function.
Reviewed By: DurhamG
Differential Revision: D19836009
fbshipit-source-id: 4a50328c0cbbba677c2de3777ebe4c34cb10c1e2
Summary: remove the need to pass mapping to `::derive` method
Reviewed By: StanislavGlebik
Differential Revision: D19856560
fbshipit-source-id: 219af827ea7e077a4c3e678a85c51dc0e3822d79
Summary: Test that mercurial can accept a unicode commit message.
Reviewed By: xavierd
Differential Revision: D19838221
fbshipit-source-id: b6333e587004b358a3883db70e40d2f32af4da29
Summary:
Currently admission to fastreplay is controlled at a global level - a
percentage of all repos being replayed are admitted without the ability to turn
off an individual repo.
Update fastreplay to use a config option that controls which repos should be skipped.
Reviewed By: krallin
Differential Revision: D19835304
fbshipit-source-id: 63b7c66b35eaa2cb1370ad96b1c6e98a2c93b712
Summary: Rechunker uses tokio 0.1.0 and legacy futures 0.1.0 . This diff changes rechunker.rs to use new std::future and tokio 0.2.0
Reviewed By: farnz
Differential Revision: D19801966
fbshipit-source-id: 3ba7936e8981c87906a881216ffd66332cdcc761
Summary:
This will allow us to distinguish `getbundle` for a normal `pull` from the one
for infinitepush pull.
Reviewed By: StanislavGlebik
Differential Revision: D19833206
fbshipit-source-id: 86534320fbb4d60bac04d458a0953701201cba87
Summary:
Even when memcache would be able to prefetch everything, this would always call
into the underlying remote store with an empt key set. For things like `hg
prefetch` and a large number of keys, the effect of doing that is minimum, but
for EdenFS or `hg log -p`, the roundtrip to the server for every file/revision
would add a significant amount of overhead. Let's simply stop iterating when we
no longer need to fetch anything.
Reviewed By: DurhamG
Differential Revision: D19835797
fbshipit-source-id: 54ad704428c3b20d973cfa87f7171899ec44b3f9
Summary:
Don't rebuild the memcache store everytime. This avoid spawning a thread and
reconnecting to memcache everytime.
Reviewed By: DurhamG
Differential Revision: D19797911
fbshipit-source-id: 7ecfa111a98ab67c4791b91410a22e458fe169f6
Summary:
Added "rust" to the languages in the if-service and its dependent targets, added a unique `rust_crate_name` to each one.
Adjusted import names in the dependent code.
Reviewed By: dtolnay
Differential Revision: D19818989
fbshipit-source-id: d1cf5c5f7b82dd64d7c045e50b304b7a58cf2ffc
Summary:
We don't use the feature in production. The main motivation is to remove the
flaky test. People wanting this feature can use normal hooks instead.
Reviewed By: xavierd
Differential Revision: D19830033
fbshipit-source-id: 0b1df7bb6c5de116ee27df4c38071912568d893e
Summary:
We never actually enabled it in production, and our new storage (ex. metalog,
indexedlog) are SIGKILL-safe. Only the changelog (revlog) might be subject to
double Ctrl+C issues. And we have debugfixcorrupt which can fix changelog
corruptions. So nointerrupt becomes less important.
The direct motivation is the test has been too flaky recently (and it's hard to
reproduce locally).
Differential Revision: D19830032
fbshipit-source-id: 87722d730fab49efbfd231772b1a5dddf8ec0d7b
Summary: Not very useful on today's hardware.
Reviewed By: quark-zju
Differential Revision: D19783741
fbshipit-source-id: d8ff29486cdc5f8e31d929d9fbba827e37012381
Summary:
We don't have confidence that python 2 is only using bytes (vs unicode
or other byte-like objects). These asserts break users, so let's disable them
except for in tests. We should uncover issues here as we port to python 3.
Reviewed By: xavierd
Differential Revision: D19819699
fbshipit-source-id: 9e3a9c34e7661fac4db9ee2a79f65be3d5e48cb3
Summary:
As part of the py3 migration we've started enforcing that encodeutf8 is
only called on bytes. graphql is returning unicode, so we need to change it to
return bytes on python 2.
Reviewed By: xavierd
Differential Revision: D19819701
fbshipit-source-id: 433ced2da87d522e4234ff453d11cacfaeebdd8d
Summary:
ui.log apparently receives non-utf8 data occasionally, which makes the
encoding fail. Let's change this to not encode on python2 (by using pycompat).
As we migrate to python 3 all the strings should become unicode, and this will once
again enforce utf8 encoding
Reviewed By: xavierd
Differential Revision: D19819700
fbshipit-source-id: 66da26d00d1b1c4286e325f0574ad7ff478e5d1f
Summary: logs information about daemon shutdowns, including duration, if it was a graceful restart or not, and if it was successful or not.
Reviewed By: fanzeyi
Differential Revision: D19817811
fbshipit-source-id: a851de8872f98952d046fb148a27fbf9f05b2212
Summary:
This commit manually synchronizes the internal move of
fbcode/scm/mononoke under fbcode/eden/mononoke which couldn't be
performed by ShipIt automatically.
Reviewed By: StanislavGlebik
Differential Revision: D19722832
fbshipit-source-id: 52fbc8bc42a8940b39872dfb8b00ce9c0f6b0800
Summary:
Attach the content to the exception object directly so if the exception is
handled, no verbose message will be printed.
Reviewed By: DurhamG
Differential Revision: D19817404
fbshipit-source-id: 9b79952559d07eb62bd7eb2398c6dd022ffe7b28