Summary:
When changing a file to a folder in a commit this function would try to query
the storage using the path and the id of the file. Storage would not have
such an entry and would cause the operation to fail.
Fixing the issue by filtering out ids of files in the section for known
manifests. Filtering already occurs in the section for transmitted manifest.
Reviewed By: quark-zju
Differential Revision: D18797368
fbshipit-source-id: 2ed727b5eaaab2f17cd988d6c8a3a1055ac8bd2a
Summary:
Each entry in InodeMap::unloadedInodes_ stored the InodeNumber
twice. Remove one of them.
Reviewed By: wez
Differential Revision: D18651007
fbshipit-source-id: be85c34cb2b38fc0b2875d0874cecd1ef274aca4
Summary: Small readability improvements I noticed while reading the diff algorithm.
Reviewed By: genevievehelsel
Differential Revision: D18650987
fbshipit-source-id: 19346362711455ae87d6770812e647544a4576bf
Summary:
On the LFS Server, we've noted that LFS batch requests from Mercurial have
about ~40ms latency, but that the same request sent through curl doesn't.
Those requests, when sent through curl, complete in < 1ms, so thats a lot of overhead.
After adding more logging in the LFS server (D18636703), it turned out that that is
because the body of the request shows up 40ms after the request itself.
After capturing the traffic on the backend (D18689109), it turned out that
Mercurial is sending the headers and the response body in 2 separate TCP
packets.
The reason for that is delayed TCP acks. What happens is that when we call
`h.endheaders()`, that calls `send(2)`, which in turns send the packet with the
headers. Later, when we call `h.send(data)`, we make another `send(2)` syscall,
but that doesn't go over the wire immediately: because of Nagle's algorithm, we
wait for an ACK for the first packet we send before sending a new one, and
because of delayed ACKs, the ACK takes 40ms for this ACK to show up (which is
over 40 times the roundtrip latency!).
This diff passes the data to `h.endheaders()`, which in Python's httplib will
concatenate the headers and the data into a single `send(2)` syscall if the
data isn't a file-like object (which it isn't for batch requests), and
otherwise fall back to doing exactly what we're doing right now.
This will result in everything in a single packet if we're fetching a single
file (and AFAICT, without further delay if we're fetching enough files that the
request doesn't fit in a single packet).
An alternative approach would be to set `TCP_NODELAY` on our socket here, but
for now this seems less intrusive and just as effective, so I've opted for that
approach.
Reviewed By: farnz
Differential Revision: D18811419
fbshipit-source-id: 4cf5719e5eed90e5dd994e6c8861aceb69373d89
Summary:
The Mercurial codebase contains over 500 errors, let's ignore them for now, we
can go back to them later to fix them.
Besides the manual change to .pyre_configuration.local, the changes were
generated with:
pyre --output=json check | pyre-upgrade fixme
Reviewed By: singhsrb
Differential Revision: D18803908
fbshipit-source-id: 724db7bd864c0de47a97ef2092bdee9f2cda531f
Summary:
The line causes pyre to enter an infinite loop, for now, let's simply ignore it
when type checking is enabled.
Reviewed By: singhsrb
Differential Revision: D18803909
fbshipit-source-id: d89b4cd0311a4a5416dd31197a8c69f4a6b65944
Summary:
Now that we've transitioned to the newer redirections
configuration we can remove the older bind mounts configuration
parsing, and that is what this diff does.
This is helpful because in situations where the user has run
`hg co null` as part of some troubleshooting advice, the legacy
bind mounts would remain mounted and obstruct some of the
steps that would have fixed up the users repo.
Reviewed By: pkaush
Differential Revision: D18337246
fbshipit-source-id: 23f27787d609e1c38a9c98b8b6596bb40743b9ca
Summary:
When user types Dxxx as a revset locally they usually mean the latest version
of the commit - not neccesarily the one in phabricator. This usecase was
usually handled by doing local lookup which can be very slow in case of slow
commits: see for example those user complaints:
https://fb.workplace.com/groups/scm/permalink/2487795837936688
Reviewed By: farnz
Differential Revision: D18809252
fbshipit-source-id: b3442d6fa2ef9c9c0dff4909c874689810fbfa88
Summary:
The intent is to allow macOS to make a best effort attempt
to detach any disk images associated with redirections.
D18795800 taught redirect to detach rather than a simple unmount,
and we need to give it an opportunity to detach when we unmount
gracefully so that all resources are released on shutdown.
The implementation of this diff ties into the same mechanism that
we use to shutdown the buck daemon when we unmount or restart which
has the nice side effect of showing the output from the underlying
`eden redirect unmount` command.
In an earlier iteration of this diff I tried making the server code
run `eden redirect unmount` but it happened too late in the process
to be effective: the thrift server had already been torn down in the
shutdown case and we need privs to perform the unmount.
Reviewed By: pkaush
Differential Revision: D18804080
fbshipit-source-id: 0b409130752121c56a46c9b2e46b50e5abee8200
Summary:
The purpose of this command is to unmount/unlink any configured
redirections without removing their configuration.
The intent is to call this for a mount when we are unmounting; I'll do
that bit in a follow on diff.
Reviewed By: pkaush
Differential Revision: D18801872
fbshipit-source-id: 096d9595091da72aa85f4259cbab022a1fe0c01f
Summary:
unmount will unmount from the VFS but still leave the image
associated with the disk image machinery and visible to `hdiutil info`.
This leaves resources associated with the mount even after it has
logically gone away, and that is undesirable.
This diff switches to using `detach` instead which unmounts and removes
the image from the disk image state.
The effect of this is that running `hg co null ; eden redirect fixup`
will completely unmount the redirections.
However, it doesn't cause them to be unmounted when the eden daemon is
stopped: that is something to address in a separate diff.
Also worth noting: only HFS+ mounts get detached successfully using this method. APFS mounts will remain regardless, so its another reason for switching to HFS+ in D18795799.
Reviewed By: pkaush
Differential Revision: D18795800
fbshipit-source-id: dfc86d86016a0c78e67f6ae2651db681669a5b14
Summary: The projects have been moved to a "shed/" subdirectory, so the root Cargo.toml with workspace has to be adjusted to that move.
Reviewed By: farnz
Differential Revision: D18807189
fbshipit-source-id: 0fd66fa7edd38ab4fdf905872f38fac57ae0230e
Summary:
Update Eden's CMake build to use the new `install_fb_python_executable()`
function to install `edenfsctl`
Reviewed By: wez
Differential Revision: D18774538
fbshipit-source-id: 462c0127d79edcb6235a629cb97ea481493e1906
Summary:
Add a `install_fb_python_executable()` function to `FBPythonBinary.cmake` for
helping to install python executables generated with
`add_fb_python_executable()`. This primarily helps by automatically looking
up the correct output file to install from the generated targets.
Reviewed By: wez
Differential Revision: D18774539
fbshipit-source-id: 4b397580d72ac448f21d1db6d2cdd653cf3635df
Summary: Log files that required manual merge during a rebase to dev command timers so we have an idea of how often they happen and on which files
Reviewed By: simpkins
Differential Revision: D18648175
fbshipit-source-id: 83ffe6d9177ca00b8fd1095745c585186bc2c8e9
Summary:
It's impossible for a consumer of a released version of
rocksdb to do anything about this except not use it, and this particular
version of rocksdb ships with a number of shadow warnings.
Disable warning to error promotion.
See also: https://twitter.com/pcwalton/status/1201679307552083968
Reviewed By: chadaustin
Differential Revision: D18785637
fbshipit-source-id: 1db2b00b3c397d6c0b8f05b9d1c658877685c961
Summary:
We've seen occasional weirdness with "resource temporarily unavailable" when working with disk images.
The internet, in the form of this stackoverflow post: https://stackoverflow.com/q/48368389/149111
suggests that this is related to APFS.
This commit switches over to HFS+ in the hope that we won't see it any more.
Reviewed By: chadaustin
Differential Revision: D18795799
fbshipit-source-id: 68e08e852770e311bcf04a8d12cb20670babf889
Summary:
OpenBCM libraries are stored with git LFS. As a result, fetcher fetches LFS pointers and not the contents. Use git-lfs to pull the real contents before copying to install dir using NoopBuilder.
In future, if more builders require git-lfs, we would consider installing
git-lfs as part of the sandcastle infra as against repeating similar
logic for each builder that requires git-lfs.
Reviewed By: wez
Differential Revision: D18759806
fbshipit-source-id: f988a0460107bc0685e7aba107daba9ed88f71e7
Summary:
As titled. This is needed by OpenBCM, and in future, possibly by other
manifests as well.
Reviewed By: wez
Differential Revision: D18759807
fbshipit-source-id: d445dfa382cea4bf96443ab9889926a4abbf0757
Summary:
These are not currently used by mercurial and are superseded
by the newer more general `watchman_client` crate on crates.io:
https://docs.rs/watchman_client/0.2.0/watchman_client/
Some of the implementation of that crate was based on the work I'm
removing here.
I was going to update hg_watchman_client to be implemented in terms
of the new crate, but since it isn't used and it would only be a thin
wrapper, I figure that it is best to delete this code instead so that
we can integrate it in terms of the new API when we're ready.
Reviewed By: singhsrb
Differential Revision: D18777161
fbshipit-source-id: efacb749e6d35ff4cc1fdc7d99cdca9ed42c9cfe
Summary:
`edenfsctl` on macOS is not returning the status code correctly when build with `make-client.py`.
We should be using [`zipapp_main`](https://fburl.com/diffusion/3eot1k7a) that uses `sys.exit` to propagate the exit code from [`main`](https://fburl.com/diffusion/k7t8vqmx).
Reviewed By: simpkins
Differential Revision: D18772986
fbshipit-source-id: 1714be9665b0779d30e5c86fb1e498466fee56f9
Summary:
Since it's not calling conduit anymore we can remove all conduit related
functions.
Reviewed By: quark-zju
Differential Revision: D18733121
fbshipit-source-id: 20a6022d9ac7ae6e9afea2791a8daac57cefb22c
Summary:
conduit is going away, phabricator team wants us to stop using
it in hg.
Reviewed By: quark-zju
Differential Revision: D18732257
fbshipit-source-id: c4faf83e36af80fc616e91adede141ba12de5446
Summary: conduit is going away, phabricator team wants us to stop using it in hg
Reviewed By: quark-zju
Differential Revision: D18727399
fbshipit-source-id: c208f5fec5fefd83aa37629b56d2bbaa2532d30e
Summary:
It would be nice if the insert method would give back the path that
it failed to insert.
Reviewed By: dtolnay
Differential Revision: D18739978
fbshipit-source-id: 4c16d09750ade2f01397161129c31bcf0059a957
Summary:
The Mercurial code has the bad habbit of inserting files in what can
appear impossible locations. Sometimes files with directory names and
sometimes directories in file locations. This happens because the initial
code would do additions before deletions on Manifest implementations that
were rudimentary. As soon as we introduce validation various code paths
surface.
I tried to fix the codepaths that modify manifests but it's a losing game.
I fixed the issue that appeared in tests and a couple of issues people
reported but more situations crop up.
This is giving up on the python code.
Reviewed By: quark-zju
Differential Revision: D18737678
fbshipit-source-id: 0c97128ff67e5ba2334942b6afc404aa64a2e5f4
Summary:
In NFS v4.x, flock ends up inheriting some semantics of fcntl,
including that write locks can only be created against file handles
opened for O_WRONLY or O_RDWR.
This diff changes the mode of 'lockpath' to O_RDWR to solve this.
Reviewed By: simpkins
Differential Revision: D18714734
fbshipit-source-id: 84ba4a6a5de034a4942b1ca3aa8b0c92f882ce38
Summary:
This is an update to the fbcode_builder codebase to allow setting up the python virtualenv with python dependencies installed. I've included wheel and cython (with a pinned version to 0.28.6 which is the only version that works with thriftpy3 at the moment, due to https://github.com/cython/cython/issues/2985) as standard packages since these are required by some of our top-level dependencies (folly and thrift)
As far as I know, there are no other projects that use PYTHON_VENV at the moment except LogDevice so the impact should be minimal.
Reviewed By: lucaspmelo
Differential Revision: D18758383
fbshipit-source-id: 264941311c5e3a19dc4ef2bb78c9a1baa34dfd8c
