Summary:
It is considered an anti-pattern (https://rust-unofficial.github.io/patterns/anti_patterns/deny-warnings.html)
and is causing Github CI breakage unnecessarily (https://github.com/facebookexperimental/eden/runs/2402094456):
error: function is never used: `example_blob`
--> src/lfs.rs:1860:8
|
1860 | fn example_blob() -> (Sha256, usize, Bytes) {
| ^^^^^^^^^^^^
|
note: the lint level is defined here
--> src/lib.rs:125:9
|
125 | #![deny(warnings)]
| ^^^^^^^^
= note: `#[deny(dead_code)]` implied by `#[deny(warnings)]`
Reviewed By: andll
Differential Revision: D27911477
fbshipit-source-id: df8d642fe74fe311eb0f329d977b9b8270c27bf4
Summary: This is needed for rendering multiple progress bars in `cmd.exe`.
Reviewed By: andll
Differential Revision: D27887601
fbshipit-source-id: 9aa987cef327de91408f2e38b4d2e551fe10e39b
Summary: Many users have asked what scratch directory is when they look to free some disk space. Placing a README.txt under the scratch root could be helpful to explain what is in there.
Reviewed By: fanzeyi
Differential Revision: D27710277
fbshipit-source-id: e3ccd92fa1920ac4c791026b8d98aa05a1c8b268
Summary:
When running `sudo mkscratch path /home/markamendoza`, the folder `/data/users/markamendoza/scratch/homeZmarkamendoza` will be created, including all parents that did not already exist. All these new directories will be created with owner = root.
Anticipating this kind of call, `mkscratch` already goes in and manually modifies the owner of the leaf directory (e.g. `/data/users/markamendoza/scratch/homeZmarkamendoza`), **but will not do so for any of the parent folders it created**.
This is normally not a problem, as long as `/data/users/markamendoza/scratch/` already exists. If it doesn't then these directories are created with the wrong ownership, which causes problems if we (for example) call `mkscratch` again, without sudo.
This diff remedies this by calling the same `set_file_owner` function (that's already called on the leaf directory) on all parents that were created by `mkscratch`
Reviewed By: chadaustin
Differential Revision: D27378153
fbshipit-source-id: 26c40f4ca478cacf9117093c7b70cbedd679cea6
Summary:
Similar to D26518021 (3aba8d89b0), provide a way to obtain the "output" stream
that implements `std::io::Write`.
The "output" stream is then used to replace the `std::io::Write`
implementation on the `IO` itself.
This has 2 benefits:
- Removes the `&mut` methods on `IO`. So all `&mut IO` can be replaced by `&IO` without exceptions.
- Make it possible to drop `Clone` on `IO` (strong ref is not clonable, but weak ref can). So we can control the lifetime of the streams more predicatably.
Reviewed By: sfilipco
Differential Revision: D26538452
fbshipit-source-id: b00c65ae0ef5ef4a609104111706365028f47ef7
Summary:
Set the global IO reference to the IO created by hgmain. This allows
other crates like `pypager` to use the `IO` without cloning or passing
the reference across layers (Rust references do not actually work for
across the Python boundary).
Reviewed By: sfilipco
Differential Revision: D26518027
fbshipit-source-id: 6144b06035d0ef9384cc1a37245c306071e35a9d
Summary:
This diff ensures no one can exploit mkscratch's scratch directory. In reality probably no one would do this but just to add it as a precaution.
And also the renaming, I didn't land the rename changes in my previous diff and this diff fix that so we don't roll the "nested" mode name out.
Reviewed By: xavierd
Differential Revision: D26526702
fbshipit-source-id: 04ec883e4d1c0c3e9b54b6274e5113c8b4845b3f
Summary:
Make it possible for IO to be cloned. This allows, for example, tracing or
progress bar logic to keep a reference of IO.
Reviewed By: sfilipco
Differential Revision: D26518028
fbshipit-source-id: d5ca7f4347251c68f7491f784f647ee80c47d159
Summary:
Address yarn's `node_modules` problem on Winodws.
The design is debatable and probably not the best. Another approach could be simply special casing subdir path ends with `node_modules`. This won't require any user configuration but it would be a special case.
Reviewed By: chadaustin
Differential Revision: D26149393
fbshipit-source-id: b3e66cb2d4b70078bb25e7329988cd5ff8fdeadd
Summary:
If your disk1 is an external HFS-formatted disk, then
eden_apfs_mount_helper will fail to create apfs subvolumes on
it. Instead, use the disk backing the mount.
Reviewed By: fanzeyi
Differential Revision: D26096296
fbshipit-source-id: baa45181afb6610a095c864eb3183e5af76ec4e0
Summary:
In Mercurial we're finding use cases for conditioning on buck vs
non-buck builds. For instance, in some cases we specify #[link] directives for
non-buck builds, but don't need to for buck builds
(https://fb.workplace.com/groups/rust.language/permalink/4568487309866515/). In
other cases we want to depend on internal buck libraries when building with
buck, and not otherwise.
Let's add a fb_buck_build cfg so other projects may distinguish the same.
Reviewed By: jsgf
Differential Revision: D24493974
fbshipit-source-id: 1d558cbe0ae01ab4a7b4b5d6d4be75bb8ab0f41a
Summary:
On Windows, some Mercurial commands may create files in `.hg` directory even if
EdenFS is not running. As a result, the command itself will fail while the file
still left in the directory. Users typically will then start EdenFS and repeat
the same command.
However, due to the current design of EdenFS, it will not be able to recognize
these files created when it is stopped and return errors for any attempts to
write/remove the file, generate another error. Users then need to stop EdenFS
and manually remove the file in `.hg` directory to recover the repository. This
creates a very bad user experience.
The correct way to fix this is to teach EdenFS to track the modifications
happened when it is not running, however this will take a few weeks to get
there. For a temporary measurement, we teach Mercurial to abort when EdenFS
isn't running to avoid trickier recoveries.
Reviewed By: quark-zju
Differential Revision: D24001090
fbshipit-source-id: abc1ebcdae3819756fe64b5321f52a6e62c0c360
Summary:
`scratch` provided by `fb-scratch` was replaced by `mkscratch` provided by
the Mercurial package. See linked task for details.
Reviewed By: quark-zju
Differential Revision: D23773840
fbshipit-source-id: de0582069ce1a09c3cd9fc6b02d2d149f70d0d78
Summary:
For repositories that have the old-style LFS extension enabled, the pointers
are stored in packfiles/indexedlog alongside with a flag that signify to the
upper layers that the blob is externally stored. With the new way of doing LFS,
pointers are stored separately.
When both are enabled, we are observing some interesting behavior where
different get and get_meta calls may return different blobs/metadata for the
same filenode. This may happen if a filenode is stored in both a packfile as an
LFS pointers, and in the LFS store. Guaranteeing that the revisionstore code is
deterministic in this situation is unfortunately way too costly (a get_meta
call would for instance have to fully validate the sha256 of the blob, and this
wouldn't guarantee that it wouldn't become corrupted on disk before calling
get).
The solution take here is to simply ignore all the lfs pointers from
packfiles/indexedlog when remotefilelog.lfs is enabled. This way, there is no
risk of reading the metadata from the packfiles, and the blob from the
LFSStore. This brings however another complication for the user created blobs:
these are stored in packfiles and would thus become unreadable, the solution is
to simply perform a one-time full repack of the local store to make sure that
all the pointers are moved from the packfiles to to LFSStore.
In the code, the Python bindings are using ExtStoredPolicy::Ignore directly as
these are only used in the treemanifest code where no LFS pointers should be
present, the repack code uses ExtStoredPolicy::Use to be able to read the
pointers, it wouldn't be able to otherwise.
Reviewed By: DurhamG
Differential Revision: D22951598
fbshipit-source-id: 0e929708ba5a3bb2a02c0891fd62dae1ccf18204
Summary:
Generated by formatting with rustfmt 2.0.0-rc.2 and then a second time with fbsource's current rustfmt (1.4.14).
This results in formatting for which rustfmt 1.4 is idempotent but is closer to the style of rustfmt 2.0, reducing the amount of code that will need to change atomically in that upgrade.
---
*Why now?* **:** The 1.x branch is no longer being developed and fixes like https://github.com/rust-lang/rustfmt/issues/4159 (which we need in fbcode) only land to the 2.0 branch.
---
Reviewed By: zertosh
Differential Revision: D23568779
fbshipit-source-id: 477200f35b280a4f6471d8e574e37e5f57917baf
Summary: This diff fixes the issue that mkscratch does not build on Rust < 1.45.0
Reviewed By: xavierd
Differential Revision: D23325087
fbshipit-source-id: 7e737eb5bc0d75255a84c8ac353bf3a17c269300
Summary:
On Windows, Rust's `std::fs::canonicalize` [1] will generate extended-length path that will include a `\\?\` prefix [2]. This has subsequently cause `encode` to generate a path that contains a question mark, which is an invalid path on Windows.
This diff teaches `encode` to handle extended-length path on Windows. It essentially converts the path back so it no longer contains the prefix.
[1] http://doc.rust-lang.org/1.45.2/std/fs/fn.canonicalize.html
[2] https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file#maximum-path-length-limitation
Reviewed By: wez
Differential Revision: D22956659
fbshipit-source-id: 54691e204d7cb481bdb40f62c6520c0f70c3f648
Summary: The original function is changing behavior on non-UNIX platforms. Let's honor it in tests too.
Reviewed By: wez
Differential Revision: D22901850
fbshipit-source-id: 7c4e7dbfa0526d482f86794758e7ad39feacce10
Summary: APFS volume names have a length limit of 128, so we need to hash mount point to make sure we do not error out. Right now what happens if we get an error, but we do still successfully mount and the volume name is just truncated. Since we errored, we don't store this information anywhere. Instead now, we will check the volume name, and if it is too long, we hash the volume name in order to mitigate the "name too long" error.
Reviewed By: wez
Differential Revision: D22327372
fbshipit-source-id: d68f90472c1d952879078d0541b341d271c17092
Summary:
We'll be adding a bunch of Facebook specific configuration and values
here. Let's move it to someplace not open source.
Reviewed By: quark-zju
Differential Revision: D21241038
fbshipit-source-id: 2ac9cdce40b1b46f15f171d9d1f6b6692dcd29bf
Summary:
Implement the core API to start pager in native Rust. For now it is only
enabled for the entire command if `--pager=always` is set.
Reviewed By: DurhamG
Differential Revision: D20849644
fbshipit-source-id: 860b4e18d841da607864c3447d78dbac126f5f18
Summary: This will allow bash users to use things like `hg commit -l <(echo foo)`.
Reviewed By: krallin
Differential Revision: D20810653
fbshipit-source-id: 42e420e608d41704387a9011cf14a28f92192e5d
Summary: The compiler was complaining about these on Windows.
Reviewed By: quark-zju
Differential Revision: D20250719
fbshipit-source-id: 89405e155875a4a549b243e93ce63cf3f53b1fab
Summary: It is helpful in CI to be able to remove all eden managed volumes at the end of a job, so that is what this command does.
Reviewed By: simpkins
Differential Revision: D19794482
fbshipit-source-id: d832d093d0a6369a4b533d66afbdce626fa78e2c
Summary:
In the initial iteration of eden_apfs_mount_helper I parsed the human
readable version of the `diskutil apfs list` output to get all of the useful
information from it. Later, I switched to using `diskutil apfs list -plist`
because it simplified the parsing dramatically.
However, it was overlooked that the `mount_point` field was not present in
the plist output. That was usually fine, except in the one case that I
didn't test after changing the parser: after reboot macOS picks a volume
name to remount these volumes. The lack of the mount_point field meant
that we would simply skip the logic that deals with unmounting and remounting
the volume in the correct place.
In order to address this gap we now need to parse the mount table to determine
where the device is mounted, so that is what this diff does.
Reviewed By: simpkins
Differential Revision: D19794478
fbshipit-source-id: ace2df145a46aad7df78c3f4b15fb2198aef3e6f
Summary:
This is done on a sort of best effort basis; the problem
we're aiming to solve (or at least avoid in the majority of cases)
is in these scenarios:
* `buck clean` gives up cleaning as soon as it finds a directory that
it doesn't have permissions to remove.
* `yarn` tries to look for node modules inside the `.Trashes` dir,
which it doesn't have permissions to access
macOS doesn't give us a solid way to indicate that these things should
be disabled at the time that we mount the volume; the various docs
and suggestions online all involve creating marker files to block the
system processes from performing their usual actions, and we could
set those up here, but if we create them with root permissions we'll
trigger the bad behavior in buck and yarn. If we create them with
regular user permissions then eg: running buck clean would remove them
and allow the system to recreate them.
The system will recreate the `.Trashes` directory when the user sends
something to the trash via Finder. Similarly, macOS will recreate
the fsevents directories when an application establishes a watch
on the volume using fsevents.
So we can't permanently prevent this problem, but by deleting these
things at mount time we should at least avoid it bubbling up for
the majority of users, and if those things come back and get in the
way, running `eden redirect unmount ; eden redirect fixup` should
remount and re-remove the problematic things.
Reviewed By: fanzeyi
Differential Revision: D19841514
fbshipit-source-id: f530ab3d68edfa643096bd27efae71c80b505184
Summary:
I think I might want to call this from more than one place in
a later diff, so extract it into a helper function.
Reviewed By: chadaustin
Differential Revision: D19770166
fbshipit-source-id: e044003736c6ba21984a9129da1df50ce92b2f35
Summary:
We've had a small proportion of our users run into problems
with hdiutil and diskimages-helper, where those components get into
an unhappy state and effectively block operating on the redirections.
This diff introduces a new utility that is intended to replace the
use of disk image files with APFS volumes that we mount in the
appropriate places.
The intention is that we will teach `eden redirect` to use this tool
when available, rather than disk images.
macOS's security model is weird: it is perfectly valid for a non-privileged
user to create and delete APFS volumes in the APFS storage container,
but root privs are required to mount it into the VFS.
The intent is that we deploy this utility setuid root to minimize
the fan out--this way we won't need to teach the priv helper about
this kind of redirection.
There are a couple of subcommands demonstrated in the test plan.
Reviewed By: chadaustin
Differential Revision: D19323850
fbshipit-source-id: 35556f841e49e5c4b77679b756af9093222f4500
Summary:
The builtin interpreter currently only responds to "python" as the
input arg. Let's also support python3.
Reviewed By: farnz
Differential Revision: D19613372
fbshipit-source-id: 5d2eed85c2d9546808c1661f12681b03f1edc40f
Summary:
This makes `make hg3` work. It requires cleaning up the `build` directory when
switching between py2 and py3 build, which will be fixed later.
Reviewed By: DurhamG
Differential Revision: D19604824
fbshipit-source-id: 060ff313420126a5dba935c4451b45dc9af45f13
Summary:
The memcache code assumes it's initialized, let's actually do it.
The test change is required due to fbinit installing its own SIGTERM handler
that shows a backtrace. Installing our own SIGTERM handler makes it closer to
what Mercurial does already.
Reviewed By: quark-zju
Differential Revision: D19518698
fbshipit-source-id: bc8c2311e65c9c00678756abae3979ccda4453ea
Summary: Without this, `hg status --print0` can output nothing and break tests.
Reviewed By: xavierd
Differential Revision: D19387317
fbshipit-source-id: 4feb82889df9bd1705526027f491db90a2a5f946
Summary:
The hgpython interpreter was used to run Python scripts in tests that might
rely on Mercurial modules.
The previous hgpython implementation is Python PAR based, which does not have
access to native Rust modules like bindings. Change it to use the native
"hg debugpython" implementation that is more compatible.
Reviewed By: markbt
Differential Revision: D19190496
fbshipit-source-id: 9791dbf9ba0ed92de702291faa9145f01b05ec40
Summary:
The buck generated binary is 1.2GB in size. Drop dependency to reduce the size.
Dependent functions are re-invented.
New size is 31MB under `mode/opt` build, and 28MB under `mode/dev` build.
Reviewed By: wez
Differential Revision: D18900827
fbshipit-source-id: 536fa969d69f6261d812c2320795780d839b6ced
Summary:
This diff replaces eden's dependencies on failure::Error with anyhow::Error.
Failure's error type requires all errors to have an implementation of failure's own failure::Fail trait in order for cause chains and backtraces to work. The necessary methods for this functionality have made their way into the standard library error trait, so modern error libraries build directly on std::error::Error rather than something like failure::Fail. Once we are no longer tied to failure 0.1's Fail trait, different parts of the codebase will be free to use any std::error::Error-based libraries they like while still working nicely together.
Reviewed By: xavierd
Differential Revision: D18576093
fbshipit-source-id: e2d862b659450f2969520d9b74877913fabb2e5d
Summary:
This diff replaces code of the form:
```
use failure::Fail;
#[derive(Fail, Debug)]
pub enum ErrorKind {
#[fail(display = "something failed {} times", _0)]
Failed(usize),
}
```
with:
```
use thiserror::Error;
#[derive(Error, Debug)]
pub enum ErrorKind {
#[error("something failed {0} times")]
Failed(usize),
}
```
The former emits an implementation of failure 0.1's `Fail` trait while the latter emits an impl of `std::error::Error`. Failure provides a blanket impl of `Fail` for any type that implements `Error`, so these `Error` impls are strictly more general. Each of these error types will continue to have exactly the same `Fail` impl that it did before this change, but now also has the appropriate `std::error::Error` impl which sets us up for dropping our various dependencies on `Fail` throughout the codebase.
Reviewed By: Imxset21
Differential Revision: D18523700
fbshipit-source-id: 0e43b10d5dfa79820663212391ecbf4aeaac2d41
Summary:
This diff is preparation for migrating off of failure::Fail / failure::Error for errors in favor of errors that implement std::error::Error. The Fallible terminology is unique to failure and in non-failure code we should be using Result<T>. To minimize the size of the eventual diff that removes failure, this codemod replaces all use of Fallible with Result by:
- In modules that do not use Result<T, E>, we import `failure::Fallible as Result`;
- In modules that use a mix of Result<T, E> and Fallible<T> (only 5) we define `type Result<T, E = failure::Error> = std::result::Result<T, E>` to allow both Result<T> and Result<T, E> to work simultaneously.
Reviewed By: Imxset21
Differential Revision: D18499758
fbshipit-source-id: 9f5a54c47f81fdeedbc6003cef42a1194eee55bf
Summary:
Merge the fb-mercurial code into the Eden repository, under the
`eden/scm` subdirectory.
Reviewed By: quark-zju
Differential Revision: D18445774
fbshipit-source-id: fc3307f9937e0c7e1c8f7d03c5102c4fe5dedb10
Summary:
In preparation for merging fb-mercurial sources to the Eden repository,
move everything from the top-level directory into an `eden/scm`
subdirectory.
