Summary:
This newly added config controls the default behavior of `eden clone` and
whether a new clone will use NFS or FUSE. This is intended to facilitate the
transition to NFS from FUSE on macOS.
Differential Revision: D30110056
fbshipit-source-id: ea6b493aa803297952b46434f6d11d8edf58e40b
Summary: 2 of the disabled are passing, and the last one just needed a simple tweak.
Reviewed By: fanzeyi
Differential Revision: D30029260
fbshipit-source-id: ad386632e8b6fbf9c76530c8ce2df7d38990bbbd
Summary:
The mode bits don't make a lot of sense on Windows, so let's use the plain
0o777 as that's what they will look like on Windows.
Reviewed By: chadaustin
Differential Revision: D30028892
fbshipit-source-id: ced737f1290438708536b732095cee18e1b1c073
Summary:
The create_editor_that_writes_commit_messages function uses bash which is not
available on Windows. Instead, we can simply pass --message to Mercurial for
the same effect.
Reviewed By: chadaustin
Differential Revision: D30028893
fbshipit-source-id: 0a588c37bd4da85176546568126893192b29a574
Summary:
If Mercurial asks EdenFS to update to a commit that it has just created, this
can cause a long delay while EdenFS tries to import the commit.
EdenFS needs to resolve the commit to a root manifest. It does this via the
import helper, but the import helper won't know about the commit until it is
restarted, which takes a long time.
To fix this, we add an optional "root manifest" parameter to the checkout or
reset parents thrift calls. This allows the Mercurial client to inform EdenFS
of the root manifest that it already knows about, allowing EdenFS to skip this
step.
Reviewed By: chadaustin
Differential Revision: D29845604
fbshipit-source-id: 61736d84971cd2dd9a8fdaa29a1578386246e4bf
Summary:
We need to set the infinitepush path too so that the Mercurial autopull code
won't try to pull from Mononoke
Reviewed By: genevievehelsel
Differential Revision: D29943227
fbshipit-source-id: a67dbfe97e7ab46dee885d9ec91a4d194dc2bd37
Summary: There are a lot of places in user visible text such as command line help where EdenFS is mentioned as eden/Eden/edenfs/EdenFS. This will make it consistent to 'EdenFS' in most cases. In the places where it is referring to the process/daemon, 'edenfs' will be used.
Reviewed By: chadaustin
Differential Revision: D29419151
fbshipit-source-id: 7b8296f0a0c84fdcb566ff811f7fcedbe7079189
Summary:
On case insensitive mounts, updating between commits that have a different
casing for a file/directory would lead to the update failing due to EdenFS
believing that an untracked file is present in the mount. That conflict is
however bogus and EdenFS simply gets confused in
TreeInode::processCheckoutEntry about the entry with the different casing.
To fix this, we should avoid comparing paths in a case sensitive manner and
instead compare then in a case insensitive fashion. This allows the rest of the
checkout code to update the directory/file in place.
On Windows, there is one more subtlety: we can't change the casing of a
file/directory that is already a placeholder and thus we need to force the
entire hierarchy to be removed, this will also make the checkout fail in case
of untracked files in the hierarchy, which is also the behavior on case
sensitive systems.
Reviewed By: genevievehelsel
Differential Revision: D29121741
fbshipit-source-id: 3d2cdacf296a3d061fc828cd6d04d249542cb63f
Summary:
Recent versions of Mercurial have a little race condition during copy
clones where the file may still be held open when it tries to be overwritten. We
don't really use this clone mechanism in production, so let's just have the test
not use it either.
Reviewed By: chadaustin
Differential Revision: D29076195
fbshipit-source-id: c19b5dc0b870653737a242200772c179549f9220
Summary:
Similarly to the work that kmancini did for the integration tests, this
enables the Mercurial integration tests to run against NFS. Only 3 tests are
failing due to 3 separate reasons: graceful restart isn't support, permissions
given to the clients are wrong, and a lifetime issue for the Nfsd3 object.
As a bonus, I've cleaned the code a bit since TreeOnly was the only supported
configuration (and will likely stay that way).
Reviewed By: chadaustin
Differential Revision: D28459245
fbshipit-source-id: 21f043bb054d684eacaee57179f0bdf4853b0a35
Summary:
An NFS client caches the attributes of files to avoid having to request these
very frequently. What this means is that a file changed by another client (or
by the server itself) may take some time to be reflected on the client, that
time depends on the attribute caching configuration of the mount point.
For EdenFS, files can changed in 2 ways:
- Either it is changed by the user via the mount point,
- Or the user runs an `hg update`
For the first one, the client will simply update its attributes appropriately,
but for the second one, the cached attributes will only be updated when the
user does opens the file, any calls to stat prior will return the old
attributes. Since EdenFS runs on the same host, we can force the attributes
caches to be discarded by simply issuing an open call on the file that changed.
Reviewed By: chadaustin
Differential Revision: D28456482
fbshipit-source-id: 91022d35a33e436c47d94403d0c139992f880cf9
Summary:
The telemetry wrapper didn't validate the SNAPSHOT header, which makes
migrating to a new format harder. Fortunately, it doesn't even need to
read the SNAPSHOT file. The dirstate file is maintained even in EdenFS
checkouts.
Reviewed By: quark-zju
Differential Revision: D28650333
fbshipit-source-id: 174cf7039adcbb28224ec528c2462e0a9232b6cd
Summary:
This applies the formatting changes from black v21.4b2 to all covered
projects in fbsource. Most changes are to single line docstrings, as black
will now remove leading and trailing whitespace to match PEP8. Any other
formatting changes are likely due to files that landed without formatting,
or files that previously triggered errors in black.
Any changes to code should be AST identical. Any test failures are likely
due to bad tests, or testing against the output of pyfmt.
Reviewed By: thatch
Differential Revision: D28204910
fbshipit-source-id: 804725bcd14f763e90c5ddff1d0418117c15809a
Summary:
This is unused, and the Thrift file mentions that it should be fine to remove
in July 2020. It's now March 2021, time to kill it.
Reviewed By: chadaustin
Differential Revision: D26852134
fbshipit-source-id: 2872185edd834f889b78802210071d16b881e14c
Summary:
As Rice is replacing the overlay, introduce some integration tests
around the subtler interactions of mode bits and status.
Reviewed By: genevievehelsel
Differential Revision: D26707622
fbshipit-source-id: 429ba404c1c0655f8fff393664bc5ba9e3b0301f
Summary:
I'm not entirely sure why these started failing, but enabling ui.allowmerge
made these run again.
Reviewed By: chadaustin
Differential Revision: D24697462
fbshipit-source-id: ec5ca987e7116edb12658eb7b4d03f1cf0f876d3
Summary:
On Windows, computing the sha1 of a materialized file requires opening up the
file in the working copy, as the file is cached there. Interestingly, this
potentially means that for computing the sha1 of a file, EdenFS may receive a
callback from ProjectedFS about that file or a parent directory. At this point,
EdenFS just refuses to serve this callback, as doing so may trigger an infinite
loop, or simply deadlocks. While this may sound weird, recursive callbacks are
not expected, as this signify that EdenFS view of the working copy doesn't
match what it actually is.
To close the loop, and from a code perspective, this means that computing the
sha1 of a file can fail and can throw an exception. Unfortunately, the code
didn't reflect this fact and exceptions were simply ignored, when that happens
during a checkout operation, this can leave the working copy in a weird state,
further agravating the mismatch between EdenFS view of the working copy, and
what it actually is.
Reviewed By: wez
Differential Revision: D24282048
fbshipit-source-id: 745af03189fe345150f0b1792ee1b37a1b8fb0d4
Summary:
While on Linux these can't fail (or, to be more precise: it doesnt' matter),
they can on Windows. One such exemple is when a user lock a file and triggers
an update that modifies this file. The invalidation will fail, and thus the
update should keep track of that file not being updated properly.
Previously, the invalidation would raise an exception, but that proved to be
the wrong approach as some state would need to be rolled back which the
exception didn't help in. For that, let's just return a Try and make sure that
we handle all the cases properly.
Reviewed By: chadaustin
Differential Revision: D24163672
fbshipit-source-id: ac881984138eefa65c053478a160e2a653fd3fdf
Summary:
I observed an integration test fail because Mercurial aborted with an
error about not handling EINTR, but it had no traceback. Always run
Mercurial with --traceback to try to debug these.
Reviewed By: xavierd
Differential Revision: D24204308
fbshipit-source-id: 44960bc645e5f97f61761e511b372328430fcda7
Summary:
Most of the fixes are pretty trivial as the code was using functions not
present on Windows, either work around them, or switch to ones that are
multi-platform.
Of note, it looks like `hg doctor` doesn't properly detect when Mercurial and
EdenFS are out of sync, disabling the tests until we figure out why.
Reviewed By: genevievehelsel, fanzeyi
Differential Revision: D23409708
fbshipit-source-id: 3314c197d43364dda13891a6874caab4c29e76ca
Summary:
On Windows, paths are separated by \, but the test was comparing them against
/. We can simply ask Mercurial to return / with the slashpath template filter.
Reviewed By: chadaustin
Differential Revision: D22871407
fbshipit-source-id: 421bd14f752f29265b12eb25609d4f65e593dda8
Summary:
Cache invalidation is hard, and on Windows we avoided doing a lot of them. It
turns out, this was the wrong decision as it's fairly easy to find cases where
the filesystem view is different from the manifest state.
Since the Linux code is most likely correct in where the invalidation is done,
let's also do the same on Windows, removing a whole lot of #ifdef. It is very
likely that as a result of this diff we end up invalidating more than needed,
thus slowing down EdenFS, but at this point I'd prefer to err on the side of
correctness, performance will come later.
While invalidating files should use PrjDeleteFile, for directories, we simply
need to mark them as placeholder, as directories created by a user won't have a
placeholder, thus ProjectedFS would bypass EdenFS when listing in.
Reviewed By: chadaustin
Differential Revision: D22833202
fbshipit-source-id: d807557f5e44279c49ab701b7a797253ef1f0717
Summary:
Bypass truncation-based transaction if narrow-heads is on.
The transaction abort still works logically because commit references stay
unchanged on abort.
Related EdenFS and Mononoke tests are updated. Mononoke tests probably
shouldn't rely on revlog / fncache implementation details in hg.
Reviewed By: DurhamG
Differential Revision: D22240186
fbshipit-source-id: f97efd60855467b52c9fb83e7c794ded269e9617
Summary:
An assertion error is raised if `eden doctor` is in the middle of a merge. This is because we enter a specific "if" condition in the case that mercurial has two parent commits, and EdenFS only ever tracks `p0`, so EdenFS simply sets `p1` to the null commit in `_select_new_parents()`. Specifically, this is in the case in which both `_old_dirstate_parents` and `_old_snapshot` are not None.
Because `_old_dirstate_parents` has `p1` set to nonnull, and Eden thinks it is null , the check `self._new_parents != self._old_dirstate_parents` would be `True` even though there was actually no error.
Reviewed By: chadaustin
Differential Revision: D22048525
fbshipit-source-id: 9a19cc092e2bd80db0e01fb38533a1007640bee6
Summary:
Addressing issues simpkins brought up on D21207287 when we upgraded and introduced some pyre bugs.
Temporarily upgrading just this project, once we resolve some sandcastle capacity issues we'll release this via another global upgrade in fbcode.
Reviewed By: simpkins
Differential Revision: D21316793
fbshipit-source-id: f0c79f53d97f7182e7d8fe6e081c58ef53ce0c9a
Summary:
Previously we only included `basic_test.py` and `hg/status_test.py` in the
integration tests during CMake-based builds. This updates the code to now
include all of the test files, with just a few exclusions based on platform
type and what dependencies were available at build time.
Reviewed By: wez
Differential Revision: D21239912
fbshipit-source-id: b8826d249a6323ac3bcc555c9ceba54a4cbcfde9
Summary:
In the past, if an EdenMount got stuck while shutting down (inodes still referenced), the EdenMount would already be removed from the server's `mounPoint's` map while stuck. This resulted in `eden list` showing that the mount was not mounted, and then if we tried to remount the mount, we'd get a confusing error message about the not being able to grab the overlay lock. This also made it so if we tried to attach with gdb, or if we core dumped and tried to look at the data, we wouldn't be able to examine the object.
Instead, we should only remove the mount from the `mounPoints_` map after it has completely been shutdown. This will allow us to get better signal on the state of the mount (it will be in the `SHUTTING_DOWN` state in `eden list` instead of `(not mounted)`) and it will allow us to better debug these situations (where a mount is stuck shutting down) since we can get more information about the mount that is in this state.
Reviewed By: simpkins
Differential Revision: D21211544
fbshipit-source-id: 97ff11df67d0a80f47cf6b9f563876e17bd76622
Summary:
Update the `test_rebase_commit_with_independent_folder()` test to explicitly
disable in-memory updates. The in-memory update code ends up changing the
number of update operations performed (it skips the initial and final update),
causing one of the assertions in this test to fail.
Reviewed By: xavierd
Differential Revision: D21271902
fbshipit-source-id: 3e40adce5fac1c851c8f277d2209d48a19deed55
Summary:
Improve some failure messages to include the full command output that we are
checking.
Reviewed By: genevievehelsel
Differential Revision: D21084135
fbshipit-source-id: 65be3adce7bf54bab6beccf5d0c463a749d585d2
Summary:
We no longer use repository configs, so remove the `repository` subcommand
that supported adding and listing these configurations.
The main information that used to be included in the repository configuration
was the bind mount settings. This has since been replaced with the
`.eden-redirections` file that is placed directly in each repository.
Reviewed By: wez
Differential Revision: D20876462
fbshipit-source-id: cc7d8e6f0a6a2e04fbf3159417af41a44908b3a8
Summary:
Add a command line argument to `edenfsctl doctor` to cause it to only report
problems with the current EdenFS instance, and to skip checks for system-wide
problems or other running EdenFS processes.
This does refactor a fair amount of the top-level doctor logic to encapsulate
most of the state into an `EdenDoctor` class.
This also updates the integration tests to use this flag when invoking
`edenfsctl doctor`. Previously the integration tests could spuriously fail
due to other pre-existing problems on the system, or due to other EdenFS
instances that are currently being started or shut down by other tests running
in parallel.
Reviewed By: wez
Differential Revision: D20357521
fbshipit-source-id: 36640cc21e7bd79fbd300c4d2c7dbba127ec9170
Summary:
D17135557 added a bunch of `pyre-fixme` comments to the EdenFS integration
tests for cases where Pyre cannot detect that some attributes are initialized
by the test case `setUp()` method.
It looks like Pyre's handling of `setUp()` is somewhat incorrect: it looks
like if a class has a `setUp()` method this currently suppresses all
uninitialized attribute errors (even if some attributes really are never
initialized). However, Pyre does not detect `setUp()` methods inherited from
parent classes, and always warns about uninitialized attributes in this case
even they are initialized.
Lets change these comments from `pyre-fixme` to `pyre-ignore` since this
appears to be an issue with Pyre rather than with this code. T62487924 is
open to track adding support for annotating custom constructor methods, which
might help here. I've also posted in Pyre Q&A about incorrect handling of
`setUp()` in derived classes.
Reviewed By: grievejia
Differential Revision: D19963118
fbshipit-source-id: 9fd13fc8665367e0780f871a5a0d9a8fe50cc687
Summary:
When checking if a commit is valid explicitly check against the backing
repository rather than the Eden checkout. This makes the commit work
correctly if the Eden checkout's `.hg` directory has been corrupted but the
backing repository is still fine.
Reviewed By: genevievehelsel
Differential Revision: D19629959
fbshipit-source-id: 57992260332cbc1d6868813263fb3768b50db07e
Summary: adds a cli debug command to inspect the working copy parent. by default just returns eden's snapshot contents, but adds optional --hg flag to print Mercurial's dirstate information
Reviewed By: chadaustin
Differential Revision: D19167518
fbshipit-source-id: b65e112df6abe4e0e7a8a528a90b2e3d17297e66
Summary: There is one instace of `getScmStatusBetweenRevisions` in use - it is used in the eden cli in a hacky way to check if a commit hash is valid. Since this is not used anywhere else in a meaningful way, this replaces that use case with a hg call and depreciates `getScmStatusBetweenRevisions`
Reviewed By: simpkins
Differential Revision: D18690026
fbshipit-source-id: 02bd2c20a0f631ec41116f9fd4e18d14369298ef
Summary: This updates the hg and telemetry wrapper callsites of getScmStatus to first try running getScmStatusV2() with fallback option. This does not retry `hg status` while a checkout is in progress.
Reviewed By: simpkins
Differential Revision: D18209899
fbshipit-source-id: e7a77b902f5a0ee624e4ea3185a1901bdac090e6
Summary: This reads `enforceParents` from a config instead of always assuming true for `getScmStatusV2()`. This will allow a easy kill switch in case throwing errors from this thrift call causes issues with something that calls hg status
Reviewed By: simpkins
Differential Revision: D18258164
fbshipit-source-id: 1ae421a941c01a678d25d5453c771262b03558d0
Summary: make the error message returned in the case of out of date parents during a new status call more user friendly and provide possible remediation instructions
Reviewed By: simpkins
Differential Revision: D18328835
fbshipit-source-id: b214f45bb055d008db8b233ddd2a1843332db838
