Summary:
Merge react and userprofile_shell into master, rewrite profile to use react and revert changes to p page
```
Reviewed By: markbt
Differential Revision: D21285297
fbshipit-source-id: f60006738af068982a25fa368fd260ba5b7f3781
Summary: The changes to server/context, gotham_ext and the code that depends on them are the only reminding places where aclchecker is used directly and it is not easy to split this diff to convert them separately.
Reviewed By: krallin
Differential Revision: D21067809
fbshipit-source-id: a041ab141caa6fe6871e1fda6013e33f1f09bc56
Summary:
Enable the unit tests under eden/fs/utils on Windows.
This does comment out a few tests related to `AbsolutePath` that are broken on
Windows. The AbsolutePath constructor does not enforce that the input path is
actually absolute. Additionally the `canonicalPath()` function ends up doing
weird things and returning paths that start with `/` followed by a drive
letter (e.g., `/C:/foo`). These issues should ideally be addressed in
subsequent diffs.
Reviewed By: xavierd
Differential Revision: D21239886
fbshipit-source-id: ef08d62353ba83b96d9fd79bd4636f4a0f961373
Summary:
When connecting to Mononoke, it's possible for the connection to hang
for a while before ultimately failing. This is a poor experience for users.
Add a timeout, such that hgcli will only wait for 15 seconds before reporting a
failure back to the user.
Reviewed By: krallin
Differential Revision: D21281472
fbshipit-source-id: 6329a108a1ae3ec337d74529a80e0fcd74e02231
Summary: Add a README to provide an overview of the walker for reviewers and maintainers
Reviewed By: ikostia
Differential Revision: D21250280
fbshipit-source-id: 9fd7cd3a076a276de904b8ae5372fc1c0c28458b
Summary:
The operation originally wanted to operate on the fuse `Attr`
structure which we don't have on Windows, so I repurposed the
`InodeBase::getattr` into `InodeBase::stat` and moved the conversion
of `struct stat` to `Dispatcher::Attr` to the `EdenDispatcher::getattr`
method (and a couple of other adhoc places that were doing a similar
conversion).
Reviewed By: chadaustin
Differential Revision: D20562459
fbshipit-source-id: 6b538110038352e9b5590fcb5ff5c33fe84ac1d8
Summary:
The only change I had to make was due to the fact that MSVC wasn't smart
enough to realize that the shift value couldn't be negative, so a manual
folly::to_unsigned was added to silence the warnings.
Reviewed By: simpkins
Differential Revision: D21268634
fbshipit-source-id: e65f15d58d5ea23bfa6796bab23cf1f5c2e7c12c
Summary:
On Windows, the expected refcount is one less than what it is on Linux, due
to the .eden directory not being present.
Reviewed By: simpkins
Differential Revision: D21268203
fbshipit-source-id: 91cfe742fa4d576917d552964d9541dc68ad2c75
Summary: Fix permission issues we are seeing with the latest Mercurial release.
Reviewed By: xavierd
Differential Revision: D21294499
fbshipit-source-id: bcfb13dd005258b2e3b74fa281dbd8df36133ef6
Summary: I thought it would be helpful to introduce and `eden uptime` command, especially with automated graceful restart on the way. This prints it in human readable format, later on if for some reason automation would like to use this, a flag could be added that allows for custom formatting. Also, this can be added to `eden rage` output later.
Reviewed By: chadaustin
Differential Revision: D21260800
fbshipit-source-id: 3f9a4f8d6264dfc38bd15c024a0209f7eeb912fa
Summary:
Previously, for logging the value of unexpected enum values, we wrote
`static_cast<int>`. Given enumerations can have any integral type
backing them, this was somewhat inaccurate. Instead, introduce an
explicit enumValue function which returns a value of the appropriate
underlying type.
Reviewed By: genevievehelsel
Differential Revision: D20975176
fbshipit-source-id: 0bb5b0d2f68f8fe9d68e4c6a847d59ae0997d0df
Summary:
D21074489 adds metrics for pending FUSE requests, this cleans up the display for
pending requests.
This removes the max duration of pending requests for FUSE requests since this
data is not available (it is not measured by the FUSE library).
Reviewed By: chadaustin
Differential Revision: D21074746
fbshipit-source-id: e5585ec091aa5fd5499deee2d8be89f47f769a6a
Summary:
This adds the number and duration of FUSE requests to eden top.
The purpose of this metric is to provide a highlevel indicator of
the health of eden and the fuse interface. Because FUSE is the
interface through which the user mostly interacts with the file
system it these metrics are reflective of edens overall
responsiveness.
The metrics are collected per mount, but we only display the
aggregate since this serves as an overall summary metric.
These metrics are displayed similarly to the import metrics,
for consistency. There is not a metric for the max duration
of pending FUSE requests D21074746 cleans this up so that
the NA is removed from the display.
Reviewed By: chadaustin
Differential Revision: D20928603
fbshipit-source-id: ce78f4274203c2302e7595ae2ef601e18893db38
Summary:
This sets up the counters that will allow us to expose the number
of pending FUSE requests in Eden top.
As D20846826 mentions adding metrics for FUSE request gives
visibility into fuse requests and overall health of eden.
This provides more insight beyond the metrics for live FUSE requests
since it shows the kernels view of FUSE requests. Looking at the difference
between the number of pending and live request, can identify issues
that arise at the interface between eden and FUSE and monitor how
quickly fuse workers are processing requests.
**note**: this is only for linux since macos has no equivalent to
`/sys/fs/fuse/connections`
Reviewed By: chadaustin
Differential Revision: D21074489
fbshipit-source-id: c0951f0dfd4fa764be28d8686d08cd0dd807db37
Summary:
It turns out that macOS, FreeBSD, and Linux all agree on the values
for dtype. If we assume dtype is just the high nibble of mode_t
shifted right by 12, then the Windows CRT agrees too. So hardcode the
values in eden.thrift and add appropriate static_asserts. This opens
the possibility of simply static_cast'ing the numeric values in
Watchman.
Reviewed By: simpkins
Differential Revision: D20975101
fbshipit-source-id: 354ffcbdf3f1d5f8b1715abf1026eaea429d16cf
Summary:
This makes it a bit easier to write integration tests
that run an isolated EdenFS instance complete with its own
unique scratch path environment.
Reviewed By: genevievehelsel
Differential Revision: D21258455
fbshipit-source-id: 3ea13b6337bb8bb58a237e67488b116c423bbe6c
Summary:
In the past, if an EdenMount got stuck while shutting down (inodes still referenced), the EdenMount would already be removed from the server's `mounPoint's` map while stuck. This resulted in `eden list` showing that the mount was not mounted, and then if we tried to remount the mount, we'd get a confusing error message about the not being able to grab the overlay lock. This also made it so if we tried to attach with gdb, or if we core dumped and tried to look at the data, we wouldn't be able to examine the object.
Instead, we should only remove the mount from the `mounPoints_` map after it has completely been shutdown. This will allow us to get better signal on the state of the mount (it will be in the `SHUTTING_DOWN` state in `eden list` instead of `(not mounted)`) and it will allow us to better debug these situations (where a mount is stuck shutting down) since we can get more information about the mount that is in this state.
Reviewed By: simpkins
Differential Revision: D21211544
fbshipit-source-id: 97ff11df67d0a80f47cf6b9f563876e17bd76622
Summary:
Update `integration/lib/linux.py` to use `ctypes` to call `statfs`, instead of
running `/bin/stat` and parsing the output. This allows this test code to
work even on systems that do not have `/bin/stat` (e.g., on Ubuntu this is
installed at `/usr/bin/stat`).
Also update the usage of ctypes in the `mmap.py` test file to allow this file
to be imported successfully on Mac OS X.
Reviewed By: xavierd
Differential Revision: D21273385
fbshipit-source-id: c2173758f8ea67cde2a68e57827acc5f73b63179
Summary:
Update the `test_rebase_commit_with_independent_folder()` test to explicitly
disable in-memory updates. The in-memory update code ends up changing the
number of update operations performed (it skips the initial and final update),
causing one of the assertions in this test to fail.
Reviewed By: xavierd
Differential Revision: D21271902
fbshipit-source-id: 3e40adce5fac1c851c8f277d2209d48a19deed55
Summary:
The new temporary file code added in D20885160 was missing a call to close the
underlying file in the `__exit__()` function.
Reviewed By: xavierd
Differential Revision: D21271901
fbshipit-source-id: 653258d471421edd08b473654ae3d0503f3c6829
Summary:
enums and i16 are not serialization-compatible in Thrift, but we want
an enumeration with defined values for dtypes. Rename the existing
Dtype type to OsDtype in preparation for introducing the Dtype enum.
Reviewed By: simpkins
Differential Revision: D20974559
fbshipit-source-id: 5d9087a79e7fd59b99b257a99351a8287ba00b5f
Summary: `termios` is not avialable on Windows. Do not import it.
Reviewed By: DurhamG
Differential Revision: D21258999
fbshipit-source-id: f4390b69fe9abceea8b1959e7506c1558778f980
Summary:
From looking at the code, it appears that the FUSE refcount is always a
uint32_t, except when serialized. Let's do the signedness conversion and
narrowing then.
Reviewed By: simpkins
Differential Revision: D21240161
fbshipit-source-id: 877c6cb6881cb36346c64cf92d99b1e588aed580
Summary:
Add a new function that initialises common parts of Mononoke:
- cachelib
- logging
- runtime
In the next diff, I will also update `init_mononoke` to initialise tunables
too.
I left some users of `init_runtime` unchanged (such as `mononoke_admin`) as
they conditionally enable cachelib.
Reviewed By: krallin
Differential Revision: D21177253
fbshipit-source-id: 40b534d1b244fd8dcd05e1cff1c9f3edfb32a4b9
Summary:
For a while now, `arc lint` wouldn't properly format the code on Windows, the reason is due to the fact that the default sparse profile doesn't include fbcode/.clang-format.
However, adding this file to the sparse profile wouldn't help as it is a symbolink link to fbsource/tools/clang-format/clang-format.ini, and as we all know, Mercurial will not create symbolic links on Windows, instead a plain file is created and its content is the link destination. The main issue with this is that clang-format simply doesn't understand what this means, and thus fails to format any code :(.
Now, it turns out with recent enough version of Windows (every Windows version we support at FB do support it) being an administrator is not a requirement to create symlinks (D21258136), once this lands. I believe that once this lands, and Mercurial is taught to properly create symlinks on Windows, this change will no longer be necessary and we can revert it. In the short term, let's make our life slightly easier by copying the fbsource clang-format configuration.
Reviewed By: simpkins
Differential Revision: D21258015
fbshipit-source-id: 21b4fda3a28bd397244b0fc9ca730ced775e96f0
Summary:
The disabled test uses chmod and symlinks that aren't present in the WIN32
build. I'm not exactly sure why the .eden couldn't be found though.
Reviewed By: simpkins
Differential Revision: D21264070
fbshipit-source-id: da7385d9de4002c99fcbe79f37bd09acd41fbec8
Summary: The new API is more generic, use it in favour of the fbcode only aclchecker
Reviewed By: StanislavGlebik
Differential Revision: D21067812
fbshipit-source-id: fdbd61697f65bd34c114e6d37108f395c77c0b17
Summary:
Add a getEntryInformation Thrift call that allows looking up existence
and dtype without fetching blobs. There is still plenty of room for
improving this - we don't need to allocate inodes for every path to
satisfy this query.
Reviewed By: genevievehelsel
Differential Revision: D20974514
fbshipit-source-id: e7fe5b6bd789aaadbb8295c3d69099cb2f17debd
Summary:
I wanted to figure out "who added this visible head", "what is the difference
between this metalog root and that root". Those are actually source control
operations (blame, diff). Add a git export feature so we can export metalog
to git to run those queries.
Choosing git here as we don't have native Rust utilities to create a more
efficient hg repo yet.
Ideally we can also make hg operate on a metalog directory as a "metalogrepo"
directly. However that seems to be quite difficult right now due to poor
abstractions.
Reviewed By: DurhamG
Differential Revision: D21213073
fbshipit-source-id: 4cc0331fbad6e1586907c0a66c18bcc25608ea49
Summary:
This would avoid issues where the Phabricator commit hash points
to a filtered commit locally. There are 2 user complaints already.
Reviewed By: DurhamG
Differential Revision: D21180833
fbshipit-source-id: 7374e236bcf5e3f3e62bae59fa53604869e22c6f
Summary:
This makes metalog easier to use in debugshell context. For example, to
investigate the "bookmarks" in the past, the code gets simplified from:
roots = b.metalog.metalog.listroots(repo.svfs.join('metalog'))
past_ml = b.metalog.metalog(repo.svfs.join('metalog'), root[10])
past_ml.get("bookmarks")
to:
roots = ml.roots()
past_ml = ml.checkout(roots[10])
past_ml.get("bookmarks")
Reviewed By: DurhamG
Differential Revision: D21162568
fbshipit-source-id: 7cc5581afe596a3d2696311a36ac11caa718428a
Summary: This uploads more details that might be useful for debugging.
Reviewed By: DurhamG
Differential Revision: D21179514
fbshipit-source-id: ba24395fc4fd153c3ceb4957f822aab70821e6ef
Summary: This allows the Python world to obtain the root ID for logging purpose.
Reviewed By: DurhamG
Differential Revision: D21179513
fbshipit-source-id: 3f289c06d3d470ff492de39fa985203b3facbf00
Summary:
This would hopefully give us more insights about when and what host uploads
10k+ remote bookmarks.
Reviewed By: DurhamG
Differential Revision: D21179515
fbshipit-source-id: 3e5c5559e2e739268add05e40f71bea08c29662f
Summary:
Design doc for the Deleted Files Manifest derived data structure - https://fb.quip.com/Wk4XAi7dyXRr.
**This diff adds derivation logic for merge commits.**
For linear case each deleted manifest node has an optional liknode, that points to the changeset id where the path was deleted. If the liknode is None it means that the path is or was a directory and its subentries were deleted, but the path itself still exists.
The idea for merge commits is that paths deleted in merges now will have liknodes pointing to the merge commit. So if the path was deleted in both branches of the merge commit, then deleted manifest of the merge commit will have a linknode for the path pointing to the merge commit itself. (For the examples see the test cases below.)
**To generate file history for the deleted file:**
If there is no unode for the given commit and path:
* search for the path in the Deleted files manifest
* if there is no node, there was no such path in the repo
* from the manifest's liknode get the changeset id where the file was deleted
* get the changeset's parents
* for each parent try to find unode, if there is unode, then traverse history
* otherwise check deleted manifest again (going back to the beginning)
This logic is going to be implemented in another diff.
Reviewed By: StanislavGlebik
Differential Revision: D21182002
fbshipit-source-id: 4dfa5ae0dab20a51d78e75c0fafb624f1a652fff
Summary: In this diff I moved some of the deleted files manifest derivation methods from the old futures to async/await. This allows me to introduce changes into the derivation logic using new standard futures.
Reviewed By: StanislavGlebik
Differential Revision: D21177622
fbshipit-source-id: fd5920cce11095885f167dad89ee974308501025
Summary:
We removed the feature in D20704618 and it does not cause complaints.
Let's remove the code supporting the chown feature.
Reviewed By: DurhamG
Differential Revision: D21170307
fbshipit-source-id: c845016219e8c681930bb1780b94e6d31ca99730
Summary:
While the change looks fairly mechanical and simple, the why is a bit tricky.
If we follow the calls of `ContentStore::get`, we can see that it first goes
through every on-disk stores, and then switches to the remote ones, thanks to
that, when we reach the remote stores there is no reason to believe that the
local store attached to them contains the data we're fetching. Thus the
code used to always prefetch the data, before reading from the store what was
just written.
While this is true for regular stores (packstore, indexedlog, etc), it starts
to break down for the LFS store. The reason being that the LFS store is
internally represented as 2 halves: a pointer store, and a blob store. It is
entirely possible that the LFS store contains a pointer, but not the actual
blob. In that case, the `get` executed on the LFS store will simply return
`Ok(None)` as the blob just isn't present, which will cause us to fallback to
the remote stores. Since we do have the pointer locally, we shouldn't try to
refetch it from the remote store, and thus why a `get_missing` needs to be run
before fetching from the remote store.
As I was writing this, I realized that all of this subtle behavior is basically
the same between all the stores, but unfortunately, doing a:
impl<T: RemoteDataStore + ?Sized> HgIdDataStore for T
Conflicts with the one for `Deref<Target=HgIdDataStore>`. Macros could be used
to avoid code duplication, but for now let's not stray into them.
Reviewed By: DurhamG
Differential Revision: D21132667
fbshipit-source-id: 67a2544c36c2979dbac70dac5c1d055845509746
Summary: implement the get() functions on the various LocalDataStore interface implementations
Reviewed By: quark-zju
Differential Revision: D21220723
fbshipit-source-id: d69e805c40fb47db6970934e53a7cc8ac057b62b
Summary:
This makes debugshell easier to use. Especially when the script wants to access
`__file__`.
Reviewed By: DurhamG
Differential Revision: D21169556
fbshipit-source-id: 88b3ebb1ca9a39fe26bc7cc5ea8e250c28fa0d6f
Summary:
When generating the deleted file manifest, there may be many duplicates of entries, particularly for deleted leaf files, which will always have the same manifest and thus the same manifest ID.
To prevent overloading of the blobstore with too many writes for the same key, deduplicate the writes before sending them.
Differential Revision: D21257640
fbshipit-source-id: 8f367496109b3e2b0a802fed96747cc290b295ec
Summary:
Similar to part of D20804854, normalize infinitepush/ to default/ in
remotenames code paths.
This might not be strictly necessary, but we never want to write out
infinitepush/ remote bookmarks and we have seen that OnDemand writes out
infinitepush/ remote bookmarks (which got synced to commit cloud and caused
user complaints) recently.
Reviewed By: DurhamG
Differential Revision: D21233658
fbshipit-source-id: cf8c12ac40348c7b475a7f3d5d5223775491650e
Summary:
The python script run using `debugpython` within the test was importing
Mercurial internal modules which causes failures on some platforms like
Windows. Instead, lets use the `debugshell` command which already imports the
common objects like `repo` correctly.
Reviewed By: xavierd
Differential Revision: D21251818
fbshipit-source-id: db1b9e92df99b736a28bc9e89fb08ae77d6e82fc
Summary:
This updates our bounds checking in hg_sync to be a bit more explicit. I
actually didn't realize at all that we were checking for > 0 somewhere for a
reason (StanislavGlebik noticed in D21228409).
Anyhow. This makes the checking a bit more explicit / obvious.
Reviewed By: ikostia
Differential Revision: D21229755
fbshipit-source-id: 88071b9bfbab428785fcb929ae2c04fee942067f
Summary:
It can be quite convenient when you have a bundle ID to be able to quickly
translate it to a Bonsai and get information about said Bonsai in order to e.g.
identify whether it was particularly large. This adds that functionality.
Reviewed By: StanislavGlebik
Differential Revision: D21228409
fbshipit-source-id: fc2ff938ff16e99c88b3e522b7ac39c4f39d60f2
