Summary: On Windows, we create a .eden on the root of the repo as compared to in each directory. Updating get_eden_mount_name() to look for .eden in the current parent path.
Reviewed By: strager
Differential Revision: D16742335
fbshipit-source-id: 2d27c54895917524d3d9c233ccc723b51f33f032
Summary: On Linux and Mac, we provision .eden inside every directory, which contains symlinks to other locations. On Windows this won't work as well. First, Projected FS will cache all the .eden and will bump up the number of files on the disk. Second, it would be hard to simulate symlinks from Projected FS.
Reviewed By: strager
Differential Revision: D16724354
fbshipit-source-id: a560698fae8c7f14e475f1c1fb9ecc53e60c189f
Summary: Until now we were running Edenfs on the console. Now we could use edenfsctl start.
Reviewed By: chadaustin
Differential Revision: D16565624
fbshipit-source-id: 9a12e52234d36d7b8ce4e3741ffb62655ae334b2
Summary:
Basic tool to collect read/written files on eden
This is v0. The next steps are:
* Add collection of sizes for all paths
* Add as an optional step in duplo/jobs
* Add some smarter analysis options as we discover need
Reviewed By: strager
Differential Revision: D16663820
fbshipit-source-id: 1c25289c9840fc23318e0653bc632b097256a5bd
Summary: Peer-to-peer DBus sockets do not have an org.freedesktop.DBus object on the bus, nor anything (or any reason) to accept the Hello message. Enable users of DBusAddress to explicitly use this behavior if desired.
Reviewed By: zeroxoneb, simpkins, aleivag
Differential Revision: D16780064
fbshipit-source-id: e10e51aef7754f5b022b8d7455b51baee3db8db8
Summary:
Implement OverlayChecker::computePath().
This also updates it to return a structure which can better represent
information if the inode is part of an orphaned subtree, which prevents us
from figuring out the full path to the root of the checkout.
Reviewed By: chadaustin
Differential Revision: D16577697
fbshipit-source-id: b543e2210fe8bc561d78194852962bf57ad9644a
Summary:
This ports much of the Python logic in `eden/cli/fsck.py` to C++, so that
the edenfs daemon can invoke it directly during mount initialization when
necessary.
This initial diff just contains the code to scan for errors, without any
repair logic.
This also includes some improvements to the logic for efficiency. For
instance, we compute less state during the initial scan phase. This makes the
scan phase faster, at the expense of making some of the repair steps slightly
more expensive if errors are found. This newer C++ scan logic is
significantly faster than the old Python version: scanning 260,000 inode files
took around 1m20 second for the Python code, and runs in under 3 seconds with
the new C++ code. (On an SSD with the files in question already warm in the
kernel cache.)
Reviewed By: strager
Differential Revision: D16577699
fbshipit-source-id: 2f8713a219ad882848cee0c57dca34f2c7808340
Summary: Redirecting the stdout and stderr to the eden log file. The log categories are not working as expected on Windows and will need to be fixed. At this point this doesn't log the "eden=DBG2" on Windows. One workaround for now is to remove "FOLLY_INIT_LOGGING_CONFIG()" to get the default logs.
Reviewed By: chadaustin
Differential Revision: D16653632
fbshipit-source-id: 028cdf1d698798ff3fb0284b3f0fa2df5570eaf0
Summary: Moving these functions to EdenInit so we could use them on Windows. This diff doesn't change the behavior.
Reviewed By: chadaustin
Differential Revision: D16653498
fbshipit-source-id: 5cc8c2f4b12f54035fb86ae20f3274245642d032
Summary:
Moves the CMD column to the right, to prevent long commands from being truncated.
https://pxl.cl/GG37
Reviewed By: chadaustin
Differential Revision: D16626771
fbshipit-source-id: 291e862d2fcff6c252ae6263536a799428577137
Summary:
Drop the `2` from the `fb303_thrift_cpp2` rule name.
The old `cpp` version of fbthrift is gone, `cpp2` is the only supported C++
version, so lets drop the `2` from this build rule name.
I plan to change some of the CMake thrift rule generation code soon, to
support multiple languages (namely Python). As part of this I plan to
automatically add a language-specific suffix(`_py` or `_cpp`) to the build
rules. It seems better to use `_cpp` rather than `_cpp2` for C++ libraries.
Changing the name for this fb303 rule will make it fit this planned naming
scheme.
Reviewed By: chadaustin
Differential Revision: D16653264
fbshipit-source-id: e4da70c79ff14e9981717ab909d982770d104512
Summary:
Now that we have unsafe access to the contents of a
folly::Synchronized, InodeMap::updateOverlayForUnload can just access
the storage because it knows nobody else has access.
Reviewed By: simpkins
Differential Revision: D16632778
fbshipit-source-id: 4b2c1d6679e2655671d43899407bac1a19f2cab9
Summary:
`StartupLogger::exitUnsuccessfully()` is declared as `noreturn`, and its last
statement is to call another `noreturn` method. However, this other method is
virtual, and the behavior of the `noreturn` attribute on virtual functions
doesn't seem consistently implemented across compilers. gcc doesn't seem to
treat the virtual method as `noreturn`, and therefore still warns that
`exitUnsuccessfully()` may return.
This suppresses that warning by adding a `folly::assume_unreachable()`
statement.
Reviewed By: fanzeyi
Differential Revision: D16653184
fbshipit-source-id: e0a2d8a4420d3582f849e93568a5081827a788f5
Summary:
Edencli will throw and crash the cli when the edenfs is not running.
Before:
```
PS C:\open\fbsource\fbcode\eden\facebook\win> C:\open\fbsource\fbcode\eden\win\build\package\edencli\edenfsctl.exe status
Traceback (most recent call last):
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\Lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\Lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\open\fbsource\fbcode\eden\win\build\package\edencli\edenfsctl.exe\__main__.py", line 3, in <module>
File "C:\open\fbsource\fbcode\eden\win\build\package\edencli\edenfsctl.exe\eden\cli\main.py", line 1546, in main
File "C:\open\fbsource\fbcode\eden\win\build\package\edencli\edenfsctl.exe\eden\cli\main.py", line 140, in run
File "C:\open\fbsource\fbcode\eden\win\build\package\edencli\edenfsctl.exe\eden\cli\config.py", line 604, in check_health
File "C:\open\fbsource\fbcode\eden\win\build\package\edencli\edenfsctl.exe\eden\cli\util.py", line 164, in check_health
File "C:\open\fbsource\fbcode\eden\win\build\package\edencli\edenfsctl.exe\eden\thrift\client.py", line 74, in __enter__
File "C:\open\fbsource\fbcode\eden\win\build\package\edencli\edenfsctl.exe\eden\thrift\client.py", line 86, in open
File "C:\open\fbsource\fbcode\eden\win\build\package\edencli\edenfsctl.exe\thrift\transport\THeaderTransport.py", line 247, in open
File "C:\open\fbsource\fbcode\eden\win\build\package\edencli\edenfsctl.exe\eden\thrift\windows_thrift.py", line 109, in open
File "C:\open\fbsource\fbcode\eden\win\build\package\edencli\edenfsctl.exe\eden\thrift\windows_thrift.py", line 66, in connect
File "C:\open\fbsource\fbcode\eden\win\build\package\edencli\edenfsctl.exe\eden\thrift\windows_thrift.py", line 45, in _checkReturnCode
eden.thrift.windows_thrift.WindowsSocketException: Windows Socket Error: 10061
```
After:
```
PS C:\open\fbsource\fbcode\eden\facebook\win> C:\open\fbsource\fbcode\eden\win\build\package\edencli\edenfsctl.exe status
ps: illegal option -- p
Usage ps [-aefl] [-u uid]
-f = show process uids, ppids
-l = show process uids, ppids, pgids, winpids
-u uid = list processes owned by uid
-a, -e = show processes of all users
-s = show process summary
-W = show windows as well as cygwin processes
edenfs not healthy: edenfs not running
```
It internally uses a bash ps command to look for edenfs when the thrift connection fails. Still looking into good ways to fix that.
Reviewed By: strager
Differential Revision: D16565180
fbshipit-source-id: 3c12c538451293e20fabc51c2fe4f1a9bd5b84c0
Summary:
The toHash field is unnecessary in HashUpdateDeltas since we only ever iterate in reverse. We instead keep track of the current hash in the journal itself instead of looking at the toHash of the latest HashUpdateDelta.
Delta Struct Sizes:
> File Change: 88
> Hash Update: 96
Reviewed By: strager
Differential Revision: D16522519
fbshipit-source-id: 43baccc8ef2579f72609cc84e81e218794b11725
Summary: Create the second delta type HashUpdateJournalDelta that keeps track of changes to the hash [and the unclean files associated with that hash change]. Journal methods were updated to account for the different delta types.
Reviewed By: strager
Differential Revision: D16520444
fbshipit-source-id: 2a5cea11c9e70e30f6db55d9c8e33f9322ae91fc
Summary: Add in two tests for deltas that change commit hashes since a test for this did not exist before. Previously a bug in setting the fromHash of a result only showed up as a failure in Watchman's integration tests with Eden.
Reviewed By: chadaustin
Differential Revision: D16528248
fbshipit-source-id: 56eede749ef2da4dc492a1f7376dc07ca8aa3050
Summary: Deltas are stores in a deque and therefore not malloced independently so therefore there is no reason to be getting the goodMallocSize of them. We also need to do accounting for the buckets in the deque that the deltas fall into.
Reviewed By: strager
Differential Revision: D16566675
fbshipit-source-id: 4506fbbcc2044b8fdfe6244313ef7480cfa5151e
Summary: The original `eden top` trimmed `chg[worker/12345]` to `12345]` since the code trims the basenames of commands like `/usr/bin/ls` to `ls`. Now `eden top` only trims commands with exact paths and leaves commands with relative paths as is.
Reviewed By: simpkins, strager
Differential Revision: D16572655
fbshipit-source-id: a563ed5d4eb311244211e7db903fb64a37dcc4a4
Summary: Record a rolling sum of the time taken by any FUSE call on a per-process basis
Reviewed By: strager
Differential Revision: D16553149
fbshipit-source-id: 54f1e453916727a40f245b294239dc1b232a8967
Summary:
Change the behavior of `newEdenError()` to simply join its message arguments
together using `folly::to<std::string>()` rather than using
`folly::sformat()`.
Only a couple locations in the code were actually passing in a format string
correctly. Several other locations in the code were incorrectly passing in
arguments that they expected to be joined. This resulted in the error
messages being truncated, as the initial string did not contain any format
placeholders, so the trailing arguments were simply ignored.
Reviewed By: strager
Differential Revision: D16592709
fbshipit-source-id: d56f5387e05da2a68d94e6badac28ffcf68d6eab
Summary:
Renamed columns to save space, but maintain readability. Numeric columns are right aligned, and other columns are left aligned.
https://pxl.cl/G09n
Reviewed By: chadaustin
Differential Revision: D16470237
fbshipit-source-id: 8ff745e5b6cc23c4add85f8ff6ff5a6cbd388dfa
Summary:
Use the left and right arrow keys to choose which column is sorted! Uses the terminal's default colors.
https://pxl.cl/FNs1https://pxl.cl/FNs2
Reviewed By: chadaustin
Differential Revision: D16434277
fbshipit-source-id: f8353c8b3f2fe0d9c7bef776e56d4a2f0402d271
Summary: Ability to sort by any field in the `Process` object. Coming next: Keyboard controls for selecting which column to sort by
Reviewed By: chadaustin
Differential Revision: D16431719
fbshipit-source-id: 1fadb622e56cd86da68d224b99ecf8cfc39240c1
Summary: Uses the existing RequestData class to make calls to static functions to set and get the `didImportFromBackingStore` flag.
Reviewed By: simpkins
Differential Revision: D16461868
fbshipit-source-id: e3ed39249f5dd1a842ad06a204b5933014b12f7f
Summary: hg debugedenimporthelper should either have the repo path as argument or the current working directory should be inside a repo. Setting the current working directory for the process.
Reviewed By: strager
Differential Revision: D16565042
fbshipit-source-id: d6e826bebf58fd58f5250e6c477849ca147628c8
Summary:
Update `EdenServer::mount()` to correctly handle errors that occur during the
mount `INITIALIZING` phase. Previously the code did not add error callbacks
to the `Future` result to handle errors during initialization. As a result we
would propagate the exception back to the thrift caller, but the `EdenMount`
object would remain in our mount point list, stuck forever in the
`INITIALIZING` state.
Reviewed By: strager
Differential Revision: D16590032
fbshipit-source-id: 9adbdf05441dad815096b195ece36f3d958c96a9
Summary:
Update `FsOverlay::validateHeader()` to throw `EdenError` exceptions rather
than `std::system_error`. These exceptions are generated when we find invalid
data, rather than from system calls that return errno values. Previously the
code was using `folly::throwSystemErrorExplicit()` and
`folly:throwSystemError()`, and passing in a made-up `EIO` value.
However, the latter two calls were incorrectly using
`folly::throwSystemError()` rather than `throwSystemErrorExplicit()`. As a
result the `EIO` parameter was being treated as part of the error message, and
the code was appending the error description for whatever value `errno`
happened to be set to at the moment.
This fixes the code to just throw `EdenError` exception types, since these
errors aren't caused by a real `errno` value.
Reviewed By: strager
Differential Revision: D16577698
fbshipit-source-id: 9487c34f04da99d397611f005f00f02114b12094
Summary:
Fix `FsOverlay::initOverlay()` to also create the `tmp` subdirectory, which is
required for the `FsOverlay` code to function properly. Previously this
directory was created by the `Overlay` class. However, `FsOverlay` should be
responsible for creating it so it can be used from other locations besides
just `Overlay`.
I also removed the code that checks if the `tmp` directory needs to be created
on restart. The code to create `tmp` has been present for over a year
(D8330070), and we no longer have anyone using EdenFS versions older than
this.
Reviewed By: strager
Differential Revision: D16577700
fbshipit-source-id: 3eaa28003159a03084c4f367c7b96d69fc1702d0
Summary:
There is no point to call `readBE<uint64_t>()` here. The only way these calls
can fail is if the buffer is too short for the data, but we already checked
the buffer length at the top of the function.
Reviewed By: strager
Differential Revision: D16577701
fbshipit-source-id: 8b57cec05453b9b7224470671e032e3aefe41033
Summary: Replace the uses of std::unique_ptr<JournalDelta> with just the JournalDelta itself to avoid an extra allocation.
Reviewed By: chadaustin, strager
Differential Revision: D16572089
fbshipit-source-id: be080b2fb9096f6c8783e2ecae21a99466336f6f
Summary:
Adds a debug command to eden such that users can flush the journal and cause the subscribers to get a truncated result, this will be useful for debugging [to debug we won't have to lower the memory limit and repeatedly touching files to fill up the journal].
Can be used with "eden debug flush_journal"
Reviewed By: chadaustin
Differential Revision: D16348811
fbshipit-source-id: fdbe6729d0393c424addcd42a091de440383035b
Summary: Refactoring the addDelta function to allow other Journal functions to add deltas while holding the deltaState lock. This refactoring will be used to create the flush function for the journal (which needs to hold the lock to empty the journal but wants to add a delta before releasing the lock).
Reviewed By: chadaustin
Differential Revision: D16341196
fbshipit-source-id: 7b7b1d933802b466efe624378206c72c71469129
Summary: Delete tests that rely on flatmanifest support in Mercurial.
Reviewed By: strager
Differential Revision: D15302489
fbshipit-source-id: 7e46f21bfe3ae910e205e73bca95d9a33f4a6816
Summary: D16461081 broke the windows build of Eden by not updating the Windows mount code, this diff makes the same change as that diff in the Windows portion of the code.
Reviewed By: chadaustin
Differential Revision: D16549392
fbshipit-source-id: eee1bede08c1b27329b026746dda6863519ed10b
Summary: Keep track of the longest query we had to resolve in accumulateRange. This stat will be helpful in determining how large the memory limit of the journal should be since we will know how far back people usually need to go.
Reviewed By: strager
Differential Revision: D16227920
fbshipit-source-id: a41c3b9f16b701cd8165e20409888983b8899dab
Summary: The journal will now keep of how many reads from accumulateRange have been truncated. This is a useful metric that will allow us to see how often we are forcing Watchman to use its fallback of creating a fresh instance and help us in calibrating if we find our memory limit is too small.
Reviewed By: strager
Differential Revision: D16017968
fbshipit-source-id: 95f4fbd1fd2d8523ff397202172408e1c89669be
Summary: D16096960 accidentally caused the truncation code to be run twice (once inline and once via a function code), this shouldn't cause any difference in outcome but is unnecessary.
Reviewed By: strager
Differential Revision: D16508798
fbshipit-source-id: 12781aee98e70e5105c5476d29cf5cdd1e31062d
Summary:
D15528156 inadvertently changed the return type of the `getPid()` call from
`i64` to `i32`. This is a backwards-incompatible change, and causes new
clients to reject the 64-bit response from older EdenFS daemons. This breaks
`edenfsctl status`, `edenfsctl clone` and other commands.
Reviewed By: pkaush
Differential Revision: D16522765
fbshipit-source-id: eecd344ee4b963d638576f146a87fc88a5003e55
Summary: The Windows and Mac cli build scripts were not building the new fb303 thrift file. Also updated few cli code to use the new fb303 module.
Reviewed By: chadaustin
Differential Revision: D16508590
fbshipit-source-id: 386fa1bf77124d258ffb00edf56c9c827f32e795
Summary: Shows in real time how many seconds have elapsed since the last FUSE call was made by a process with the given (name, mount) combination.
Reviewed By: chadaustin
Differential Revision: D16364105
fbshipit-source-id: 925316e0b7b3a41aa44257bfbaf0a031e4f7bdd1
Summary: Tests for `eden top` and small refactor to make testing easier
Reviewed By: chadaustin
Differential Revision: D16273817
fbshipit-source-id: ab4c9512b7cf3c11741e9dfa7bd875d8f2360365
Summary: Include as much of the command as fits. We still trim the base command (e.g. "/bin/ls" -> "ls") but now, we include as many of the arguments as can fit. Before, only the first command line argument was shown.
Reviewed By: chadaustin
Differential Revision: D16268460
fbshipit-source-id: 5dba0ff35bec8655a4981e13492ac4e35f135b82
Summary: Taking advantage of the Process class for better readability and more modular code
Reviewed By: chadaustin
Differential Revision: D16265834
fbshipit-source-id: 89ffd84ec6eef54be5b7a160c0b2ef736908ffd4
Summary:
Processes are stored by PID; proceses are still grouped by (cmd, mount) pairs before they are rendered
{F168171902}
Reviewed By: chadaustin
Differential Revision: D16265116
fbshipit-source-id: 4ec76b9e9d2e6001ccdaaa31a1b2f00158907440
Summary: Set up the infrastructure to add in Timeseries to the journal so that we can add in stats for the journal that relate to the whole process. For example, allow us to add in a truncatedRead TimeSeries easily as done in D16017968
Reviewed By: chadaustin
Differential Revision: D16461081
fbshipit-source-id: 964ff32e62aed0369da434793491b857c136b074
Summary: Not checking whether journal duration was None caused old Edens to have an error when running "eden stats", this should fix it such that it does not print out duration info if Eden does not have it.
Reviewed By: strager
Differential Revision: D16502597
fbshipit-source-id: 1e28732208a3ad660b021b9e19c2694302a569f4
