Commit Graph

32 Commits

Author SHA1 Message Date
Chad Austin
1134b3cbd0 fix eden debug overlay to support the new non-materialized entry format
Summary: Materialized entries are stored with an unset hash. Fix the CLI.

Reviewed By: simpkins

Differential Revision: D7465192

fbshipit-source-id: 41e50c7214feb8afb1f3bf47252e4538945a5737
2018-04-03 14:21:45 -07:00
Chad Austin
f4fa2129fe add eden debug log command
Summary:
My muscle memory doesn't quite know where the edenfs log file
is, so add a shortcut command to view it.

Reviewed By: simpkins

Differential Revision: D7418743

fbshipit-source-id: 290b51b63fc3573663ad53f44e50dfd073c211c8
2018-03-30 01:59:44 -07:00
Puneet Kaushik
e2e7edd785 Fixed a race condition between initialization of RequestContext and its lookup.
Summary:
Request data was coming back as null because of a race between ctx->getContextData() in  FuseChannel::getOutstandingRequests() and the it initialization.

Also, added a check to verify rdata is valid before dereferencing it.

Plus, fixed debug.py where output gave an error.

Reviewed By: chadaustin

Differential Revision: D7377993

fbshipit-source-id: 8343119983c74185fd5d8cc05c2f5af63dcff99e
2018-03-26 14:48:23 -07:00
Puneet Kaushik
a7f99f7f2c Added thrift request to report outstanding FUSE calls
Summary:
Added a thrift call to return the outstanding FUSE requests.
Cli will call the thrift and print the output.
Added a unit test to test getOutstandingRequests().

Reviewed By: simpkins

Differential Revision: D7314584

fbshipit-source-id: 420790405babdb734f598e19719b487096ec53ca
2018-03-20 10:25:49 -07:00
Chad Austin
00d6bcc115 allow eden mount and eden unmount to take multiple paths
Summary: A minor convenience for mass mounting and unmounting.

Reviewed By: simpkins

Differential Revision: D6803003

fbshipit-source-id: 16c0d6982ba0ce2dba9900ee15013fcec8bc5ad5
2018-01-29 11:36:19 -08:00
Chad Austin
adce4eba1e fix determining which inodes can be unloaded
Summary:
I'm not sure what was wrong with the old code, but I
simplified and clarified all of the time math and now `eden debug
unload` behaves as I'd expect it should.

Reviewed By: simpkins

Differential Revision: D6764962

fbshipit-source-id: 3ed359d4ab4652e95d1538a0982c24185999351c
2018-01-24 15:29:16 -08:00
Chad Austin
b0fac29b0b enable python typechecking from buck
Summary: Turn on check_types for CLI's python_binary.

Reviewed By: simpkins

Differential Revision: D6668636

fbshipit-source-id: abde1e1cedf2a5104cdaa5433377b1d2adc372fd
2018-01-08 12:10:19 -08:00
Michael Bolin
48f727ee3a Use bytes string as string templates, where possible.
Summary:
In making this change, I was hoping to remove `.encode()` calls, but the net
number is zero.

Reviewed By: chadaustin

Differential Revision: D6606997

fbshipit-source-id: ea1851d719aff0bae424f15c94a7cc48b8f516f0
2017-12-19 16:51:27 -08:00
Michael Bolin
6362dff4c1 Include output from doctor, buildinfo, and uptime in eden rage report.
Summary:
This additional debugging info should help when investigating user issues.
While here, I also fixed things up so that `eden rage` does not crash if Eden
is not running (it just prints less information and tells the user).

Reviewed By: chadaustin

Differential Revision: D6599309

fbshipit-source-id: cf5c63dcc6f4dbf224122579c3bd458730629a66
2017-12-19 15:51:12 -08:00
Michael Bolin
47607cf33b Introduce eden debug uptime.
Summary:
This reports the difference between what is returned by the `aliveSince()`
Thrift method and the current time.

Reviewed By: chadaustin

Differential Revision: D6566127

fbshipit-source-id: 449103bcb31a87e4efd780299131eb38a45a6bd7
2017-12-14 21:18:14 -08:00
Michael Bolin
bb3994be82 Add an eden debug buildinfo command.
Summary:
`eden debug buildinfo` lists the exported values from the Thrift daemon that
start with `build_`.

Reviewed By: simpkins

Differential Revision: D6565668

fbshipit-source-id: 62009d7a23211765039a5045a517113043b5d8a9
2017-12-14 13:52:12 -08:00
Michael Bolin
5d738193e5 Store Hg dirstate data in Hg instead of Eden.
Summary:
This is a major change to how we manage the dirstate in Eden's Hg extension.

Previously, the dirstate information was stored under `$EDEN_CONFIG_DIR`,
which is Eden's private storage. Any time the Mercurial extension wanted to
read or write the dirstate, it had to make a Thrift request to Eden to do so on
its behalf. The upside is that Eden could answer dirstate-related questions
independently of the Python code.

This was sufficiently different than how Mercurial's default dirstate worked
that our subclass, `eden_dirstate`, had to override quite a bit of behavior.
Failing to manage the `.hg/dirstate` file in a way similar to the way Mercurial
does has exposed some "unofficial contracts" that Mercurial has. For example,
tools like Nuclide rely on changes to the `.hg/dirstate` file as a heuristic to
determine when to invalidate its internal caches for Mercurial data.

Today, Mercurial has a well-factored `dirstatemap` abstraction that is primarily
responsible for the transactions with the dirstate's data. With this split, we can
focus on putting most of our customizations in our `eden_dirstate_map` subclass
while our `eden_dirstate` class has to override fewer methods. Because the
data is managed through the `.hg/dirstate` file, transaction logic in Mercurial that
relies on renaming/copying that file will work out-of-the-box. This change
also reduces the number of Thrift calls the Mercurial extension has to make
for operations like `hg status` or `hg add`.

In this revision, we introduce our own binary format for the `.hg/dirstate` file.
The logic to read and write this file is in `eden/py/dirstate.py`. After the first
40 bytes, which are used for the parent hashes, the next four bytes are
reserved for a version number for the file format so we can manage file format
changes going forward.

Admittedly one downside of this change is that it is a breaking change.
Ideally, users should commit all of their local changes in their existing mounts,
shutdown Eden, delete the old mounts, restart Eden, and re-clone.

In the end, this change deletes a number of Mercurial-specific code and Thrift
APIs from Eden. This is a better separation of concerns that makes Eden more
SCM-agnostic. For example, this change removes `Dirstate.cpp` and
`DirstatePersistance.cpp`, replacing them with the much simpler and more
general `Differ.cpp`. The Mercurial-specific logic from `Dirstate.cpp` that turned
a diff into an `hg status` now lives in the Mercurial extension in
`EdenThriftClient.getStatus()`, which is much more appropriate.

Note that this reverts the changes that were recently introduced in D6116105:
we now need to intercept `localrepo.localrepository.dirstate` once again.

Reviewed By: simpkins

Differential Revision: D6179950

fbshipit-source-id: 5b78904909b669c9cc606e2fe1fd118ef6eaab95
2017-11-06 19:56:49 -08:00
Michael Bolin
1d7bdf0d0b Ensure all the fields of StdoutPrinter are str.
Reviewed By: simpkins

Differential Revision: D6180343

fbshipit-source-id: 8036300174991a5225de5f8c28deec4ff638b894
2017-10-30 21:38:14 -07:00
Michael Bolin
a286ad53e5 Introduce eden debug hg_dirstate command.
Summary:
Add an `eden debug hg_dirstate` command to dump the contents of the Hg dirstate.
This data is stored in a binary format, so we need a custom command to view it
easily.

Reviewed By: simpkins

Differential Revision: D6139172

fbshipit-source-id: 622c0b7bcaa471a88483c6c4ddef7e0be95a3dfa
2017-10-25 22:36:06 -07:00
Chad Austin
3240f2dff9 warn when creating new logger, allow disabling inheritance for log level
Summary: This diff helps some common pitfalls when using set_log_level.

Reviewed By: simpkins

Differential Revision: D6142849

fbshipit-source-id: 7fa35392dda148af90d0aefdb872b6e8a8b770db
2017-10-25 09:56:16 -07:00
Chad Austin
73dc6620fc add a debug CLI command to set a log category's level
Reviewed By: simpkins

Differential Revision: D6068539

fbshipit-source-id: 45d0baecc2291c83c48ad22403d44a790b270b9a
2017-10-16 16:37:10 -07:00
Michael Bolin
ee2c0f2cff Make the destination of eden rage output configurable in ~/.edenrc.
Summary:
This adds support for specifying where the output of `eden rage` should be sent
by adding a `reporter` property to the `[rage]` section of `~/.edenrc` like so:

```
[rage]
reporter = arc paste
```

If specified, Eden will take the command as an opaque string and use it as an
argument to `sh -c <reporter_from_edenrc>` when spawning a subprocess that will
receive the output of `eden rage` on its stdin. If no reporter is specified,
then the stream will be written directly to stdout.

Reviewed By: wez

Differential Revision: D6049355

fbshipit-source-id: beafe37eed8c48dac7499a6978cacfb44d8f253e
2017-10-16 11:56:23 -07:00
Braden Watling
ab43c66a8d Add test to verify that eden debug getpath indicates when inodes are unloaded
Summary:
This test was supposed to be a part of D5627411 but it was causing strange behaviour so was brought to a separate diff for further investigation.

After investigating, the test didn't pass because the UnloadedInodeData struct only contained the name of the file, not the path to it. The fix for this was to implement a way to get the relative path of the file even after the inode is unloaded.

Reviewed By: simpkins

Differential Revision: D5646929

fbshipit-source-id: f166398a651e8aea49da7e4474a5ad7fde2eaa4e
2017-08-25 08:34:31 -07:00
Michael Bolin
8ff2ba5d5b Use pwd as the default argument for eden debug overlay.
Summary:
Also modified the `getpath` handler to defer the `os.getcwd()` call.

(Note: this ignores all push blocking failures!)

Reviewed By: simpkins

Differential Revision: D5698232

fbshipit-source-id: 9483907771fd1fd2918f62120664bb0d8c431cf3
2017-08-24 14:23:39 -07:00
Michael Bolin
0508c0b819 Change eden debug getpath to use pwd as the default mount point.
Summary: This seems a little more user-friendly.

Reviewed By: bradenwatling

Differential Revision: D5686562

fbshipit-source-id: 8142fb9105a3a44823f935fc04187cf0ed2258d7
2017-08-23 11:27:22 -07:00
Michael Bolin
303655c4b1 Add integration test for hg copy.
Summary:
Note that this feature was mostly implemented before this commit, but never
tested. Unsurprisingly, there were bugs.

This change also introduces a new `eden debug hg_copy_map_get_all` subcommand
because that was a straightforward way to verify the internal state of the copy
map on the server side from an integration test.

Adding this test uncovered a key copy/paste bug in `EdenThriftClient.py`
(`hgCopyMapGet` was being invoked instead of `hgCopyMapPut`.)

It also uncovered a bug in `LameThriftClient` because the `compile()` and
`eval()` calls on the output are not appropriate when the return type of the
Thrift endpoint is `string`.

Reviewed By: simpkins

Differential Revision: D5686114

fbshipit-source-id: f0093d2b67062c01982dc5bc1f0db2774b3a9356
2017-08-22 21:06:07 -07:00
Jyothsna Konisa
371cfa097d TestCase to verify unloadChildrenNow with age
Summary:
Added a test case `test_unload_free_inodes_age` to verify the behaviour of unloadChildrenNow with age parameter.
Added new parameter age to `unloadInodeForPath` in eden.thrift, and `EdenServiceHandler`.
Modified `do_unload_inodes` function in `debug.py` to support the new behaviour.

Reviewed By: simpkins

Differential Revision: D5565859

fbshipit-source-id: a35053725be26bc906cf158969cbe21db1cbadde
2017-08-22 19:50:00 -07:00
Simon Jensen
3a0e66677b Cache invalidation thrift api
Summary: Provide a thrift interface to invalidate the cache for an inode denoted by path.

Reviewed By: simpkins

Differential Revision: D5655387

fbshipit-source-id: 887aa4963d216a0d8eed93b6fb8721632cc31d19
2017-08-21 16:05:44 -07:00
Michael Bolin
70050affcc Introduce eden debug hg_get_dirstate_tuple.
Summary:
This is a convenient way to test the `hgGetDirstateTuple()` endpoint in
`eden.thrift`.

Reviewed By: quark-zju, wez

Differential Revision: D5654237

fbshipit-source-id: 0b285e056002d4556733a53293582345f36780b2
2017-08-18 21:49:59 -07:00
Braden Watling
cf297e0106 Add subcommand getpath to eden debug
Summary: Add a command to lookup the path for an inode given the inode number and eden mount path.

Reviewed By: bolinfest

Differential Revision: D5627411

fbshipit-source-id: 25928f506d3f48d8a6784fe81fb17fa0500d6bc9
2017-08-16 20:04:30 -07:00
Wez Furlong
39b21f383b compensate for the new header when debugging the overlay
Summary:
I'm trying to troubleshoot why a new dir that I added isn't showing up in the overlay and I found that the debug tool doesn't know about the overlay.

This is the dumbest minimal thing I could do to make this basically functional again

Reviewed By: simpkins

Differential Revision: D5602439

fbshipit-source-id: 4ae7fa34136697f9f915ccd95275cb3a7923ae4a
2017-08-10 11:42:25 -07:00
Adam Simpkins
8be3b57eed fix issues updating TreeInode materialization status during checkout
Summary:
This updates the TreeInode code to remove the redundant materialized flag.
A TreeInode should have a Tree Hash if and only if it is dematerialized, so
there is no need for an extra `materialized` boolean.

This diff also fixes an issue in TreeInode::saveOverlayPostCheckout() where it
was not correctly informing it's parent TreeInode of the change if it moved
from one dematerialized state to another (with a different TreeInode hash).
This fixes the code to correctly call `parent->childDematerialized()` when it
needs to inform the parent that it now refers to a different source control
hash.

Reviewed By: wez

Differential Revision: D5336629

fbshipit-source-id: b4d86ecdef2f5faefbc243a09d869c02384ae95c
2017-07-07 18:45:02 -07:00
Adam Simpkins
305325319c add an "eden debug overlay" command
Summary:
Add a command to deserialize and display information about files in the
overlay.  This can be used to help debug the current state of files in the
overlay.

Reviewed By: wez

Differential Revision: D5332014

fbshipit-source-id: 25b01579df33aa9f1926c0144e9f03aa8ece38fc
2017-07-07 18:45:01 -07:00
Jyothsna Konisa
299f7d1373 Integration Test for UnnloadFreeInodes
Summary:
1. Moved read, write, mkdir, rm methods in hg/lib/hg_extension_test_base.py to lib/test_case.py.
2. Added integration test case to test unload free inodes.

Reviewed By: simpkins

Differential Revision: D5277870

fbshipit-source-id: b93b6049a10357cf8c92366e6dca3968f7f30c30
2017-06-22 22:38:47 -07:00
Jyothsna Konisa
b8be1633bc Cli command to unload free inodes of a under a given directory.
Summary:
1.Added a new thrift method to unload free inodes of a directory and its sub directories.
2.Added a new debug sub command 'eden debug unload <path>' to cli tools to unload free inodes.

Reviewed By: simpkins

Differential Revision: D5261038

fbshipit-source-id: 85b4c5ae18c0ae24c666a44ac9892765e753397f
2017-06-16 16:22:32 -07:00
Jyothsna Konisa
8da0797e92 Fix in eden debug for relative path in get_path_info()
Summary: made the relative path return "" when mount point directory is returned "."

Reviewed By: simpkins

Differential Revision: D5250543

fbshipit-source-id: bd0d648db8030cfc6ca952fccfb9972f18290b5e
2017-06-15 11:12:11 -07:00
Adam Simpkins
ce0ce6fa4e move eden/fs/cli to eden/cli
Summary:
Move the code for the command-line tool up one directory, out of eden/fs.
This better separates the code so that eden/fs contains code for the edenfs
daemon, while eden/cli contains code for the command line tool.

Reviewed By: bolinfest

Differential Revision: D4888633

fbshipit-source-id: 5041e292c5353d05122eefe5db3257289e31239a
2017-04-14 11:39:01 -07:00