Commit Graph

182 Commits

Author SHA1 Message Date
Michael Bolin
0792dc88e9 Remove some unused Python imports.
Summary: I did what the linter told me to do.

Reviewed By: wez

Differential Revision: D3836659

fbshipit-source-id: a5d3fc8974cf6cb7c7e2d88a6215ac5c54479780
2016-09-08 12:45:02 -07:00
Caren Thomas
670297fa88 add ClientConfig method that compiles repository data
Summary: Restructure the current logic used for loading the config data into a ClientConfig object. Rather than having loadFromClientDirectory iterate through all the config files and parse them to find the necessary information, abstract that logic out into a new method that compiles all of the relevant data so that all loadFromClientDirectory has to do is pull out the needed information. Since this change separates the two steps, this will make it easier to move the first step of compiling config information outside of ClientConfig - the goal here is to have the eden server load all of the config data at start up and cache it in memory so that it doesn't need to be done every time a ClientConfig object is created, and this change is an intermediate step.

Reviewed By: simpkins

Differential Revision: D3580757

fbshipit-source-id: c340a0fe715856066a554238249574f8177bc4d7
2016-08-05 12:50:29 -07:00
Michael Bolin
e17e7c15eb Use the realpath to normalize the path to the clone.
Summary: This should facilitate things by making the way to reference the path unambiguous.

Reviewed By: simpkins

Differential Revision: D3672311

fbshipit-source-id: b2a21e4dba60ea5ded28643f76aa402c4250d7ac
2016-08-05 11:48:07 -07:00
Adam Simpkins
59319918bd re-add the "eden mount" command
Summary:
Add back the "mount" CLI command.

While end-users may not need this command under normal circumstances, during
development it is sometimes necessary to re-mount unmounted clients.  This adds
back the "mount" command which simply re-mounts an already configured client.

This also adds a "--no-forget" option to "unmount" which causes it to unmount
the client without forgetting its configuration.  This allows it to be
remounted with "mount", rather than having to re-run the "clone" command and
specify the repository name again (and potentially restore the desired snapshot
hash).

Reviewed By: wez

Differential Revision: D3653706

fbshipit-source-id: 0e3f3fe1efdf72e9e54bdd1e0fb308ae67b7c058
2016-08-03 16:29:21 -07:00
Adam Simpkins
a03cdb4d57 update the integration tests to allow running under gdb
Summary:
When the EDEN_GDB environment variable is set, start eden under gdb when
running the integration tests.  Have gdb automatically start the daemon,
and exit automatically if everything is successful.  If the daemon crashes, gdb
will break and the user can interact with gdb normally.

This is generally only useful when manually running a single test.  Trying to
use this in combination with "buck test" will probably cause problems if
multiple gdb instances all try to interact with the terminal together.

Reviewed By: wez

Differential Revision: D3459575

fbshipit-source-id: 1359735a391e93a332a4c673c6525ad44034cc73
2016-07-25 15:14:33 -07:00
Adam Simpkins
98db6e0e01 make sure to close the thrift socket
Summary:
Update the CLI to always close the thrift client socket, to avoid resource leak
warnings on exit.

I also updated the code to just monkey-patch a nicer EdenError.__str__()
method, rather than having to explicitly catch and modify this exception in
multiple different places.

Reviewed By: bolinfest

Differential Revision: D3560662

fbshipit-source-id: 900fe74c793ffd99f4a2c1f1ddd94b96e48f5eb7
2016-07-22 17:33:05 -07:00
Adam Simpkins
90e1a87409 improvements to the repository command
Summary:
The "eden repository <name> <path>" command had a bug that it would duplicate
the existing edenrc contents when writing out a new config: it opened the file
in append mode rather than truncate mode when performing the write.

This addresses that issue, but also does a bigger overhaul of the config update
code.  We now acquire a lock for the duration of the modification, so that
another eden CLI process cannot modify the file between when we read it and
when we write out our modifications.  We also perform the write using an atomic
rename so the file contents are always valid at all points in time.

I also updated the CLI command to print the repositories in sorted order, and
to only catch expected exceptions, and show backtraces for unexpected errors.

Reviewed By: bolinfest

Differential Revision: D3554550

fbshipit-source-id: 5920ccb2447330673eac3f9956a8ae5d8a66a67e
2016-07-22 17:33:00 -07:00
Caren Thomas
e6bbd1e098 fix exception messages
Summary: This change fixes exception messages so that they all match the same format and don't include 'error: ' twice

Reviewed By: simpkins

Differential Revision: D3544034

fbshipit-source-id: 32d600c712d1782d2a1a35331b66e06a842ed8e0
2016-07-11 13:07:21 -07:00
Caren Thomas
96a63ff972 restructure eden directory
Summary:
These changes restructure the eden directory so that 'client' directories are created during the `eden clone` command and are associated with a single mount path.
The new eden directory looks as follows:
  ~/.eden
      config.json
      clients/
          abcd08d/
              edenrc
              SNAPSHOT
              overlay/
          efgh19i/
              edenrc
              SNAPSHOT
              overlay/
              ...

Where the config.json file holds the mapping of mount paths to their respective client directory which is a hash, and the edenrc files in each client directory is an INI file which holds the name of repository associated with the mount path. This INI file follows the current format:
    [repository]
    name = fbsource

This restructuring required a couple other changes:
- unmount command now cleans up the client directory and removes the mapping of its mount path from config.json
- eden list command now lists all of the mount paths rather than the client names

Reviewed By: bolinfest

Differential Revision: D3506119

fbshipit-source-id: dc07a8baf1052be731ff335d9cf74a07ab8e661a
2016-07-06 16:15:50 -07:00
Caren Thomas
ad7f71f0b4 update ClientConfig class to parse INI file
Summary: Change the ClientConfig class to parse client data via INI config file rather than json file. This class uses boost::property_tree::ini_parser and the ptree data structure to hold the parsed INI file contents. This change makes it possible for eden to no longer rely on json files for getting client data, and the json files will be completely taken out in a separate diff.

Reviewed By: bolinfest

Differential Revision: D3498567

fbshipit-source-id: 3298047a014beda0c250475c0809a7a1ebd95b2b
2016-07-06 16:15:50 -07:00
Caren Thomas
05c63233c4 move snapshot update to clone command
Summary: Previously the repo snapshot id was collected and stored when the repository was initialized, but this diff moves the update to the clone command so now the snapshot is only stored right before a repository is mounted.

Reviewed By: bolinfest

Differential Revision: D3489550

fbshipit-source-id: 1271c1d7c7709ed332307a636c23e26e913483c9
2016-07-06 16:15:49 -07:00
Caren Thomas
5d7b826b19 change unmount to use mount path as arg
Summary: Update the unmount command to use the mount point rather than the client name. This diff also removes the mount command completely since tests are no longer dependent on it.

Reviewed By: bolinfest

Differential Revision: D3489290

fbshipit-source-id: 9bfbcb78a78d1593f6d0950502bdd4a55e3e6ca4
2016-07-06 16:15:49 -07:00
Caren Thomas
c9779be011 add 'eden clone' command
Summary: Introduce new 'clone' command that takes in repo name and mount path. This command replaces the current mount command which uses the client name as an argument

Reviewed By: bolinfest

Differential Revision: D3489098

fbshipit-source-id: 719fb94387da1d4ccab770a0e659f717888a7dc6
2016-07-06 16:15:49 -07:00
Caren Thomas
ce44a06b80 add repository command
Summary: Expand functionality of repository command to also create repositories. The current command line format is 'eden repository <name> <path>' and will soon replace the existing init command.

Reviewed By: bolinfest

Differential Revision: D3489064

fbshipit-source-id: 3e6946cbd80a64ad23b52383f3f14a92d4492a21
2016-07-06 16:15:49 -07:00
Caren Thomas
86238c4098 add simple eden repository command
Summary: Add a simple eden repository command that takes no arguments. The behavior mimics that of git remote and lists all repositories by parsing both the global and local INI config files. The INI file currently needs to be manually added and updated to maintain information about existing repositories.

Reviewed By: bolinfest

Differential Revision: D3479543

fbshipit-source-id: b0d36f33c9b8e0c7b9fe20ec2f362ca15c0adeb8
2016-07-06 16:15:49 -07:00
Michael Bolin
911b0bde9b Update _find_default_daemon_binary() because the binary is always named edenfs.
Reviewed By: simpkins

Differential Revision: D3492527

fbshipit-source-id: b74859ebd2cfac55568bdb170a863d78607ad108
2016-06-28 11:02:05 -07:00
Caren Thomas
35add0f007 change eden cli to run with python 3
Summary: Update eden/fs/cli directory to run with Python 3.

Reviewed By: simpkins

Differential Revision: D3479252

fbshipit-source-id: 3e3dc023fc54e99b7839a1a4dc8605dd7ef2d8a3
2016-06-27 11:59:24 -07:00
Adam Simpkins
142aba2058 add repository helper functions in the integration tests
Summary:
Start adding utility code to make it easier for the integration tests to
manipulate git and mercurial repositories.

This adds an HgRepository class, but does not update the tests to use it yet.
I will do that in a separate diff.

Reviewed By: bolinfest

Differential Revision: D3449328

fbshipit-source-id: caea5befb683a388cbad30a29017c566e22702af
2016-06-20 13:40:02 -07:00
Adam Simpkins
c51e282dfb import git objects on demand
Summary:
This moves git import logic from the GitImporter class to GitBackingStore.
The logic is simpler now, since GitBackingStore only needs to import a single
Tree or Blob at a time.

Reviewed By: bolinfest

Differential Revision: D3448752

fbshipit-source-id: da2d59f953ada714d8512545ae83dd48e5d3e410
2016-06-20 11:45:09 -07:00
Adam Simpkins
94a26b5503 improvements to the daemon and shutdown commands
Summary:
- Update "eden daemon" to wait until edenfs is healthy before returning.
- Accept additional command line arguments and pass them through to edenfs.
- When starting edenfs as a background daemon, direct stdout and stderr to a
  log file in the eden config directory.
- Update "eden shutdown" command to wait for edenfs to exit.
- Update "eden shutdown" command to print an error message if edenfs was not
  running.

Reviewed By: bolinfest

Differential Revision: D3446403

fbshipit-source-id: 2ca1874256b7d124a85092b886548bea8c198327
2016-06-17 15:12:48 -07:00
Adam Simpkins
cbb493f716 support running in the foreground, and improve test cleanup
Summary:
This adds a --foreground flag to the "eden daemon" command, which causes the
daemon to run in the foreground, rather than daemonizing as a background
process.  Additionally, this drops the --preserve-environment flag and instead
updates the CLI to always build a sane environment to run with.

This also updates the integration test code to use this flag, and wait for eden
to shut down during test clean up.  Without this, the test case code could try
to clean up temporary directories before they were unmounted, which would fail.
This would leave many temporary directories behind after test runs.

This also re-organizes the temporary directory set up a bit, so that each test
uses a single top-level temporary directory.  All other directories it creates
are put inside this one directory.

Reviewed By: bolinfest

Differential Revision: D3439232

fbshipit-source-id: 85305f7db60da6bae589d28c802ee260e42e4dea
2016-06-16 12:44:06 -07:00
Michael Bolin
26403df852 Exclude the cpp_binary for the daemon from the python_binary for the CLI.
Summary:
This changes the way that Eden is built and deployed.

* To build the binary that must be run as `root` (but quickly drops privileges), run `buck build eden-daemon`.
* To build the CLI that communicates with the daemon (and does not require privileges), run `buck build eden-cli`.
* To build both, run `buck build eden`.

There is an example of how to build the various parts of Eden using
Buck and how to package them up in the `install` script introduced by this revision.

While here, I also cleaned up some of our build files and changed them to be
parameterized between internal and external use. In both cases, the user gets the
"unadorned" version of their primary build targets. This ensures that shortcuts such as:

```
buck test eden/fs/integration
```

do the right thing by default.

Finally, I also made `find_default_config_dir()` and `find_default_daemon_binary()`
lazy whereas `find_default_config_dir()` was previously eager.

Reviewed By: simpkins

Differential Revision: D3436245

fbshipit-source-id: 4dfbd59ed0d198620324f0705c462334bb5a7daf
2016-06-15 17:07:58 -07:00
Adam Simpkins
346a4cdc4e fix CLI issues initializing mercurial mount points
Summary:
The _get_hg_dir() function had an incorrect return statement caused "eden init"
to fail with mercurial repositories, unless the path pointed to a shared
working directory.

Also update the code to store the path to the mercurial repository, rather than
the path to the .hg directory.  Unlike git, mercurial requires the repository
path itself, and refuses to work if given a path to the .hg directory.

Reviewed By: bolinfest

Differential Revision: D3409808

fbshipit-source-id: 22ee7748422b614f5af57aff07fcfa517af846d9
2016-06-13 15:16:29 -07:00
Michael Bolin
11169bcf95 Add an eden shutdown command.
Summary:
This revision introduces two complementary changes:
* `eden daemon` no longer runs in the foreground.
* There is now an `eden shutdown` command to kill the daemon.

When `shutdown` is called, it tells the Thrift server to shutdown.
In turn, this causes `EdenServer::runThriftServer()` to exit,
which causes `EdenServer::run()` to exit.

Reviewed By: simpkins

Differential Revision: D3402347

fbshipit-source-id: 80032ba53eb69b3f69bef9d7cd169f93500c833c
2016-06-10 14:16:09 -07:00
Michael Bolin
7a48628bb8 Create dot eden folder if it does not already exist when running eden daemon.
Summary:
Previous to this change, if `~/local/.eden` did not exist when `eden daemon` was
run, then it would fail.

Now the logic to create `~/local/.eden` is encapsulated in `_ensure_dot_eden_folder_exists()`.
We do not call this for all subcommands because we want to make sure that running
`eden --help` does not have the side-effect of writing `~/local/.eden` if it does not exist.

Reviewed By: wez

Differential Revision: D3397057

fbshipit-source-id: a3f974f367058d9e4ebd515c78423e54edc179cc
2016-06-07 13:03:13 -07:00
Michael Bolin
5e69f112eb Add an eden health subcommand.
Summary:
`eden health` will return with exit code 0 if both of the following are true:

* The Thrift client is up and running.
* The status of the client is either ALIVE or STARTING.

Reviewed By: wez

Differential Revision: D3395582

fbshipit-source-id: ba668d26acae73a51fbae8aca2b4979156c0c50f
2016-06-07 13:03:13 -07:00
Michael Bolin
1391f7725c Fix some Python lint warnings.
Reviewed By: simpkins

Differential Revision: D3394466

fbshipit-source-id: 3dd670785e0eb2586ed38be87e74caeb3262966b
2016-06-06 21:01:03 -07:00
Caren Thomas
203be051a1 add unmount parser and wrapper functions
Summary: Updated python CLI to include subparser for unmount command and added wrapper functions that hand over execution to privhelper process. Unmount currently requires client_name at the command line.

Reviewed By: simpkins

Differential Revision: D3359517

fbshipit-source-id: ff05e90bcdb96ecad63f37634c69dbeef429c90f
2016-05-27 17:41:07 -07:00
Michael Bolin
6e4cb1ebf1 Refactor logic to create a EdenService.Client in Python.
Summary: This logic should be shared by the Eden CLI as well as unit tests.

Reviewed By: simpkins

Differential Revision: D3348300

fbshipit-source-id: c87b1f03f16560323f3d7685063bb6466c39efe2
2016-05-25 21:44:07 -07:00
Adam Simpkins
98dcc73503 initial CLI support for creating hg mounts
Summary:
Update the CLI to support running "init" with a mercurial repository.

This is just some bare bones framework code at the moment.  It doesn't actually
import any data from mercurial at the moment, and mounting doesn't work.

Reviewed By: bolinfest, wez

Differential Revision: D3345426

fbshipit-source-id: 72c31ac8d2aac2a16e0a7d6f0425eb4ca218d487
2016-05-25 13:15:01 -07:00
Wez Furlong
deaf75a6b6 eden: init: require --mount and --repo options
Summary: This avoids an ugly error when --mount isn't passed.

Reviewed By: bolinfest

Differential Revision: D3302970

fbshipit-source-id: e60475e6b94fb4228e75fb392fb472886274aa0a
2016-05-17 18:17:11 -07:00
Facebook Github Bot 5
2eeea32117 Initial commit
fbshipit-source-id: 2bcefbd0cd127cc5ea982e074ea6819d7aac3d7a
2016-05-12 14:09:13 -07:00