Commit Graph

88 Commits

Author SHA1 Message Date
Adam Simpkins
c3ebc91fdc update hg integration test inheritance to allow type checking
Summary:
Python 3 type checking currently complains about most of our integration
testing since the tests use an `hg_test` decorator to inherit from the base
test class.  This prevents the type checker from being able to figure out this
inheritance.

This updates all of the test cases to explicitly derive from the test case base
class, rather than using the decorator to do so.  I also renamed the base test
case class to `EdenHgTestCase` to be slightly more succinct and to follow the
common pattern of calling `unittest.TestCase` subclasses `FooTestCase`

Reviewed By: bolinfest

Differential Revision: D6268258

fbshipit-source-id: 09eef2f8217932a6516f78d17dddcd35c83b73da
2017-11-07 19:04:20 -08:00
Adam Simpkins
017f636ad3 add an integration test to exercise importing during "hg status"
Summary:
This adds an integration test that exercises a deadlock we could encounter in
the past.  An "hg status" operation could trigger many trees and files to be
imported.  Unfortunately the file import code currently blocks waiting for file
import futures to complete.  This could result in a state where all threads in
the pool were waiting for a file import to complete, and the file import was
waiting for a free thread to complete.

Reviewed By: bolinfest

Differential Revision: D6216871

fbshipit-source-id: e1795a543a71fccbed035febb159e126e27d1950
2017-11-07 19:04:20 -08:00
Michael Bolin
4c24e5bd9f hg prev should leave untracked files alone if no conflicts, even if in a directory.
Summary:
This fixes an issue where the `DIRECTORY_NOT_EMPTY` conflict type reported by
the server was not handled by the client. Somewhat ironically, the fix appears
to be to explicitly "do nothing," though the important part of this revision is
the new integration test.

As this is only one test, I'm not convinced this covers all possible corner
cases, but it's certainly better than blowing up, which is what we did before.

Reviewed By: wez

Differential Revision: D6264069

fbshipit-source-id: a7c45a43776a903a4d6b6cdfb0ce75db9549c380
2017-11-07 19:04:20 -08:00
Michael Bolin
836b0e7f18 Minor cleanup to eden/integration/lib/hgrepo.py.
Summary:
A few fixes:

* Fix a bug where `date_str` would not get set when `date` was specified.
* Remove `from __future__ import` stuff since this code is Python 3.
* Add type annotations to the `commit()` method.

Reviewed By: simpkins

Differential Revision: D6261874

fbshipit-source-id: 5f942d01c107cd0265c2d6ec6e1f46295bb3ec24
2017-11-07 19:04:20 -08:00
Michael Bolin
a43eb5afaf Add proper support for the UNTRACKED_ADDED conflict type.
Summary:
If you have an untracked file and you `hg update` to a commit that has
that file in the tracked state, then the contents of the untracked version
should be ignored, as they are replaced with the contents of the file in the
commit you are updating to. The untracked version should be backed up
as specified by `ui.origbackuppath`.

Previously, our code in `eden/hg/eden/__init__.py` mapped this to a merge action
named `c`, but we did not include that in our set of `actions`, so we were
getting a `KeyError` if you exercised this code path.

I discovered this while trying to reproduce the issue that I fixed in D6199215.

Reviewed By: simpkins

Differential Revision: D6204916

fbshipit-source-id: b70153428291bda9a8853a37c0955ad7cb3bd89d
2017-11-07 17:50:52 -08:00
Chad Austin
bf03a9420d add test verifying eden help succeeds
Summary:
Since Eden's integration tests do not run in CI yet, this adds a
test that verifies the Eden CLI can start without Python errors.

Reviewed By: wez

Differential Revision: D6250515

fbshipit-source-id: 907bffaff122c9929a7623d97f665de5b2a6f2d3
2017-11-07 16:06:32 -08:00
Michael Bolin
5d738193e5 Store Hg dirstate data in Hg instead of Eden.
Summary:
This is a major change to how we manage the dirstate in Eden's Hg extension.

Previously, the dirstate information was stored under `$EDEN_CONFIG_DIR`,
which is Eden's private storage. Any time the Mercurial extension wanted to
read or write the dirstate, it had to make a Thrift request to Eden to do so on
its behalf. The upside is that Eden could answer dirstate-related questions
independently of the Python code.

This was sufficiently different than how Mercurial's default dirstate worked
that our subclass, `eden_dirstate`, had to override quite a bit of behavior.
Failing to manage the `.hg/dirstate` file in a way similar to the way Mercurial
does has exposed some "unofficial contracts" that Mercurial has. For example,
tools like Nuclide rely on changes to the `.hg/dirstate` file as a heuristic to
determine when to invalidate its internal caches for Mercurial data.

Today, Mercurial has a well-factored `dirstatemap` abstraction that is primarily
responsible for the transactions with the dirstate's data. With this split, we can
focus on putting most of our customizations in our `eden_dirstate_map` subclass
while our `eden_dirstate` class has to override fewer methods. Because the
data is managed through the `.hg/dirstate` file, transaction logic in Mercurial that
relies on renaming/copying that file will work out-of-the-box. This change
also reduces the number of Thrift calls the Mercurial extension has to make
for operations like `hg status` or `hg add`.

In this revision, we introduce our own binary format for the `.hg/dirstate` file.
The logic to read and write this file is in `eden/py/dirstate.py`. After the first
40 bytes, which are used for the parent hashes, the next four bytes are
reserved for a version number for the file format so we can manage file format
changes going forward.

Admittedly one downside of this change is that it is a breaking change.
Ideally, users should commit all of their local changes in their existing mounts,
shutdown Eden, delete the old mounts, restart Eden, and re-clone.

In the end, this change deletes a number of Mercurial-specific code and Thrift
APIs from Eden. This is a better separation of concerns that makes Eden more
SCM-agnostic. For example, this change removes `Dirstate.cpp` and
`DirstatePersistance.cpp`, replacing them with the much simpler and more
general `Differ.cpp`. The Mercurial-specific logic from `Dirstate.cpp` that turned
a diff into an `hg status` now lives in the Mercurial extension in
`EdenThriftClient.getStatus()`, which is much more appropriate.

Note that this reverts the changes that were recently introduced in D6116105:
we now need to intercept `localrepo.localrepository.dirstate` once again.

Reviewed By: simpkins

Differential Revision: D6179950

fbshipit-source-id: 5b78904909b669c9cc606e2fe1fd118ef6eaab95
2017-11-06 19:56:49 -08:00
Michael Bolin
95fd684e2e Both ENOTDIR and ENOENT should be ignored when stat'ing a possibly missing file.
Summary:
There is logic in `eden_dirstate.walk()` that looks to see if any of the files
that are reported as "removed" by `hg status` are still on disk, and if so,
should be considered for a walk. Because the files are likely removed, we were
catching `ENOENT` for a failed `os.stat()`, but we also needed to be catching
`ENOTDIR`. This turned out to be the reason `hg add` was failing in a specific
case, for which we already had an integration test, but it wasn't passing until
now.

Reviewed By: simpkins

Differential Revision: D6207233

fbshipit-source-id: 44e5252bb0130ca279160f0a64286053fa5509d5
2017-11-01 20:49:44 -07:00
Michael Bolin
a0a0f9a63c Use a more direct stringification of an exception.
Summary:
Relying on the toString of an Exception in Python seems a little gross,
especially when the `stderr` field is available directly. Cleaned up two
instances of this so it doesn't get copypasta'd further.

Reviewed By: simpkins

Differential Revision: D6195633

fbshipit-source-id: 9ae77796c287a454cb169ebf6de2953909a1e6c3
2017-10-31 17:06:08 -07:00
Michael Bolin
e07ef44b1c Print a sensible error message when the user tries to mount an existing mount.
Summary:
This was an error that an end-user ran into. Previously, we did not fail
gracefully and the user was faced with an intimidating stacktrace.

Reviewed By: simpkins

Differential Revision: D6195529

fbshipit-source-id: bde3c2a3e6f49457a4c6ac5c87103cf52cd227c2
2017-10-31 17:06:08 -07:00
Michael Bolin
0ec034fde4 Add an integration test for removing a directory and replacing it with a file.
Summary:
I ran into this issue while manually testing Eden.

Currently, this integration test fails, so it is tagged with `unittest.skip`.
There are substantial changes to our distate logic coming in D6179950, so I
will attempt to make the test pass as part of that revision.

Reviewed By: simpkins

Differential Revision: D6199789

fbshipit-source-id: cd7ce48b72bf0b54e13547b23823f4d496fa5b0b
2017-10-31 14:02:57 -07:00
Michael Bolin
ed155d84cc Add p and pr to the list of potential merge actions when dealing with conflicts.
Summary:
Upstream, some new merge actions were added:

* `p` https://phab.mercurial-scm.org/D776
* `pr` https://phab.mercurial-scm.org/D777

We must include entries for these in the list of `actions` that we build up in
`eden/hg/eden/__init__.py` because the `actions` dict gets passed through to
Mercurial's own `applyupdates()` function in `merge.py` that contains this line:

```
for f, args, msg in actions['p']:
```

Therefore, without an entry for `p` in `actions` here, we get a `KeyError`.

Reviewed By: markbt

Differential Revision: D6199215

fbshipit-source-id: a7408e5ef84a659f37e7771a7c15f6a4b14ae0f9
2017-10-31 12:24:16 -07:00
Michael Bolin
28214295bb Add some extra assertions to GraftTest.
Summary:
In practice, if the `hg graft` succeeds in a weird way, `assert_status_empty()`
tells a lot more about what went wrong than the number of commits not matching up.

While here, I also added the following entry to the default `.hgrc` used in integration tests:

```
[ui]
origbackuppath=.hg/origbackups
```

I needed this for the change to `graft_test.py`. As we were already setting this option in
the `histedit_command.py` utility as a one-off and this is the default value of this setting
for our internal Mercurial use at Facebook, it seemed best to make it the default for all
of our integration tests. As such, I removed the one-off setting in `histedit_command.py`.

Reviewed By: simpkins

Differential Revision: D6180342

fbshipit-source-id: 6f0487624a1824459403126997ea52d1a7921feb
2017-10-30 21:38:14 -07:00
Adam Simpkins
74c1027bba flush pending transaction data in eden_dirstate_map.setparents()
Summary:
Previously we flushed the pending transaction data in
eden_dirstate.setparents().  However, some dirstate code paths (particularly
dirstate.rebuild()) can directly call eden_dirstate_map.setparents().

We need to make sure the transaction data is flushed in this case.

Reviewed By: bolinfest

Differential Revision: D6175410

fbshipit-source-id: 256cb07f57ada02d6c1f118ec5075fb8ac93506c
2017-10-27 14:26:30 -07:00
Michael Bolin
9ead3fe4f3 Print an appropriate error message if an invalid repo is passed to eden clone.
Summary:
Previous to this change, the user got an inscrutable error message. It turns out
that it is easy to make this mistake, typing `eden clone fbsource/` instead of
`eden clone fbsource` if you accidentally use tab completion.

Reviewed By: simpkins

Differential Revision: D6153889

fbshipit-source-id: 3642fdd207d6abf896d6a12891d5eb68ad984acc
2017-10-25 22:36:06 -07:00
Michael Bolin
ac5b213e92 Include the dirstate tuples and copymap when backing up the dirstate.
Summary:
Previously, the `savebackup()` and `restorebackup()` methods in `eden_dirstate`
only retained the parent commit hashes. With this change, now the dirstate tuples
and entries in the copymap for the dirstate are also included as part of the saved
state.

Failing to restore all of the state caused issues when doing things like aborting
an `hg split`, as observed by one of our users. Although this fix works, we ultimately
plan to move the responsibility for persisting dirstate data out of Eden and into the
Hg extension. Then the data will live in `.hg/dirstate` like it would for the default
dirstate implementation.

Reviewed By: simpkins

Differential Revision: D6145420

fbshipit-source-id: baa077dee73847a47cc171cd980cdd272b3a3a99
2017-10-25 22:36:06 -07:00
Michael Bolin
a286ad53e5 Introduce eden debug hg_dirstate command.
Summary:
Add an `eden debug hg_dirstate` command to dump the contents of the Hg dirstate.
This data is stored in a binary format, so we need a custom command to view it
easily.

Reviewed By: simpkins

Differential Revision: D6139172

fbshipit-source-id: 622c0b7bcaa471a88483c6c4ddef7e0be95a3dfa
2017-10-25 22:36:06 -07:00
Adam Simpkins
2e6ed25612 flush kernel caches properly for empty directories removed by checkout
Summary:
When performing an source control checkout operation, we attempt to remove
directories that are empty after the checkout.  However, this code path was
missing a call to flush the kernel cache for these directories.

As a result, even though eden thought the directory not longer existed, and
would not report it in `readdir()` results, the kernel would return stale
information from its cache when explicitly accessing this path.

Reviewed By: bolinfest

Differential Revision: D6151543

fbshipit-source-id: 6031feb268ff6f980c885efb26c3d43243dec3f4
2017-10-25 16:51:56 -07:00
Michael Bolin
264d6fa9b7 Fix bug where hg histedit --abort does not restore file as "normal".
Summary:
This ports some logic from Mercurial's `dirstate.py` to our `eden_dirstate.py`.
It was known that Eden was missing this behavior (D5686636), but we did not have
an integration test that demonstrated the importance of this logic until now.

Admittedly, the current implementation ports the logic verbatim from
`dirstate.py`, though this will yield quite a number of Thrift calls in the Eden
implementation. We will address this in a subsequent revision.

Reviewed By: simpkins

Differential Revision: D6046273

fbshipit-source-id: f7a27ba6dca36cddac898f19637f29f3bc79a0cb
2017-10-12 19:28:11 -07:00
Michael Bolin
9e8e24d7df Create fix and test for hg merge.
Summary:
Running Mercurial's own integration tests revealed that we had a bug here:
https://www.mercurial-scm.org/repo/hg/file/tip/tests/test-histedit-arguments.t

Somewhat unsurprisingly, it was time to finally address a longstanding `TODO`
in `Dirstate.cpp`. The issue was that, after running `hg merge --tool :local`,
`hg status` was not including a merged file in the list of modified files. Because
the information from `hg status` is used to create a commit context, that meant
that when a commit was made after running `hg merge`, the commit did not
include the merged file in the list of files for the commit, which differs from
Mercurial's behavior.

Most of the implementation of `hg status` on the Eden side is done by
`EdenMount.diff()`. However, in this case, `diff()` does not categorize the
merged file by invoking one of the methods of `InodeDiffCallback` because
as far as `EdenMount` is concerned, the file has not changed because `EdenMount`
is unaware of the `Dirstate`. We already have some existing cases where we have
to do some post-processing on the result of `EdenMount.diff()` using information
in the `Dirstate` (e.g., files that are marked for addition or removal), so the fix was
to add a check for the case when the file is flagged as "needs merging" and
then including it as modified in the `hg status` output, as appropriate.

Reviewed By: wez

Differential Revision: D6005603

fbshipit-source-id: 7d4dd80e1a2e9f4b98243da80989e0e9119a566d
2017-10-09 11:55:34 -07:00
Michael Bolin
ef6f17696e Update RollbackTest to reflect error message change in Mercurial.
Summary:
Note that the original motivation for this test was to verify
`savebackup()` and `restorebackup()` in `eden_dirstate`: D5485950.

As singhsrb recently updated Mercurial to remove a redundant commit when doing
`hg amend` in upstream Mercurial (https://phab.mercurial-scm.org/D636), I
suspect that is responsible for the change in behavior that is necessitates the
change in our test.

We now use a precommit hook failure to trigger the rollback rather than an editor
with a non-zero exit code. As you can see, `transaction abort!\nrollback completed\n`
still appears in the error message, so we are still verifying the behavior of interest.

Differential Revision: D5826751

fbshipit-source-id: bcbf00042c3f26b6e9aa1a980060a0561725a56c
2017-09-19 19:14:43 -07:00
Michael Bolin
1c6403e435 Fix hg grep so it can be run over the entire repo.
Summary:
Although this is not the type of behavior we want to encourage, we should make
it possible. It turns out that this was throwing an exception becuase
`make_glob_list()` was erroneously mapping the pattern to `/**/*` instead of
`**/*` in this case.

Reviewed By: wez

Differential Revision: D5826753

fbshipit-source-id: 659d67c13cdcda39abb7d6893a57ef046804da73
2017-09-13 17:51:19 -07:00
Michael Bolin
250ee8c1eb Fix hg grep so it works when run from a subdirectory.
Summary:
It turns out that we had a small bug with our matcher code that did not account
for pattern normalization. I discovered this while dogfooding Eden and using
`hg grep <pattern> <directory>` from a subdirectory in my working copy. Given
that the fix was to patterns, in general, this likely fixes other `hg` commands
that take a file pattern when used someplace other than the repo root.

Reviewed By: wez

Differential Revision: D5825483

fbshipit-source-id: 0d639cbb2fc678c5459e02e965bf6fc6d7c10959
2017-09-13 12:57:40 -07:00
Michael Bolin
1c695e6dc6 Fixed a bug in how we update the dirstate upon a snapshot change.
Summary:
Previously, we were clearing entries in `hgDirstateTuples` for which:

```
mergeState == NotApplicable
```

but we should have been checking for:

```
mergeState == NotApplicable AND status == Normal
```

The previous logic was causing us to erroneously clear entries in a state like:

```
mergeState == NotApplicable AND status == MarkedForRemoval
```

This bug manifested itself when grafting a change that removed a file.
The file was removed from disk, but Eden did not know that it had been
`MarkedForRemoval`, so it would report the removed file as "missing" in
`hg status`.

Reviewed By: wez

Differential Revision: D5797270

fbshipit-source-id: 29740dfaa8102db868b95e932716773787f317ac
2017-09-08 19:25:34 -07:00
Michael Bolin
595c2684f0 Print out the reason why we take the slow path during a merge update.
Summary:
This should help us audit the source of the slow path when we hit it.

I took a look at `eden/integration/hg/rebase_test.py`, which we know exercises
the slow path. With this change, I manually rebased a short stack of two commits
onto another stack of two commits with the `--debug` flag and saw two instances
of this message:

```
falling back to non-eden update code path: branchmerge is "truthy:" True.
```

so it seems like we should work to update the `branchmerge` case to take the
fast path, when possible.

Reviewed By: simpkins

Differential Revision: D5779633

fbshipit-source-id: a76d72408d6115aa37ae563d3f7165f404fc8332
2017-09-06 21:20:45 -07:00
Michael Bolin
83b3c38095 Fix for hg split in Eden.
Summary:
Before this change, `hg split` crashed complaining that `node` was a
`changectxwrapper` instead of a 20-byte hash when it was sent as `parent1`
of `WorkingDirectoryParents` in `resetParentCommits()`. Now we use `node()` to
get the hash from the `destctx` that we have already extracted via this line
earlier in `merge_update()`:

    destctx = repo[node]

The change to `eden/hg/eden/__init__.py` eliminated the crash, but was
not sufficient on its own to make `hg split` work correctly. There was also a fix
required in `Dirstate.cpp` where the `onSnapshotChanged()` callback was clearing out
entries of both `NotApplicable` and `BothParents` from `hgDirstateTuples`.
It appears that only `NotApplicable` entries should be cleared. (I tried leaving
`NotApplicable` entries in there, but that broke `eden/integration/hg/graft_test.py`.)

I suspected that the logic to clear out `hgDestToSourceCopyMap` in
`Dirstate::onSnapshotChanged` was also wrong, so I deleted it and all of the
integration tests still pass. Admittedly, we are pretty weak in our test coverage
for use cases that write to the `hgDestToSourceCopyMap`. In general, we should
rely on Mercurial to explicitly remove entries from `hgDestToSourceCopyMap`.
We have a Thrift API, `hgClearDirstate()`, that `eden_dirstate` can use to categorically
clear out `hgDirstateTuples` and `hgDestToSourceCopyMap`, if necessary.

Finally, creating a proper integration test for `hg split` required creating a value for
`HGEDITOR` that could write different commit messages for different commits.
To that end, I added a `create_editor_that_writes_commit_messages()` utility as a
method of `HgExtensionTestBase` and updated its `hg()` method to take `hgeditor`
as an optional parameter.

Reviewed By: wez

Differential Revision: D5758236

fbshipit-source-id: 5cb8bf4207d4e802726cd93108fae4a6d48f45ec
2017-09-06 21:20:45 -07:00
Wez Furlong
79ca5bff6e adjust test expectations for fixed typo
Summary: bolinfest fixed the spelling in upstream hg

Reviewed By: simpkins

Differential Revision: D5712622

fbshipit-source-id: 50f98493483a3371bbd26318507b0bb1dcdc1e6d
2017-08-25 21:57:28 -07:00
Wez Furlong
948b584229 ensure that HGUSER doesn't impact the tests
Summary:
I had this set and it broke some of the integration tests.
Force it to be unset before running the tests.

Reviewed By: simpkins

Differential Revision: D5712624

fbshipit-source-id: 7d4aef86ef56f5880180b417e356e8a85abf11d7
2017-08-25 21:57:28 -07:00
Braden Watling
ab43c66a8d Add test to verify that eden debug getpath indicates when inodes are unloaded
Summary:
This test was supposed to be a part of D5627411 but it was causing strange behaviour so was brought to a separate diff for further investigation.

After investigating, the test didn't pass because the UnloadedInodeData struct only contained the name of the file, not the path to it. The fix for this was to implement a way to get the relative path of the file even after the inode is unloaded.

Reviewed By: simpkins

Differential Revision: D5646929

fbshipit-source-id: f166398a651e8aea49da7e4474a5ad7fde2eaa4e
2017-08-25 08:34:31 -07:00
Michael Bolin
c2a541f84b Fix typo in comment.
Summary: (Note: this ignores all push blocking failures!)

Reviewed By: simpkins

Differential Revision: D5699075

fbshipit-source-id: 98ca3b395dd9895fe3d0a43e7daaa6bcedd90689
2017-08-24 14:23:39 -07:00
Michael Bolin
7393494f2e Add integration test for hg graft.
Summary:
Fortunately, this passed on the first try: it did not require any bug fixes in
Eden!

(Note: this ignores all push blocking failures!)

Reviewed By: simpkins

Differential Revision: D5698953

fbshipit-source-id: c5ce39725f8d14b5ea93bd3cafeb5e566f92d326
2017-08-24 14:23:39 -07:00
Michael Bolin
442feff98f Add integration test for hg move.
Summary:
Fortunately, this passed on the first try: it did not require any bug fixes in
Eden! Though admittedly, most of the relevant fixes were presumably done in
D5686114.

(Note: this ignores all push blocking failures!)

Reviewed By: simpkins

Differential Revision: D5696055

fbshipit-source-id: 0099db501ae1a5d72528d222dee0176fc1fc4332
2017-08-24 14:23:38 -07:00
Adam Simpkins
227d851a9b update integration tests to support multiple hg configs
Summary:
Update the integration test framework so that we can run the hg integration
tests with several different hg config settings, using different sets of
mercurial extensions.

This adds code to test using flat manifest, treemanifest in hybrid mode, and
treemanifest in tree only mode.  However, the treeonly configuration is
disabled at the moment due to some bugs in treeonly behavior preventing it from
being able to create test repositories in treeonly mode.

Reviewed By: bolinfest

Differential Revision: D5685880

fbshipit-source-id: 081ead4e77cd14a7feb03381783395bd5a8fef4f
2017-08-23 18:49:33 -07:00
Adam Simpkins
55ea4f78a7 fix rebase test to work with new hg release
Summary:
The most recent mercurial release updated the 'successors()' revset to be the
same as 'allsuccessors()', and it always includes the argument itself in the
output now.  This updates the revset to exclude the input commit.

Reviewed By: bolinfest

Differential Revision: D5694826

fbshipit-source-id: 3e931a39675262f33a5298701b4559e0d9906490
2017-08-23 18:49:33 -07:00
Adam Simpkins
4d5a58444d add a new integration test for hg commit --amend
Summary: Add an test that runs `hg commit --amend`

Reviewed By: bolinfest

Differential Revision: D5685875

fbshipit-source-id: d23f760bf0ab3517d16d3b4658b1128d07577951
2017-08-23 16:53:36 -07:00
Adam Simpkins
9b6759e9d7 update hg.log() behavior in the integration tests
Summary:
This makes some minor tweaks to the behavior of the HgRepository.log() helper
function in the integration tests.

Previously this command did not take a revset argument, and instead relied on
the Facebook tweakdefaults extension to use the `--follow` behavior when no
revset was specified.  (Without tweakdefaults mercurial uses `tip:0` by
default, which is not what the histedit tests expect.)

I added a revset parameter now, and updated it to default to `::.`.  This is
close to the previous behavior, although I intentionally left it reporting
commits from oldest to newest now.

I also updated the log code to add its own delimiter to the template, rather
than requiring callers to always append an escaped nul byte to the template.

Reviewed By: bolinfest

Differential Revision: D5685876

fbshipit-source-id: 01578f62d553be1cd8002b5718d7f12a2f41d4d8
2017-08-23 16:53:36 -07:00
Adam Simpkins
14c865bb88 disable evolve and fsmonitor extensions in integration tests
Summary:
Update the integration tests to no longer enable the evolve or fsmonitor
extensions in the test repositories.

evolve has been deprecated at Facebook for a while now and isn't even shipped
as part of our mercurial installation any more.  This settings was just causing
a warning to be printed that this extension could not be found.

The fsmonitor extension also didn't have any real effect, even in the backing
repository.  We don't create .watchmanconfig files in the test repositories, so
watchman won't watch them.  Therefore fsmonitor simply printed warnings that
watchman wasn't watching this repository.

Reviewed By: bolinfest

Differential Revision: D5685879

fbshipit-source-id: 85b8a725bd17890a93be5c71dd5a0f3f1d744598
2017-08-23 15:06:47 -07:00
Adam Simpkins
8327c003e4 update integration tests to edit the repo .hgrc file
Summary:
Fix the integration tests to store hg config settings in the .hg/hgrc file in
the backing repository.  Previously the tests saved settings to a temporary
file, and then always invoked hg with HGRCPATH pointing at this temporary file.

Unfortunately this resulted in the integration test code using different hg
settings than edenfs, since edenfs was never aware of this temporary file.

Defining the settings in the backing repository's normal .hg/hgrc file means
that edenfs will be able to see these settings as well.  The eden post-clone
hooks will also automatically copy these settings in to the mount point, so
that we do not need to use a custom HGRCPATH setting inside the eden mount
either.

Reviewed By: bolinfest

Differential Revision: D5685877

fbshipit-source-id: 1857554d0cf1a585fe55577eb48a87686f9476ca
2017-08-23 15:06:47 -07:00
Michael Bolin
0508c0b819 Change eden debug getpath to use pwd as the default mount point.
Summary: This seems a little more user-friendly.

Reviewed By: bradenwatling

Differential Revision: D5686562

fbshipit-source-id: 8142fb9105a3a44823f935fc04187cf0ed2258d7
2017-08-23 11:27:22 -07:00
Michael Bolin
303655c4b1 Add integration test for hg copy.
Summary:
Note that this feature was mostly implemented before this commit, but never
tested. Unsurprisingly, there were bugs.

This change also introduces a new `eden debug hg_copy_map_get_all` subcommand
because that was a straightforward way to verify the internal state of the copy
map on the server side from an integration test.

Adding this test uncovered a key copy/paste bug in `EdenThriftClient.py`
(`hgCopyMapGet` was being invoked instead of `hgCopyMapPut`.)

It also uncovered a bug in `LameThriftClient` because the `compile()` and
`eval()` calls on the output are not appropriate when the return type of the
Thrift endpoint is `string`.

Reviewed By: simpkins

Differential Revision: D5686114

fbshipit-source-id: f0093d2b67062c01982dc5bc1f0db2774b3a9356
2017-08-22 21:06:07 -07:00
Jyothsna Konisa
72b61a5ddc Changes to return unloaded inode count for TreeInode::unloadChildrenNow
Summary:
1.Modified `TreeInode::unloadChildrenNow()` to return number of inodes that have been unloaded.
2.Modified `EdenServiceHandler::unloadInodeForPath()` to return number of inodes that are unloaded.

Reviewed By: simpkins

Differential Revision: D5627539

fbshipit-source-id: 4cdb0433dced6bf101158b9e6f8c35de67d9abbe
2017-08-22 19:50:00 -07:00
Jyothsna Konisa
371cfa097d TestCase to verify unloadChildrenNow with age
Summary:
Added a test case `test_unload_free_inodes_age` to verify the behaviour of unloadChildrenNow with age parameter.
Added new parameter age to `unloadInodeForPath` in eden.thrift, and `EdenServiceHandler`.
Modified `do_unload_inodes` function in `debug.py` to support the new behaviour.

Reviewed By: simpkins

Differential Revision: D5565859

fbshipit-source-id: a35053725be26bc906cf158969cbe21db1cbadde
2017-08-22 19:50:00 -07:00
Michael Bolin
f946ae76f9 Wrap exact() from Mercurial's match.py.
Summary:
In `hg/eden/__init__.py`, we wrap `match()` in Mercurial's `match.py` in an
attempt to annotate every `basematcher` created in the system with a special
`_eden_match_info` property that we can use in `_eden_walk_helper()` to perform
walks more efficiently. Unfortunately, we missed a case where `scmutil.py`
has a `matchfiles()` function that calls `exact()` in `match.py` directly rather
than going through the generic `match()` function.

This was causing a failure when running `hg revert <filename>` in Eden because
the matcher that was created via `exact()` did not have an `_eden_match_info`.
This commit wraps `exact()` to add the property.

Reviewed By: wez

Differential Revision: D5674660

fbshipit-source-id: 16d1e7648ebd7a23b43b9b1200d3e284e5bc07b0
2017-08-21 18:05:03 -07:00
Simon Jensen
3a0e66677b Cache invalidation thrift api
Summary: Provide a thrift interface to invalidate the cache for an inode denoted by path.

Reviewed By: simpkins

Differential Revision: D5655387

fbshipit-source-id: 887aa4963d216a0d8eed93b6fb8721632cc31d19
2017-08-21 16:05:44 -07:00
Michael Bolin
70050affcc Introduce eden debug hg_get_dirstate_tuple.
Summary:
This is a convenient way to test the `hgGetDirstateTuple()` endpoint in
`eden.thrift`.

Reviewed By: quark-zju, wez

Differential Revision: D5654237

fbshipit-source-id: 0b285e056002d4556733a53293582345f36780b2
2017-08-18 21:49:59 -07:00
Braden Watling
cf297e0106 Add subcommand getpath to eden debug
Summary: Add a command to lookup the path for an inode given the inode number and eden mount path.

Reviewed By: bolinfest

Differential Revision: D5627411

fbshipit-source-id: 25928f506d3f48d8a6784fe81fb17fa0500d6bc9
2017-08-16 20:04:30 -07:00
Jyothsna Konisa
916c129655 setting TimeStamps for TreeInode
Summary:
Updated time stamps of TreeInode accurately on mkdir,rmdir,mknode,symlink,create,unlink and readdir.
updated the `TreeInode::getattr` function to return in-memory timestamps.

Reviewed By: simpkins

Differential Revision: D5568183

fbshipit-source-id: c36f7fb767cd4342aab5cc983eea56e37cd2077e
2017-08-14 23:23:23 -07:00
Jyothsna Konisa
8bcd0f234c Setting TimeStamps for FileInode
Summary:
updating atime,ctime,mtime of FileInode on read, write and setattr system calls.
modified `FileInode::stat` function to return accurate inmemory timestamps.

Reviewed By: simpkins

Differential Revision: D5552666

fbshipit-source-id: 86d446f72908663f8db509b7b789d9f35d17df3a
2017-08-14 23:23:23 -07:00
Wez Furlong
c08890f849 do a better job at reporting "new" in watchman results.
Summary:
We're seeing that this is always set to true for eden,
which is causing buck to run slower than it should.

To make this work correctly, I've augmented our journal data structure
so that it can track create, change and remove events for the various
paths.

I've also plumbed rename events into the journal.

This requires a slightly more complex merge routine, so I've refactored the two
call sites that were merging in slightly different contexts so that they can
now share the same guts of the merge routine.  Perhaps slightly
counterintuitive in the merge code is that we merge a record from the past into
the state for now and this is a bit backwards compared to how people think.

I've expanded the eden integration test to check that we don't mix up
create/change/removes for the same path in a given window.

On the watchman side, we use the presence of the filename in the createdPaths
set as a hint that the file is new.  In that case we will set the watchman
`ctime` (which is not the state ctime but is really the *created clock time*)
to match the current journal position if the file is new, or leave it set
to 0 if the file is not known to be new.  This will cause the `is_new`
flag to be set appropriately by the code in `watchman/query/eval.cpp`;
if the sequence is 0 then it should never be set to true.  Otherwise (when
the file was in the `createPaths` set) it will be set to the current journal
position and this will be seen as newer than the `since` constraint on
the query and cause the file to show as `new`.

Reviewed By: bolinfest

Differential Revision: D5608538

fbshipit-source-id: 8d78f7da05e5e53110108aca220c3a97794f8cc2
2017-08-11 12:57:37 -07:00
Michael Bolin
c8faca5285 Updated RebaseTest to verify that Eden's fast path was exercised.
Summary:
We verify this behavior by adding the appropriate debug statements to our Hg
extension, running `hg update` with the `--debug` flag, and verifying the
expected log statements appear in stdout.

Reviewed By: simpkins

Differential Revision: D5505478

fbshipit-source-id: 389b69c7c1de4fa88fd07fb9eb4dd7e3370e766b
2017-07-27 17:24:01 -07:00