Commit Graph

22 Commits

Author SHA1 Message Date
Jun Wu
fd5cb6e225 bindings: allow Python threads during indexedlog repair
Summary: This should allow us to show a spinning progress bar when repair() is working.

Reviewed By: xavierd

Differential Revision: D19056949

fbshipit-source-id: 8e058bad002bb2de29dbf8457bead0ebd1e99372
2019-12-20 16:14:18 -08:00
Stefan Filip
ef220f3f7c manifest: add the list method to the Manifest trait
Summary:
The list functionality is required by EdenFs. We want this functionality
to be well supported by the Manifest.

Reviewed By: quark-zju

Differential Revision: D18870143

fbshipit-source-id: 1ebaa713ff521226e6ace22cbd35cc841d967298
2019-12-10 10:18:19 -08:00
Stefan Filip
67b0e3a600 manifest: rename FsNode to FsNodeMetadata
Summary:
Consistency in naming. The general idea is to have Metadata types that don't
contain paths. Then we will have File, Directory and eventually FsNode that
will contain paths.

Reviewed By: quark-zju

Differential Revision: D18870141

fbshipit-source-id: a1f09add7f1c3dd4fa0348693cd3ce2fd5767fa7
2019-12-10 10:18:18 -08:00
Stefan Filip
05aa630a18 manifest-tree: rename Tree to TreeManifest
Summary:
This rename is going to make it easier to import and use outside of
manifest specific crates.

Reviewed By: quark-zju

Differential Revision: D18870142

fbshipit-source-id: 2f3ea460170308162ee834efc038b2dcedd9e233
2019-12-10 10:18:18 -08:00
Jun Wu
d754747d01 bindings: add repair to structures
Summary: Expose the repair API to the Python world.

Reviewed By: xavierd

Differential Revision: D18737913

fbshipit-source-id: c31085727589b6938c2fafb28897925aea617bc4
2019-12-09 14:15:47 -08:00
Jun Wu
7a16996ce0 revisionstore: implement indexedlog::DefaultOpenOptions
Summary:
This simplifies the logic a bit by getting a free `repair` method from
indexedlog.

Reviewed By: xavierd

Differential Revision: D18737908

fbshipit-source-id: 4988c1a83b7709b751cd1899c5663acc0c42e313
2019-12-06 19:35:04 -08:00
Stefan Filip
dd4d4a7b85 manifest: split in two crates: manifest and manifest-tree
Summary:
This isolates the core types in manifest so that it is harder to
create unsound dependencies with specific implementations.

Reviewed By: quark-zju

Differential Revision: D18843133

fbshipit-source-id: 4b866ad84d2e7d0ff2dc4ec6bd65f66548c3fe4a
2019-12-06 14:33:02 -08:00
Jun Wu
3e205e29b3 pydag: revise bindings
Summary:
The bindings APIs are out of date.

The latest idea is to write non-master commits to disk, in a different group.
There are no "segments in memory". Therefore:
- Rename `builddisk` to `build`.
- Make sure `build` updates internal states by calling `reload`.
- Drop `buildmem`.
- In case `builddisk` gets called very frequently, reduce its overhead by
  doing nothing if all commits are built already.

The constructor is revised to match the latest interface (no
"max segment level" and "set_new_segment_size" needs to be called).

Reviewed By: sfilipco

Differential Revision: D18640897

fbshipit-source-id: 76a5d0db363713dc5b763503c238c1bfa61ea02e
2019-12-05 17:01:11 -08:00
Jun Wu
75aff36c7f dag: add group support for IdMap
Summary:
This allows us to allocate Ids for non-master commits in a separate range
so they wouldn't cause fragmentation for commits in the master group.

Reviewed By: sfilipco

Differential Revision: D18640896

fbshipit-source-id: c19c53342583ded8795849584cf5f31bc86b2818
2019-12-04 22:43:23 -08:00
Jun Wu
05157a174a dag: replace u64 with the new Id type
Summary: Migrate the codebase to use the new Id type instead of an alias of u64.

Reviewed By: sfilipco

Differential Revision: D18640898

fbshipit-source-id: 415db9afeda1f8960e18c69e70f53a9eeb63cbbe
2019-12-04 22:43:23 -08:00
Stefan Filip
1f76f9e63f manifest: restructure InsertError to give back arguments
Summary:
It would be nice if the insert method would give back the path that
it failed to insert.

Reviewed By: dtolnay

Differential Revision: D18739978

fbshipit-source-id: 4c16d09750ade2f01397161129c31bcf0059a957
2019-12-02 15:53:41 -08:00
Stefan Filip
30045a27ba manifest: update python bindings to allow insertions before deletions
Summary:
The Mercurial code has the bad habbit of inserting files in what can
appear impossible locations. Sometimes files with directory names and
sometimes directories in file locations. This happens because the initial
code would do additions before deletions on Manifest implementations that
were rudimentary. As soon as we introduce validation various code paths
surface.

I tried to fix the codepaths that modify manifests but it's a losing game.
I fixed the issue that appeared in tests and a couple of issues people
reported but more situations crop up.

This is giving up on the python code.

Reviewed By: quark-zju

Differential Revision: D18737678

fbshipit-source-id: 0c97128ff67e5ba2334942b6afc404aa64a2e5f4
2019-12-02 15:53:41 -08:00
Jun Wu
353361701c metalog: add is_dirty API
Summary:
This helps test if there are uncommitted changes. It can help detecting
programming errors. For example, is metalog changed before starting a
transaction?

Reviewed By: xavierd

Differential Revision: D18538148

fbshipit-source-id: 9c82838d8174e6a93fde7108503a025660191cbf
2019-11-27 00:24:09 -08:00
David Tolnay
d1d8fb939a Switch from failure::Fail trait to std::error::Error for errors
Summary:
This diff replaces eden's dependencies on failure::Error with anyhow::Error.

Failure's error type requires all errors to have an implementation of failure's own failure::Fail trait in order for cause chains and backtraces to work. The necessary methods for this functionality have made their way into the standard library error trait, so modern error libraries build directly on std::error::Error rather than something like failure::Fail. Once we are no longer tied to failure 0.1's Fail trait, different parts of the codebase will be free to use any std::error::Error-based libraries they like while still working nicely together.

Reviewed By: xavierd

Differential Revision: D18576093

fbshipit-source-id: e2d862b659450f2969520d9b74877913fabb2e5d
2019-11-22 08:53:31 -08:00
Jun Wu
f19a760f5f blackbox: remove IndexFilter
Summary: Migrate callsites to use the new pattern matching filtering APIs.

Differential Revision: D18610738

fbshipit-source-id: 242182f4e6a58bbff5dddbb177287921c85519bc
2019-11-21 08:36:09 -08:00
Zeyi (Rice) Fan
7f0f541fa1 manifest: add FsNode to List::Directory
Reviewed By: sfilipco

Differential Revision: D18307923

fbshipit-source-id: 1844afb7b9a185f7e89c11476ee2eb34cac117c6
2019-11-19 18:01:22 -08:00
Zeyi (Rice) Fan
5b274b1a4c manifest: add tree HgId to FsNode
Summary: This diff adds `Option<HgId>` to `FsNode::Directory` because EdenFS needs the tree hash for the directories in the tree.

Reviewed By: sfilipco

Differential Revision: D18307205

fbshipit-source-id: 93db944ff1686a1c5927aa9a876a0fd008edbf9a
2019-11-19 18:01:22 -08:00
David Tolnay
066f461bcf Replace reliance on From<Context<_>> with an extension trait and constructor
Summary:
Edenapi uses a group of impls like the following as the canonical way to construct ApiError.

```
impl From<Context<ApiErrorKind>> for ApiError {
```

Downstream code would write:

```
use failure::ResultExt;

let stream_trees = config
    .get_or_default("edenapi", "streamtrees")
    .context(ApiErrorKind::BadConfig("edenapi.streamtrees"))?;
```

This relies on the way that ResultExt::context returns a unique type that is different from failure::Error.

This diff introduces a dedicated extension trait to allow the same code in the caller to continue to work even without a failure::Context<T> type, which will be required for dropping our dependency on failure::Fail and failure.

```
pub trait ApiErrorContext<T> {
    fn context(self, kind: ApiErrorKind) -> ApiResult<T>;
}
```

We also introduce a public constructor for ApiError and replace indirect construction that used to look like this:

```
error.context(ApiErrorKind::Curl).into()
```

with this instead:

```
ApiError::new(ApiErrorKind::Curl, error)
```

which is the same number of characters but clearer. The argument order matches [std::io::Error::new](https://doc.rust-lang.org/std/io/struct.Error.html#method.new).

Reviewed By: kulshrax

Differential Revision: D18574668

fbshipit-source-id: 0a56297bb942a26d75a62ca39fc16abeb4486345
2019-11-18 12:12:43 -08:00
Jun Wu
9561b5e22a treematcher: do not crash on non-utf-8 paths
Summary:
On Windows, the paths passed to the `match` functions are not encoded in utf-8
yet.  Do not crash if received them.

Reviewed By: sfilipco

Differential Revision: D18574951

fbshipit-source-id: c22ce95cb6538db0c15d60789286e2ff7d2148bc
2019-11-18 11:44:53 -08:00
David Tolnay
b1793a4416 rust: Rename Fallible<T> to Result<T>
Summary:
This diff is preparation for migrating off of failure::Fail / failure::Error for errors in favor of errors that implement std::error::Error. The Fallible terminology is unique to failure and in non-failure code we should be using Result<T>. To minimize the size of the eventual diff that removes failure, this codemod replaces all use of Fallible with Result by:

- In modules that do not use Result<T, E>, we import `failure::Fallible as Result`;
- In modules that use a mix of Result<T, E> and Fallible<T> (only 5) we define `type Result<T, E = failure::Error> = std::result::Result<T, E>` to allow both Result<T> and Result<T, E> to work simultaneously.

Reviewed By: Imxset21

Differential Revision: D18499758

fbshipit-source-id: 9f5a54c47f81fdeedbc6003cef42a1194eee55bf
2019-11-14 14:11:01 -08:00
Jun Wu
63c1b3001d treematcher: implement proper error handling
Summary:
simpkins encountered a case where the treematcher does not work with many
patterns.  It turns out `globset` has a hard-coded regex size limit (10MB).

Implement proper error handling so we can detect such issues and fallback to
slower paths.

Reviewed By: sfilipco

Differential Revision: D18500299

fbshipit-source-id: 0122ba9b0246c1536b2069a40e13261ee47f8bba
2019-11-14 11:43:18 -08:00
Adam Simpkins
ab3a7cb21f Move fb-mercurial sources into an eden/scm subdirectory.
Summary:
In preparation for merging fb-mercurial sources to the Eden repository,
move everything from the top-level directory into an `eden/scm`
subdirectory.
2019-11-13 16:04:48 -08:00