Commit Graph

18 Commits

Author SHA1 Message Date
Jun Wu
3d65779318 dag: add IdMapLike for minimal IdMap interface
Summary:
In the future we might have revlog or more complicated storage for maintaining
the IdMap. Define a trait for it.

Reviewed By: sfilipco

Differential Revision: D18664208

fbshipit-source-id: 242611d78852e8d82a52ab52192db87c13b848e7
2019-12-05 17:01:13 -08:00
Jun Wu
dbfc96ca15 dag: add to_first_ancestor_nth
Summary: This is the reverse of first_ancestor_nth. It will be used later.

Reviewed By: sfilipco

Differential Revision: D18664209

fbshipit-source-id: 8d3706cdc080086ec60da60870fde76c8933f6a8
2019-12-05 17:01:13 -08:00
Jun Wu
74e624297c dag: add first_ancestor_nth
Summary: This will be used in an upcoming change.

Reviewed By: sfilipco

Differential Revision: D18664207

fbshipit-source-id: 3030bb02e172b95c6dbabda74c318e7f16212f59
2019-12-05 17:01:12 -08:00
Jun Wu
f028832a62 dag: add a fast path to ancestors calculation
Summary:
With the ONLY_HEAD flag, ancestors can be sped up sometimes.  Since ancestors
is used widely in other DAG calculations, perf improvements are visible across
the benchmark. The trade-off is to spend a bit more time on building segments
to figure out ONLY_HEAD.

After this diff:

  building segments                                 576.625 ms
  ancestors                                          47.119 ms
  children                                          436.967 ms
  common_ancestors (spans)                          257.096 ms
  descendants (small subset)                        367.316 ms
  gca_one (2 ids)                                   146.315 ms
  gca_one (spans)                                   253.059 ms
  gca_all (2 ids)                                   236.075 ms
  gca_all (spans)                                   266.695 ms
  heads                                             220.766 ms
  heads_ancestors                                    36.258 ms
  is_ancestor                                        98.967 ms
  parents                                           219.173 ms
  parent_ids                                          9.282 ms
  range (2 ids)                                      99.859 ms
  range (spans)                                     205.154 ms
  roots                                             429.047 ms

Before this and previous diff:

  building segments                                 542.564 ms
  ancestors                                         316.075 ms
  children                                          428.612 ms
  common_ancestors (spans)                          452.850 ms
  descendants (small subset)                        537.483 ms
  gca_one (2 ids)                                   526.112 ms
  gca_one (spans)                                   446.726 ms
  gca_all (2 ids)                                   780.590 ms
  gca_all (spans)                                   490.906 ms
  heads                                             214.005 ms
  heads_ancestors                                   217.558 ms
  is_ancestor                                       256.769 ms
  parents                                           213.170 ms
  parent_ids                                          9.313 ms
  range (2 ids)                                     259.435 ms
  range (spans)                                     358.934 ms
  roots                                             423.005 ms

Reviewed By: sfilipco

Differential Revision: D18642797

fbshipit-source-id: 27f1ee5797a92142b2c19195eb26baef58769553
2019-12-05 17:01:12 -08:00
Jun Wu
94a09a890d dag: add a flag to indicate a segment is the only head so far
Summary: The pre-calculated flag can be used for some fast paths.

Reviewed By: sfilipco

Differential Revision: D18642796

fbshipit-source-id: bc16b3517cc29e23478f2027e83e2a2282be0d0a
2019-12-05 17:01:12 -08:00
Jun Wu
3e205e29b3 pydag: revise bindings
Summary:
The bindings APIs are out of date.

The latest idea is to write non-master commits to disk, in a different group.
There are no "segments in memory". Therefore:
- Rename `builddisk` to `build`.
- Make sure `build` updates internal states by calling `reload`.
- Drop `buildmem`.
- In case `builddisk` gets called very frequently, reduce its overhead by
  doing nothing if all commits are built already.

The constructor is revised to match the latest interface (no
"max segment level" and "set_new_segment_size" needs to be called).

Reviewed By: sfilipco

Differential Revision: D18640897

fbshipit-source-id: 76a5d0db363713dc5b763503c238c1bfa61ea02e
2019-12-05 17:01:11 -08:00
Jun Wu
744f17df8b dag: add group support for segments
Summary: Make it possible to build non-master segments.

Reviewed By: sfilipco

Differential Revision: D18641507

fbshipit-source-id: 4d0c46cdea240849af37d28219b6023578514323
2019-12-05 17:01:11 -08:00
Jun Wu
cdcbf836eb dag: fix a correctness issue that flat segment building might be incomplete
Summary:
`build_segments_volatile(id)` should build flat segments so it includes `id`.

Right now it can fail that in a corner case. Fix it.

Reviewed By: sfilipco

Differential Revision: D18641509

fbshipit-source-id: ad1de694e4d31c51f29e9bc5800291066bf16dbb
2019-12-05 17:01:11 -08:00
Jun Wu
5bac1b4598 dag: re-assign non-master ids to master ids
Summary:
This is needed to keep Ids topologically sorted.

This is reflected in the test added by D18641507, when assigning
master ids to `I`:

```
0---1---2---3---4---5---6--------H---I
     \               \          /
      N0-N1-N2-N3     N4-N5-N6-o
                \            \
                 -------------N7--N8
Lv0: R0-1[] 2-3[1] 4-6[3] N0-N1[1] N2-N3[N1] N4-N6[5] N7-N8[N3, N6]
Lv1: R0-3[] 4-6[3] N0-N1[1] N2-N3[N1] N4-N8[5, N3]
Lv2: R0-6[] N0-N3[1] N4-N8[5, N3]
```

Note: 0, 1, 2, ... are ids in the master group. N0, N1, ... are ids in the non-master group.

```
0---1---2---3---4---5---6--------11--12
     \               \          /
      N0-N1-N2-N3     7--8--9--10
                \            \
                 -------------N7--N8
Lv0: R0-1[] 2-3[1] 4-6[3] 7-10[5] 11-12[6, 10] N0-N1[1] N2-N3[N1] N4-N6[5] N7-N8[N3, N6]
Lv1: R0-3[] 4-6[3] 7-12[5, 6] N0-N1[1] N2-N3[N1] N4-N8[5, N3]
Lv2: R0-6[] 7-12[5, 6] N0-N3[1] N4-N8[5, N3]
Lv3: R0-12[] N0-N8[1, 5]
```

Notice that 7, 8, 9 replace the old non-master ids N4, N5, N6. Without this change,
N4, N5, N6 will be kept and break the topo-sorted property.

An `ensure!` was added to make sure Ids are topo-sorted.

Reviewed By: sfilipco

Differential Revision: D18641510

fbshipit-source-id: bbe642df3615c65db7f2e7c889624e00e53ef7ba
2019-12-05 17:01:10 -08:00
Jun Wu
fb718fe106 dag: allow re-assigning a non-master Id to a master Id
Summary:
The IdMap has assertions about 1:1 mapping between Id and Slice.

Practically, a non-master commit can be merged into a master commit.
That means a commit hash might map to 2 Ids: [master-id, non-master-id].
Allow that to happen (temporarily).

Note: The non 1:1 mapping causes more headache than benefit and
D18838995 restores 1:1 mapping. But it still requires temporary
non-1:1 mapping to work.

Reviewed By: sfilipco

Differential Revision: D18641508

fbshipit-source-id: bc4bc831d5ab5ebfc2588bafd48114ae49d31e0f
2019-12-05 17:01:10 -08:00
Jun Wu
81b7593249 dag: avoid panic on drop
Summary: panic on drop is a terrible idea. It can cause SIGILL kill without meaningful messages or meaningful backtrace. Remove it.

Reviewed By: xavierd

Differential Revision: D18640894

fbshipit-source-id: 3ab52ded9875e17e891acd8f9b45d82964615f01
2019-12-04 22:43:24 -08:00
Jun Wu
de9a7fecb1 dag: replace some panic! with bail!
Summary: This makes the code harder to crash in abnormal cases.

Reviewed By: sfilipco

Differential Revision: D18640895

fbshipit-source-id: ee81dd7739739d3d15052bdb7bb2ba80ed48cd0a
2019-12-04 22:43:24 -08:00
Jun Wu
75aff36c7f dag: add group support for IdMap
Summary:
This allows us to allocate Ids for non-master commits in a separate range
so they wouldn't cause fragmentation for commits in the master group.

Reviewed By: sfilipco

Differential Revision: D18640896

fbshipit-source-id: c19c53342583ded8795849584cf5f31bc86b2818
2019-12-04 22:43:23 -08:00
Jun Wu
05157a174a dag: replace u64 with the new Id type
Summary: Migrate the codebase to use the new Id type instead of an alias of u64.

Reviewed By: sfilipco

Differential Revision: D18640898

fbshipit-source-id: 415db9afeda1f8960e18c69e70f53a9eeb63cbbe
2019-12-04 22:43:23 -08:00
Jun Wu
c016535cb6 dag: add a dedicated Id type
Summary:
Background:
- Tests and Nuclide haven't removed usage of revision numbers.
- If we build segments for non-master commits on demand, there are performance
  concerns about very long release branches.
- It's very desirable to keep `ancestors(master)` continuous. In other words,
  `ancestors(master)` is a single integer span. This affects performance for
  DAG algorithms a lot.

To solve the problems, I'm going to use separate ranges for master and
non-master commit Ids (so assigning Ids to non-master commits wouldn't affect
what numbers can be used for the master group). To make the code easier to
read, I'm changing `type Id = u64` to `struct Id` where the struct is
group-aware.

This diff introduces related types and basic operations.

Reviewed By: sfilipco

Differential Revision: D18640899

fbshipit-source-id: 8f7ecbee6e0d1703d1225f18e92742981dbf3757
2019-12-04 22:43:22 -08:00
David Tolnay
d1d8fb939a Switch from failure::Fail trait to std::error::Error for errors
Summary:
This diff replaces eden's dependencies on failure::Error with anyhow::Error.

Failure's error type requires all errors to have an implementation of failure's own failure::Fail trait in order for cause chains and backtraces to work. The necessary methods for this functionality have made their way into the standard library error trait, so modern error libraries build directly on std::error::Error rather than something like failure::Fail. Once we are no longer tied to failure 0.1's Fail trait, different parts of the codebase will be free to use any std::error::Error-based libraries they like while still working nicely together.

Reviewed By: xavierd

Differential Revision: D18576093

fbshipit-source-id: e2d862b659450f2969520d9b74877913fabb2e5d
2019-11-22 08:53:31 -08:00
David Tolnay
b1793a4416 rust: Rename Fallible<T> to Result<T>
Summary:
This diff is preparation for migrating off of failure::Fail / failure::Error for errors in favor of errors that implement std::error::Error. The Fallible terminology is unique to failure and in non-failure code we should be using Result<T>. To minimize the size of the eventual diff that removes failure, this codemod replaces all use of Fallible with Result by:

- In modules that do not use Result<T, E>, we import `failure::Fallible as Result`;
- In modules that use a mix of Result<T, E> and Fallible<T> (only 5) we define `type Result<T, E = failure::Error> = std::result::Result<T, E>` to allow both Result<T> and Result<T, E> to work simultaneously.

Reviewed By: Imxset21

Differential Revision: D18499758

fbshipit-source-id: 9f5a54c47f81fdeedbc6003cef42a1194eee55bf
2019-11-14 14:11:01 -08:00
Adam Simpkins
ab3a7cb21f Move fb-mercurial sources into an eden/scm subdirectory.
Summary:
In preparation for merging fb-mercurial sources to the Eden repository,
move everything from the top-level directory into an `eden/scm`
subdirectory.
2019-11-13 16:04:48 -08:00