Commit Graph

51339 Commits

Author SHA1 Message Date
Stefan Filip
395bfba6dd manifest: move DirLink::hgid from member variable to member function
Summary:
`hgid` can be easily derived from the link member variable. Using link is
preferable because link is the owner of the data. It makes the logic more
specific, less bug prone to use the true source of the data.

Reviewed By: quark-zju

Differential Revision: D18843137

fbshipit-source-id: 443f1e5eb4498fa4f2ff9904dba7cc3dc4305add
2019-12-06 14:33:01 -08:00
Stefan Filip
e528ab68b0 manifest: remove crate visibility modifiers for DirLink
Summary:
This structure is not public. It is private to the manifest crate so there is
no point to have those modifiers.

Reviewed By: quark-zju

Differential Revision: D18843134

fbshipit-source-id: 0794bbbf578863cabbd85ada87669b791ebcc72d
2019-12-06 14:33:00 -08:00
Stefan Filip
ac6490550a manifest: rename Directory<'a> to DirLink
Summary:
The purpose of this change is to disambiguate from the other Directory struct
that we have in the Manifest. Right now that other is in manifest/lib.rs

Reviewed By: quark-zju

Differential Revision: D18843130

fbshipit-source-id: f25a60f3040cc19abcab99ac5f2cdbff83ccf7ee
2019-12-06 14:33:00 -08:00
Stefan Filip
d284c31d5a manifest: move Directory with link to link.rs
Summary:
Directory<'a> which wraps a link is closely linked with the Link abstraction.
It can be seem as a convenience wrapper around Link.

Reviewed By: quark-zju

Differential Revision: D18843129

fbshipit-source-id: 2b677d71101bff3708a65e03bbcddf5074d51387
2019-12-06 14:33:00 -08:00
Stefan Filip
d274e96880 manifest: add the diff method to the Manifest trait
Summary:
Diff is a core operation for manifests. The Manifest trait should require that
it is implemented.

Reviewed By: quark-zju

Differential Revision: D18843132

fbshipit-source-id: 5e3febab669470e266878d934561dfc323b71c7f
2019-12-06 14:32:59 -08:00
Stefan Filip
46c3c1ca3e manifest: force debugmanifestdirs to return / for repo root
Summary:
A previous change to the BFS iterator fixed it to return the root of the repository.
This influenced the `debugmanifestdirs` command which was manually
handling the root of the repository. This diff updates the command
to match the old behavior. Basically after the fix to BFS we are
not printing any characters for the path of the root. Forcing the debug command
to print of a '/'.

(Note: this ignores all push blocking failures!)

Reviewed By: quark-zju

Differential Revision: D18862167

fbshipit-source-id: 9ca6da707ae8ab037e88a2bbce3b3993b5d219e3
2019-12-06 14:13:33 -08:00
Stefan Filip
00546c00dd fsmonitor: add option to allow disabling fallback on exception
Summary:
In most cases nowadays people run into worse experiences when their operations
run through the fsmonitor path. Watchman is reliable enough that we can rely
on it to work better than the default crawling algorithms.

Reviewed By: quark-zju

Differential Revision: D18835322

fbshipit-source-id: 30873c7d4b1ec306e59b6152c20833cc0027cad5
2019-12-06 13:55:01 -08:00
svcscm
8ed504df23 Updating submodules
Summary:
GitHub commits:

08c2d412ad

Reviewed By: yns88

fbshipit-source-id: 6756fe078b4e8ada8bfb86102f6ec3f85a04eb90
2019-12-06 13:43:23 -08:00
Stefan Filip
713dea745c tests: add test-rustmanifest-t
Summary:
This is a test that aims to validate that commit manifest match between the C++
and Rust implementations.

Reviewed By: quark-zju

Differential Revision: D18521823

fbshipit-source-id: 678b82239e9be7ad078f1e04a6467e87adaac966
2019-12-06 13:36:42 -08:00
svcscm
3fca519f6c Updating submodules
Summary:
GitHub commits:

156d8a13af
abe187bac6
c8fbe63680
93d055f5ed
0d623bc3a9
dffa6cbfdd
20001aaf8a
00af1f1e50

Reviewed By: yns88

fbshipit-source-id: 830f8e651beff51d0de768c4dca6dfc2a235f2f5
2019-12-06 13:36:42 -08:00
svcscm
7c3a5a330a Updating submodules
Summary:
GitHub commits:

1cab1488c4
3130f8ecbd
6db57bc37f
5b6b860f8b

Reviewed By: yns88

fbshipit-source-id: f9054a178adf91a1ff9e8d619b617479ecb5fa85
2019-12-06 12:10:35 -08:00
Adam Simpkins
2799e45345 fix the dependencies for the Rust vendored crates
Summary:
The Rust vendored crates script also depends on another script and the
lfs-pointers file which specifies the ID to use for the download.
Update the RustStaticLibrary.cmake file to make sure that the vendored crates
are re-downloaded if either of these files change.

Reviewed By: wez

Differential Revision: D18846757

fbshipit-source-id: dd67d8d954a048501f0bdaddbd78147d39a1da5f
2019-12-06 11:59:47 -08:00
Adam Simpkins
32e625852c write build output to a log file
Summary:
Update the getdeps builders to save all command output to a log file in the
build directory.  This makes it easier to go back after the fact and look at
the config options selected by the build run.

Reviewed By: wez

Differential Revision: D18777059

fbshipit-source-id: 86d3ffa48ec992fe90c59a3835e18f08310ed88c
2019-12-06 11:59:47 -08:00
Stefan Filip
b32aecb4cf manifest: remove File and Directory creation From tuple
Summary: The conversion from tuple is not used much. I don't like them that much either.

Reviewed By: quark-zju

Differential Revision: D18843131

fbshipit-source-id: fb06b72a279b59ee8af5e43d83a10e39a576730b
2019-12-06 11:50:16 -08:00
Stefan Filip
3459d764fe manifest: add files to the Manifest trait
Summary:
Querying all the files in a Manifest is core functionality that every Manifest
implementation should implement. The main question is how do we describe that.
We want to return an interator over the files of the manifest. The problem is
that the manifest is a trait and functions on traits can't return traits.
The solution is to Box the results. Not pretty but allows us to move the code
forward.

Reviewed By: quark-zju

Differential Revision: D18822854

fbshipit-source-id: ca44b24816203049f747ad67af48e33370861024
2019-12-06 11:50:16 -08:00
Stefan Filip
37d697cd08 manifest: add more detail to manifest entry materialization
Summary: Better error handling

Reviewed By: quark-zju

Differential Revision: D18822857

fbshipit-source-id: 230f8bc52655c1ef1a72394f50c8d88fcbcf0920
2019-12-06 11:50:15 -08:00
Stefan Filip
c5e4b8b045 manifest: rename get_link to materialize_links
Summary:
The purpose of the rename is to signal that this function may potentially do
a lot of work, even calling and waiting on network requests.

Reviewed By: quark-zju

Differential Revision: D18822855

fbshipit-source-id: 98471944c8e6c7cffa7a2a11f2f733274802b376
2019-12-06 11:50:15 -08:00
Stefan Filip
4074083986 manifest: add Link::to_file
Summary:
Removing the contructor for File that takes dependency on
tree implementation details.

Reviewed By: kulshrax

Differential Revision: D18822858

fbshipit-source-id: aed33b146dfdbdb23721a0f6e96977025d71a627
2019-12-06 11:50:15 -08:00
Stefan Filip
b21d605d74 manifest: move file.rs contents into lib.rs
Summary: The types in file.rs being in a separate file confuse people.

Reviewed By: quark-zju

Differential Revision: D18822856

fbshipit-source-id: 29aa10c76bb5ef74f1fc0bf726e6b3cfaf19b428
2019-12-06 11:50:14 -08:00
Jun Wu
3f9123d94f pushrebase: also update remote names
Summary:
For now pushrebase updates working parent, bookmarks. This diff makes it also
update remote bookmarks.

This resolves a race condtion in remotenames where the pulled remote bookmarks
have unknown nodes and cause update to silently fail.

Reviewed By: sfilipco

Differential Revision: D18601035

fbshipit-source-id: 6f2c318cbf6b29a6427eeac6d374c1fb13e6155c
2019-12-06 11:39:01 -08:00
Jun Wu
ae2be4c73f test-pushrebase-remotenames: modernize the test
Summary:
Enable visibility and mutation to get modern features. Remove legacy configs.
As we're here, add more `log -G` outputs to make the test easier to reason about.

Reviewed By: xavierd

Differential Revision: D18601036

fbshipit-source-id: f0d656c696708c045c63f5b8e59b9a40f2dbbf20
2019-12-06 11:39:01 -08:00
Stefan Filip
6624293ca3 grep: remove xargs fanout from grep command implementation
Summary:
Move it to configuration.
This gives better control over how grep is executed. Previously the assumption
was that xargs was present in PATH. This updates removes that assumption.

Reviewed By: quark-zju

Differential Revision: D18715471

fbshipit-source-id: 09c3098984588cd7478172db0bcfb8d5cb8243fd
2019-12-06 11:16:28 -08:00
Jun Wu
54892cb391 dagop: fix doc about reachableroots
Summary:
`reachableroots(roots, heads, includepath=False)` returns a subset of `roots`.

I implemented `reachableroots` using the documented revset using segmented
changelog APIs and noticed smartlog graph is not correct. This diff fixes
the documentation.

This diff was also sent upstream as https://phab.mercurial-scm.org/D7518.

Reviewed By: ikostia

Differential Revision: D18723363

fbshipit-source-id: ed56ae85345dcf102159c4df0eb19c02cef2bf27
2019-12-06 11:02:32 -08:00
svcscm
52e1d8378a Updating submodules
Summary:
GitHub commits:

4dd233987b
7ab406809d

Reviewed By: yns88

fbshipit-source-id: 097aac010cbee9a323e3ed3e16f5e08976a3a553
2019-12-06 11:02:32 -08:00
svcscm
73c8fea4c6 Updating submodules
Summary:
GitHub commits:

a19ebfd8bb
547dc5d512
0ca48e9570
7a1245f093
275ad04f9f
e0461d8ab0
73ed7556aa
94bee2cc54
3b5e1ce098

Reviewed By: yns88

fbshipit-source-id: b543a2f3fc30b82cc85a941b4316aabc8931325b
2019-12-06 06:34:38 -08:00
Lukas Piatkowski
c2b400ca53 rust toolchain: use RUSTC_BOOTSTRAP when building
Reviewed By: mitrandir77

Differential Revision: D18850917

fbshipit-source-id: f211dfbfadf4b5b0cb372cf8aeb9a232c4b3a739
2019-12-06 06:05:36 -08:00
svcscm
ee73968c7b Updating submodules
Summary:
GitHub commits:

8e7e836df3

Reviewed By: yns88

fbshipit-source-id: 9d6a4216f0de4179fa654aa2796cd7183e3769c5
2019-12-05 21:33:46 -08:00
Jun Wu
24d1a7e2dd update: use a dedicated edenfs update hook
Summary:
The `update` hook gets called even for in-memory updates that does not affect
real filesystem (or edenfs). That hook is set to `edenfsctl redirect fixup`,
which adds 0.35s overhead per commit when doing in-memory rebase.

Define a edenfs-only update hook to remove the overhead.

This is part of a stack of optimization that makes a 20s rebase to 5s (0.6s per
commit to 0.07s per commit).

Reviewed By: xavierd

Differential Revision: D18838470

fbshipit-source-id: cbe81f9dd89bff6d7c4bec3063d6014211f9aac6
2019-12-05 19:30:39 -08:00
Jun Wu
7a66e96504 copytrace: avoid phase operations with narrow-heads
Summary:
With narrow-heads before the new dag structure, reading phases is expensive.
Avoid it.

This is part of a stack of optimization that makes a 20s rebase to 5s (0.6s per
commit to 0.07s per commit).

Reviewed By: xavierd

Differential Revision: D18838468

fbshipit-source-id: 0847d2508114203f1d16a7240a43375d325103a5
2019-12-05 19:30:39 -08:00
Jun Wu
b6f0f1c184 rebase: avoid phase operations with narrow-heads
Summary:
With narrow-heads, rebase operation does not need to read existing phases to
decide new phases. Therefore remove it.

This is part of a stack of optimization that makes a 20s rebase to 5s (0.6s per
commit to 0.07s per commit).

Reviewed By: xavierd

Differential Revision: D18838469

fbshipit-source-id: 1298d4f4b0d72a764a286478e76c90b42b3fc569
2019-12-05 19:30:38 -08:00
Adam Simpkins
f1b827d7c7 update eden top to make use of the last line
Summary:
Use all available lines, now that we ignore the spurious errors that
can be thrown when calling `addnstr()` on the last line.

Reviewed By: wez

Differential Revision: D18826918

fbshipit-source-id: 7e5739207ba1a5bfa0d290da1345c9d9606b89fb
2019-12-05 19:27:11 -08:00
Adam Simpkins
0f7f9e9d11 make eden top ignore errors when writing data to curses
Summary:
Add a wrapper function around `addnstr()` that silently ignores any errors.

When writing to the last cell of a terminal curses will successfully display
the data, but report an error since the logical cursor cannot be moved to the
next cell.  This error is not important and can be ignored.

The errors reported by the `curses` module unfortunately are not very useful
(they normally just report `addnwstr() returned ERR`).  For now we silently
ignore all errors, although this diff does contain some stub code that can be
enabled to examine them during development.

Reviewed By: wez

Differential Revision: D18826917

fbshipit-source-id: a4c5394b49aea1334def1d047eec7b5e743fb9f2
2019-12-05 19:27:11 -08:00
Jun Wu
3d65779318 dag: add IdMapLike for minimal IdMap interface
Summary:
In the future we might have revlog or more complicated storage for maintaining
the IdMap. Define a trait for it.

Reviewed By: sfilipco

Differential Revision: D18664208

fbshipit-source-id: 242611d78852e8d82a52ab52192db87c13b848e7
2019-12-05 17:01:13 -08:00
Jun Wu
dbfc96ca15 dag: add to_first_ancestor_nth
Summary: This is the reverse of first_ancestor_nth. It will be used later.

Reviewed By: sfilipco

Differential Revision: D18664209

fbshipit-source-id: 8d3706cdc080086ec60da60870fde76c8933f6a8
2019-12-05 17:01:13 -08:00
Jun Wu
74e624297c dag: add first_ancestor_nth
Summary: This will be used in an upcoming change.

Reviewed By: sfilipco

Differential Revision: D18664207

fbshipit-source-id: 3030bb02e172b95c6dbabda74c318e7f16212f59
2019-12-05 17:01:12 -08:00
Jun Wu
f028832a62 dag: add a fast path to ancestors calculation
Summary:
With the ONLY_HEAD flag, ancestors can be sped up sometimes.  Since ancestors
is used widely in other DAG calculations, perf improvements are visible across
the benchmark. The trade-off is to spend a bit more time on building segments
to figure out ONLY_HEAD.

After this diff:

  building segments                                 576.625 ms
  ancestors                                          47.119 ms
  children                                          436.967 ms
  common_ancestors (spans)                          257.096 ms
  descendants (small subset)                        367.316 ms
  gca_one (2 ids)                                   146.315 ms
  gca_one (spans)                                   253.059 ms
  gca_all (2 ids)                                   236.075 ms
  gca_all (spans)                                   266.695 ms
  heads                                             220.766 ms
  heads_ancestors                                    36.258 ms
  is_ancestor                                        98.967 ms
  parents                                           219.173 ms
  parent_ids                                          9.282 ms
  range (2 ids)                                      99.859 ms
  range (spans)                                     205.154 ms
  roots                                             429.047 ms

Before this and previous diff:

  building segments                                 542.564 ms
  ancestors                                         316.075 ms
  children                                          428.612 ms
  common_ancestors (spans)                          452.850 ms
  descendants (small subset)                        537.483 ms
  gca_one (2 ids)                                   526.112 ms
  gca_one (spans)                                   446.726 ms
  gca_all (2 ids)                                   780.590 ms
  gca_all (spans)                                   490.906 ms
  heads                                             214.005 ms
  heads_ancestors                                   217.558 ms
  is_ancestor                                       256.769 ms
  parents                                           213.170 ms
  parent_ids                                          9.313 ms
  range (2 ids)                                     259.435 ms
  range (spans)                                     358.934 ms
  roots                                             423.005 ms

Reviewed By: sfilipco

Differential Revision: D18642797

fbshipit-source-id: 27f1ee5797a92142b2c19195eb26baef58769553
2019-12-05 17:01:12 -08:00
Jun Wu
94a09a890d dag: add a flag to indicate a segment is the only head so far
Summary: The pre-calculated flag can be used for some fast paths.

Reviewed By: sfilipco

Differential Revision: D18642796

fbshipit-source-id: bc16b3517cc29e23478f2027e83e2a2282be0d0a
2019-12-05 17:01:12 -08:00
Jun Wu
3e205e29b3 pydag: revise bindings
Summary:
The bindings APIs are out of date.

The latest idea is to write non-master commits to disk, in a different group.
There are no "segments in memory". Therefore:
- Rename `builddisk` to `build`.
- Make sure `build` updates internal states by calling `reload`.
- Drop `buildmem`.
- In case `builddisk` gets called very frequently, reduce its overhead by
  doing nothing if all commits are built already.

The constructor is revised to match the latest interface (no
"max segment level" and "set_new_segment_size" needs to be called).

Reviewed By: sfilipco

Differential Revision: D18640897

fbshipit-source-id: 76a5d0db363713dc5b763503c238c1bfa61ea02e
2019-12-05 17:01:11 -08:00
Jun Wu
744f17df8b dag: add group support for segments
Summary: Make it possible to build non-master segments.

Reviewed By: sfilipco

Differential Revision: D18641507

fbshipit-source-id: 4d0c46cdea240849af37d28219b6023578514323
2019-12-05 17:01:11 -08:00
Jun Wu
cdcbf836eb dag: fix a correctness issue that flat segment building might be incomplete
Summary:
`build_segments_volatile(id)` should build flat segments so it includes `id`.

Right now it can fail that in a corner case. Fix it.

Reviewed By: sfilipco

Differential Revision: D18641509

fbshipit-source-id: ad1de694e4d31c51f29e9bc5800291066bf16dbb
2019-12-05 17:01:11 -08:00
Jun Wu
5bac1b4598 dag: re-assign non-master ids to master ids
Summary:
This is needed to keep Ids topologically sorted.

This is reflected in the test added by D18641507, when assigning
master ids to `I`:

```
0---1---2---3---4---5---6--------H---I
     \               \          /
      N0-N1-N2-N3     N4-N5-N6-o
                \            \
                 -------------N7--N8
Lv0: R0-1[] 2-3[1] 4-6[3] N0-N1[1] N2-N3[N1] N4-N6[5] N7-N8[N3, N6]
Lv1: R0-3[] 4-6[3] N0-N1[1] N2-N3[N1] N4-N8[5, N3]
Lv2: R0-6[] N0-N3[1] N4-N8[5, N3]
```

Note: 0, 1, 2, ... are ids in the master group. N0, N1, ... are ids in the non-master group.

```
0---1---2---3---4---5---6--------11--12
     \               \          /
      N0-N1-N2-N3     7--8--9--10
                \            \
                 -------------N7--N8
Lv0: R0-1[] 2-3[1] 4-6[3] 7-10[5] 11-12[6, 10] N0-N1[1] N2-N3[N1] N4-N6[5] N7-N8[N3, N6]
Lv1: R0-3[] 4-6[3] 7-12[5, 6] N0-N1[1] N2-N3[N1] N4-N8[5, N3]
Lv2: R0-6[] 7-12[5, 6] N0-N3[1] N4-N8[5, N3]
Lv3: R0-12[] N0-N8[1, 5]
```

Notice that 7, 8, 9 replace the old non-master ids N4, N5, N6. Without this change,
N4, N5, N6 will be kept and break the topo-sorted property.

An `ensure!` was added to make sure Ids are topo-sorted.

Reviewed By: sfilipco

Differential Revision: D18641510

fbshipit-source-id: bbe642df3615c65db7f2e7c889624e00e53ef7ba
2019-12-05 17:01:10 -08:00
Jun Wu
fb718fe106 dag: allow re-assigning a non-master Id to a master Id
Summary:
The IdMap has assertions about 1:1 mapping between Id and Slice.

Practically, a non-master commit can be merged into a master commit.
That means a commit hash might map to 2 Ids: [master-id, non-master-id].
Allow that to happen (temporarily).

Note: The non 1:1 mapping causes more headache than benefit and
D18838995 restores 1:1 mapping. But it still requires temporary
non-1:1 mapping to work.

Reviewed By: sfilipco

Differential Revision: D18641508

fbshipit-source-id: bc4bc831d5ab5ebfc2588bafd48114ae49d31e0f
2019-12-05 17:01:10 -08:00
svcscm
a3b0c4c873 Updating submodules
Summary:
GitHub commits:

3ad0e5f766
ba6111f9ed
ad528fe5ca
35ba2c4e8a

Reviewed By: yns88

fbshipit-source-id: 1b667dd8c338455f24d21bce7647583369862118
2019-12-05 17:01:10 -08:00
svcscm
e4d9f000f7 Updating submodules
Summary:
GitHub commits:

1d485af800
9cff5f0985

Reviewed By: yns88

fbshipit-source-id: 87a61e2e9fc3fba1e8470f8f1552d4e04cec3475
2019-12-05 15:21:06 -08:00
Genevieve Helsel
1d6749828e update build eden thrift dependencys and update setup.py to know about new thrift structs
Summary: fix the `build-eden-thrift-deps.py` to generate a new zip file, upload it to lfs, and point setup.py to use this new generated thrift code

Reviewed By: quark-zju

Differential Revision: D18836545

fbshipit-source-id: 52c58c50319c878f2f6f708b952f0aa158d2ef4e
2019-12-05 15:16:33 -08:00
svcscm
572d21eb98 Updating submodules
Summary:
GitHub commits:

5c26f37c5a
0e72447d94

Reviewed By: yns88

fbshipit-source-id: 84c3752153be668f4b4d94e6b09245f2ec57c145
2019-12-05 15:16:32 -08:00
svcscm
1f9acc2691 Updating submodules
Summary:
GitHub commits:

d434927773
94d4080500
857c6a5beb
e036506a2c
9eea83005c
dd4661a2a1
1367be4188
0e81b8dc3f
ef0d20286e
800efe4f0a

Reviewed By: yns88

fbshipit-source-id: 76081e75832984837bf7c7c7005106c480668278
2019-12-05 10:03:07 -08:00
Lukas Piatkowski
23cec6bfd0 getdeps: add manifest for Rust toolchain
Reviewed By: ikostia

Differential Revision: D18814410

fbshipit-source-id: 889306e51c03a29c2b72e34b7dc60db21f11ff97
2019-12-05 08:47:12 -08:00
svcscm
33cdb3cdb2 Updating submodules
Summary:
GitHub commits:

7d303b5231
3997200095

Reviewed By: yns88

fbshipit-source-id: 902c3ed9f60d8699fd0557e5cd2b2b53e4f2b74d
2019-12-05 02:59:36 -08:00
Jun Wu
81b7593249 dag: avoid panic on drop
Summary: panic on drop is a terrible idea. It can cause SIGILL kill without meaningful messages or meaningful backtrace. Remove it.

Reviewed By: xavierd

Differential Revision: D18640894

fbshipit-source-id: 3ab52ded9875e17e891acd8f9b45d82964615f01
2019-12-04 22:43:24 -08:00