Commit Graph

51217 Commits

Author SHA1 Message Date
svcscm
e1dddcc7a6 Updating submodules
Summary:
GitHub commits:

8e4a2f5815
13fb6fb28e
7d97783503

Reviewed By: yns88

fbshipit-source-id: 282f14f69a5556856bcfa519d2bc616091c00907
2019-11-21 17:47:49 -08:00
Chad Austin
516e84d705 record the durations of the various phases of checkout
Summary:
Record the durations of each phase of checkout so it can be logged to
scribe in a future diff.

Reviewed By: simpkins

Differential Revision: D18632258

fbshipit-source-id: e447976af22cf25cffd640e79b95da2989b07448
2019-11-21 17:16:10 -08:00
svcscm
7c1158f25b Updating submodules
Summary:
GitHub commits:

b475d612b3
e50b64bdba

Reviewed By: yns88

fbshipit-source-id: 06432c065211c54575a505cad1b195a61fded24a
2019-11-21 16:56:34 -08:00
Zeyi (Rice) Fan
dee0478903 eden: add experimental:enable-blob-caching
Summary: Adding an experimental option to EdenFS so we can disable blob caching if we need.

Reviewed By: chadaustin

Differential Revision: D18441665

fbshipit-source-id: 56751c0199d6658bfbf7ec3865f746a7279324ab
2019-11-21 12:05:29 -08:00
Chad Austin
46b25acec8 introduce CheckoutResult struct
Summary:
We're about to return more information from checkout, so
introduce a CheckoutResult struct.

Reviewed By: genevievehelsel

Differential Revision: D18630820

fbshipit-source-id: 78b68ef3551ca8c9e9968631de554084c333b4b9
2019-11-21 11:50:46 -08:00
svcscm
44609b0a35 Updating submodules
Summary:
GitHub commits:

1f506c0cec
7fe1a5931a
fa022633a5
492916a16b

Reviewed By: yns88

fbshipit-source-id: e20fa707b85759ad9c2f07ee3e4f9a965e6fbe82
2019-11-21 10:41:02 -08:00
Johan Schuijt-Li
1b7dd50dbf hg: don't create tempfiles longer than allowed on filesystem
Summary:
[svcscm@hg022.prn2 /data/scm/www]$ hg log -r master
    transaction abort!
    rollback completed
    abort: File name too long: '/data/scm/www-merge-generaldelta/.hg/store/data/flib/intern/____generated____/_graph_q_l_meerkat_step/flib/intern/entschema/generated/entity/bizapps/dcp__insight/application__config/_ent_d_c_p_insights_application_config_action.php/._g_q_l_g___intern_____set_permission_manager_domain_dcp_insights_application_config_data_____ent_d_c_p_insights_application_config_action____gen_perform_graph_q_l_set_permission_manager_domain_dcp_insights_application_config_mutation_coercer.php.i-U0sMqX~'

On hg022.prn2 www currently failes under certain circumstances because filenames longer than
255 characters are currently created. I traced this back to mktempcopy and made sure that
we truncate filenames when they are created too long.

Reviewed By: quark-zju

Differential Revision: D18639433

fbshipit-source-id: 20e4b086ca5c775b6054b642ecedfb05efd00fb9
2019-11-21 10:41:02 -08:00
Jun Wu
6b6b6cf36d blackbox: add Event::Tags and its index
Summary:
Add Event::Tags so we can tag commands with one or more names.
Add index for the tags so we can search them more efficiently.

As we're here, update hgcommands to read EDENSCM_BLACKBOX_TAGS and write
Event::Tags automatically just after Event::Start.

Differential Revision: D18610733

fbshipit-source-id: c69b6e3d34b317e92a1c4d76f29e832b6b01c4fa
2019-11-21 08:36:10 -08:00
Jun Wu
49fe540b02 blackbox: build indexes to speed up certain query patterns
Summary:
Previously, the blackbox index was hardcoded on the timestamp, and is not
Event-aware. This diff changes the index to parse Event and build up indexes
for various common events (ex. Start, Finish).

For now, the index just includes start, finish timestamps and duration.

Differential Revision: D18610734

fbshipit-source-id: bf572f3138c8c18d3aab06eadcc83dd5b0b9cf02
2019-11-21 08:36:10 -08:00
Jun Wu
25e090c529 blackbox: extend match_pattern to capture matched objects
Summary:
Previously, `match_pattern` only returns a `bool`. This diff implements an
enhanced version `capture_pattern` to be able to capture matched objects.

This will be used to match query patterns and apply optimization.

Differential Revision: D18610732

fbshipit-source-id: 39e989e7633b834c04997bf8665494556f3b62b2
2019-11-21 08:36:09 -08:00
Jun Wu
f19a760f5f blackbox: remove IndexFilter
Summary: Migrate callsites to use the new pattern matching filtering APIs.

Differential Revision: D18610738

fbshipit-source-id: 242182f4e6a58bbff5dddbb177287921c85519bc
2019-11-21 08:36:09 -08:00
Jun Wu
ec9d925b69 blackbox: add timestamp field to Start and Finish event
Summary:
See the previous diff. As part of migrating away from IndexFilter. There needs
to be a way to filter commands by absolute clock time using pattern matching
against Events.  To achieve that, add the timestamp field to Start and Finish
events.

Differential Revision: D18610735

fbshipit-source-id: 9e1898a87bc93e47d0f0c77342428b5c42ef6f24
2019-11-21 08:36:09 -08:00
Jun Wu
41d6c2d42c blackbox: implement new query APIs
Summary:
The existing query API `filter` has 2 issues:
- It does querying and filtering together, making it hard to express certain
  queries. For example: "for commands taking more than 10 seconds, get all
  network requests used by those commands".
- The `IndexFilter` abstraction overlaps with `pattern`s. `IndexFilter` is
  built upon the `Entry` structure, and does not work for `Event`s. In the
  future, we want a `Event::Tags` to tag commands, and build indexes on it.
  That's not very compatible with `IndexFilter`.

To address the issues, I'm going to:
- Decouple querying and filtering by splitting it into 2 methods glued by
  `SessionId`s.
- Deprecate `IndexFilter`. Make `patterns` the only way to filter entries.
- Add dedicated indexes for certain events.

This diff adds the related APIs towards that direction.

Differential Revision: D18610737

fbshipit-source-id: c6ab92fd6fa7bb2ee7e14f567d72ac98b287c143
2019-11-21 08:36:08 -08:00
Jun Wu
34e2385f7d blackbox: remove type parameter of Entry
Summary:
The type `Entry<T>` provides flexibility on serializing different types.
Practically, only the `Event` type is used. Let's remove `T` so `blackbox` can
assume it's the `Event` type.

Differential Revision: D18610736

fbshipit-source-id: 102a5f61c758fa83cd1994c7f813576dcf5b59d5
2019-11-21 08:36:08 -08:00
Jun Wu
6b8acfe260 hgsql: add rate limiting for read commands
Summary:
Similar to D2756217. But skip sync directly if the repo has been synced
recently.

This will help reducing CPU load on the DB tier.

Reviewed By: ikostia

Differential Revision: D18621013

fbshipit-source-id: 83370669ea61eb693d06495aaa13664648b97aca
2019-11-21 07:45:31 -08:00
svcscm
1dbac1afb1 Updating submodules
Summary:
GitHub commits:

5a446fba45

Reviewed By: yns88

fbshipit-source-id: 2fa47c5a5b0f4991ee1348608d4fdc0b6efc0ea6
2019-11-21 07:45:30 -08:00
svcscm
5b41520207 Updating submodules
Summary:
GitHub commits:

571711d8dd
3b4e00b8d5
6a47d8fb8f
8477f0799f
0ce0edbe12
a8585e7f2a
c1e8df310f
ece21c3d28
69233daadc

Reviewed By: yns88

fbshipit-source-id: 440e8da368c261b1bb7fb61b28678b66790c2567
2019-11-20 19:15:23 -08:00
Jun Wu
5b61ae4c89 edenapi: add tracing spans
Summary:
Add tracing spans so we can get some insights about statistics from the pure
Rust land.

Reviewed By: xavierd

Differential Revision: D18625646

fbshipit-source-id: 15951fe2f8270a70f647a0f4e09819bbc33919c5
2019-11-20 18:56:07 -08:00
Genevieve Helsel
a47b831d54 stop processing getScmStatusV2() calls if the thrift connection is closed
Summary: make `future_getScmStatusV2` into `async_tm_getScmStatusV2` so we can short circut the status call if the request is no longer active.

Reviewed By: simpkins

Differential Revision: D18378212

fbshipit-source-id: b2bca014a37d99f7a92478c3e23bde0b874dd13c
2019-11-20 18:06:33 -08:00
Genevieve Helsel
3807e751ca make enforceParents configurable for getScmStatusV2
Summary: This reads `enforceParents` from a config instead of always assuming true for `getScmStatusV2()`. This will allow a easy kill switch in case throwing errors from this thrift call causes issues with something that calls hg status

Reviewed By: simpkins

Differential Revision: D18258164

fbshipit-source-id: 1ae421a941c01a678d25d5453c771262b03558d0
2019-11-20 17:51:21 -08:00
Jun Wu
60ccca41fc graphql: return directly for empty "landed" query
Summary:
It seems an emtpy GraphQL query always times out. So let's avoid such queries
from the client side.

Reviewed By: xavierd

Differential Revision: D18626286

fbshipit-source-id: 70f0061f7f7ad8c7d8be4e15fc419039dc07d61e
2019-11-20 16:57:35 -08:00
svcscm
7471ac41f3 Updating submodules
Summary:
GitHub commits:

96f60c7a28
01987a195c
8f815149ec
47445ecdd0
c3b48f4db4
717a31547f
0e62d3bb5f
3a85df8ead
0c319bd2c5

Reviewed By: yns88

fbshipit-source-id: 4419ddcaf3d818b908bec9f47a25be5762aa7888
2019-11-20 16:57:34 -08:00
Adam Simpkins
7d0226157f getdeps: update eden dependencies
Summary:
Include tools/lfs in the shipit path map for now, since it is needed to
download vendored Rust crates on internal builds.  Also correctly indicate
that the fb-mercurial* dependencies also apply only to internal builds.

Reviewed By: chadaustin

Differential Revision: D18621421

fbshipit-source-id: 74f00bf318ff9aefdca8696a61a8f33587749e66
2019-11-20 16:04:33 -08:00
Adam Simpkins
2eb88ab683 getdeps: automatically detect if a build appears to be Facebook-internal
Summary:
Automatically detect the `--facebook-internal` flag based on the current
repository project name.

Reviewed By: wez

Differential Revision: D18621358

fbshipit-source-id: f2b3018169b151811eec455863a8bfc17667d4d8
2019-11-20 16:04:33 -08:00
svcscm
38b9d58808 Updating submodules
Summary:
GitHub commits:

2c761ddd41
352c2d8b66
f87544dffb
dcbe9a1da3
5300258a58
c0983d0691

Reviewed By: yns88

fbshipit-source-id: fc31c6f0bb6c09bb2d1c6f55e9a9afffef51cb5f
2019-11-20 16:04:32 -08:00
svcscm
ee4183030f Updating submodules
Summary:
GitHub commits:

7cc9d9257b
93d91859c8
ab0a6495f6
3cd75736a7
fb3e6aac5d
4ac5fd6ed9
cf783ae678
6aaaa4754f

Reviewed By: yns88

fbshipit-source-id: 60b6d716aa073eda5fc6dbcbd0daeee536c25314
2019-11-20 13:00:45 -08:00
Adam Simpkins
344661b651 cmake: update RustStaticLibrary.cmake to hook up cargo dependencies
Summary:
Update `RustStaticLibrary.cmake` to let CMake know that running `cargo` will
produce the Rust static library output.  Without this CMake does not know how
to correctly wire up dependencies for libraries.  Previously the code used the
`ALL` target to ensure that the Rust static library was built, but this did
not ensure that it was built before executables that require it.

Reviewed By: pkaush

Differential Revision: D18588010

fbshipit-source-id: 1a1880993bdfa3fcb876a0f6507e5cddfea9b738
2019-11-20 12:28:40 -08:00
Adam Simpkins
55d980ee42 update RustStaticLibrary.cmake to support vendored crates on Windows
Summary:
Update the `RustStaticLibrary.cmake` code to explicitly find and invoke the
Python interpreter when running the Python script to download vendored crates.
This is required to work on Windows.

Reviewed By: pkaush

Differential Revision: D18588012

fbshipit-source-id: 4c2dfbc786adf0a7aa6b43483060800627a8aca1
2019-11-20 12:28:40 -08:00
Adam Simpkins
8dc2c5963a cmake: automatically decide whether to use vendored rust crates
Summary:
Update `RustStaticLibrary.cmake` to automatically decide to try and download
vendored Rust crates or not, based on the presence of the script it uses to
download them.

Reviewed By: fanzeyi

Differential Revision: D18588007

fbshipit-source-id: 995365f0cc2368897ddf94b5617bdb9bc0682edc
2019-11-20 12:28:40 -08:00
Adam Simpkins
a9d5178036 avoid downloading vendored crates once per library
Summary:
This updates the `RustStaticLibrary.cmake` file to download vendored
`crates.io` crates only once, rather than once per library.  It also now
downloads them into the build directory, rather than into the source
directory.

This can probaly still be improved more in the future: the vendored crates
should probably just be treated as a top-level getdeps manifest entry.
Currently if multiple projects build rust static libraries we will still end
up downloading the vendored crates once per project.

Reviewed By: fanzeyi

Differential Revision: D18548094

fbshipit-source-id: 83060e679455267bdf197e5ff2d33b1b7f0204fc
2019-11-20 12:28:39 -08:00
svcscm
8673f1fdc8 Updating submodules
Summary:
GitHub commits:

965e5ed7de
cea105e971
c5fdc21fb3
2d6825cdf3
fe92d2a5ce

Reviewed By: yns88

fbshipit-source-id: a3016332680c25d6f7c339ef7e8ac4e0ab143d89
2019-11-20 09:41:54 -08:00
Jun Wu
9027d90624 dispatch: prompt 'eden doctor' for TApplicationException
Summary:
Many people hit TApplicationException recently. Most of them can be fixed
by `eden doctor`.  Instead of crashing and showing an unhelpful backtrace, show
a shorter error message and prompt the user to run `eden doctor`.

Reviewed By: fanzeyi

Differential Revision: D18609445

fbshipit-source-id: 5492fa92008cb33e1122b72b48d0858db0d5f994
2019-11-20 09:32:24 -08:00
svcscm
31c07393a8 Updating submodules
Summary:
GitHub commits:

d14cb92418
c14bee79c8

Reviewed By: yns88

fbshipit-source-id: 37ee90acdb9384137208674c5e64bb86111283a8
2019-11-20 09:32:24 -08:00
svcscm
63a055f1fe Updating submodules
Summary:
GitHub commits:

b21fd47972
950060c67b
d5cfc73665
195d10ad15
22c4b39574
0306e01233
fc0ad8b966
6f87219b24
9c674a1271
69ac8aeb62
672beabd4c

Reviewed By: yns88

fbshipit-source-id: 96ba9389d7c7faf53c0c5775a425dbea17da217a
2019-11-19 23:21:10 -08:00
Zeyi (Rice) Fan
c7546996e9 eden: switch to using Rust code parse manifest
Reviewed By: chadaustin

Differential Revision: D18365537

fbshipit-source-id: ef4d2fc5af01ab86f50df44b7a98f7b6f2884587
2019-11-19 18:01:24 -08:00
Zeyi (Rice) Fan
34c255890d eden: implement conversion functions from Rust Tree to EdenFS Tree
Summary:
This diff adds the C++ conversation methods that converts the Rust struct introduced in the previous Diff into EdenFS's `Tree`.

We have to keep the RocksDB write batch in for storing proxy hashes.

Reviewed By: wez

Differential Revision: D18365504

fbshipit-source-id: 2a2cd2cfebd9f0ae4e279db608becc5890c5ccdf
2019-11-19 18:01:24 -08:00
Zeyi (Rice) Fan
9328adfaf9 backingstore: add Tree and TreeEntry Rust structs to pass manifest
Summary: This diff adds Rust structs to represent the data EdenFS wants from a manifest so we can pass these to EdenFS.

Reviewed By: wez

Differential Revision: D18365440

fbshipit-source-id: b474ceb698d9230e1b02c2cce1bec2a32bd3f94f
2019-11-19 18:01:23 -08:00
Zeyi (Rice) Fan
1713cfca79 backingstore: impl Drop for CBytes and add From<Vec<u8>> for CBytes
Summary:
Some clean up.

`CByte` should clean up `vec` in `Drop` since we are going to have `CBytes` in other struct's field and this will allow Rust automatically manage this memory.

Reviewed By: chadaustin

Differential Revision: D18365441

fbshipit-source-id: 7038bc6ff457415ee8234cc4c6b7df0f70005cfe
2019-11-19 18:01:23 -08:00
Zeyi (Rice) Fan
f27e1eb2e0 backingstore: add TreeContentStore to impl TreeStore
Summary: `manifest::Tree` wants to have a store that implements `TreeStore`. However at current we don't want to introduce dependency between `manifest` and `contentstore`. So we create this wrapper struct to provide the trait implementation. This will allow us to use `manifest` to parse manifest data we get from hg store.

Reviewed By: chadaustin, xavierd

Differential Revision: D18365354

fbshipit-source-id: 3687032c7f51570ef51ccc4004efc87f64ef2188
2019-11-19 18:01:23 -08:00
Zeyi (Rice) Fan
7f0f541fa1 manifest: add FsNode to List::Directory
Reviewed By: sfilipco

Differential Revision: D18307923

fbshipit-source-id: 1844afb7b9a185f7e89c11476ee2eb34cac117c6
2019-11-19 18:01:22 -08:00
Zeyi (Rice) Fan
5b274b1a4c manifest: add tree HgId to FsNode
Summary: This diff adds `Option<HgId>` to `FsNode::Directory` because EdenFS needs the tree hash for the directories in the tree.

Reviewed By: sfilipco

Differential Revision: D18307205

fbshipit-source-id: 93db944ff1686a1c5927aa9a876a0fd008edbf9a
2019-11-19 18:01:22 -08:00
svcscm
c91facaf23 Updating submodules
Summary:
GitHub commits:

107984051e
c420949847
4e0dcd36df

Reviewed By: yns88

fbshipit-source-id: 328096fad9acc47e4f801645924bf4327ba8f726
2019-11-19 18:01:22 -08:00
svcscm
9e854012ac Updating submodules
Summary:
GitHub commits:

81a1e5270d
03f8dc6a44
a80e0cab35
20b48c6478
27ea56c7c0

Reviewed By: yns88

fbshipit-source-id: b98a17c125c2f48decea86c912fe4692d056f616
2019-11-19 13:01:41 -08:00
Jun Wu
2918ef4850 test-hggit-clone: update the test
Summary:
The test needs to be updated since we now print commit hashes instead of
revision numbers in `phase` output.

Reviewed By: akushner

Differential Revision: D18596300

fbshipit-source-id: b42bb6ed68dfc33977f0c9df5018a5bf365c2909
2019-11-19 12:05:05 -08:00
svcscm
92fac21ac1 Updating submodules
Summary:
GitHub commits:

ddc3cd2e2c
547f5abfb8
75acf748b1
38cc611297
2d5fbf572b

Reviewed By: yns88

fbshipit-source-id: fc4e253e87f23f6ff79481790eb9504e50733ed5
2019-11-19 11:58:00 -08:00
Xavier Deguillard
2afcb03dd8 split: use rebase.rebase instead of restackonce
Summary:
The restackonce function warns about not using it and using `rebase.rebase`
directly, so let's just do that.

Reviewed By: quark-zju

Differential Revision: D18574767

fbshipit-source-id: 56829bccbba7f727867ba2b9bd0a81530ba29bf3
2019-11-19 09:10:37 -08:00
Stanislau Hlebik
bcfbc5c19e fix compilation warning
Summary: I got unused warning every time I build Mononoke.

Reviewed By: singhsrb

Differential Revision: D18573764

fbshipit-source-id: 23921f581bdc74655041a2413f80cb159b4ba010
2019-11-19 00:43:36 -08:00
Jun Wu
0bab8ac4b2 tests: revive silenttestrunner.py
Summary: D18541783 accidentally removed it. Revive the file to unbreak tests.

Reviewed By: singhsrb

Differential Revision: D18587477

fbshipit-source-id: 7a7a0cf8a53e5e1fc1856aab18a3da02e53e019a
2019-11-18 20:34:04 -08:00
Jun Wu
1f5ccb5939 test-hggit-bookmark-workflow: update the test
Summary: D18544475 changed `phase` output. Update this test.

Reviewed By: singhsrb

Differential Revision: D18587462

fbshipit-source-id: 82b5a75a006c792f259af9ca038c24a032069f55
2019-11-18 20:34:04 -08:00
Jun Wu
48722a4f20 test-contrib: remove the test
Summary: The test was about `simplemerge`, which was removed. Therefore, remove the test.

Reviewed By: singhsrb

Differential Revision: D18587461

fbshipit-source-id: daf12f0c38ec08a0f73d834b8f32dc1a65620a8f
2019-11-18 20:34:04 -08:00