Commit Graph

58981 Commits

Author SHA1 Message Date
Jun Wu
ad3c659616 test-rebase-interruptions: enable modern configs
Summary: This enables modern setup for the test.

Reviewed By: DurhamG

Differential Revision: D22175003

fbshipit-source-id: 6e153d4c48e02420c43989e317139d66b89da1da
2020-07-02 13:22:30 -07:00
Jun Wu
4dd49baba7 test-rebase-inmemory: enable modern configs
Summary:
The test used a lot of revision numbers. Changing them to non-revision-numbers
fixes it.

Reviewed By: DurhamG

Differential Revision: D22174974

fbshipit-source-id: 51915afa481ac0263eaeaa3339df8e10a2bc35d5
2020-07-02 13:22:30 -07:00
Jun Wu
dfd8ab5938 test-rebase-detach: enable modern configs
Summary:
This turns on modern setups including visibility, mutation and narrow-heads.
The test used revision numbers. Changing them to non-revision-numbers fixes it.

Reviewed By: DurhamG

Differential Revision: D22174958

fbshipit-source-id: 2a8c178021be9931e66b77f00d3de59d8c794ce5
2020-07-02 13:22:30 -07:00
Jun Wu
a539962c38 test-rebase-conflicts: enable modern configs
Summary:
This turns on modern setups including visibility, mutation and narrow-heads.
The test used revision numbers. Changing them to non-revision-numbers fixes it.

Reviewed By: DurhamG

Differential Revision: D22175005

fbshipit-source-id: c6292f10c27d03102df13a71332ef7a0304d5807
2020-07-02 13:22:30 -07:00
Jun Wu
f4f263fef9 test-rebase-bookmarks: enable modern configs
Summary:
This turns on modern setups including visibility, mutation and narrow-heads.
The test used revision numbers. Changing them to non-revision-numbers fixes it.

Reviewed By: DurhamG

Differential Revision: D22174964

fbshipit-source-id: 74d45b1b497497dfb6fa4b685e862d1bce4d4e5c
2020-07-02 13:22:30 -07:00
Jun Wu
942bc0cf0c test-rebase-flag: enable modern configs
Summary:
This turns on modern setups including visibility, mutation and narrow-heads.
Test seems fine - just revision number changes.

Reviewed By: DurhamG

Differential Revision: D22174998

fbshipit-source-id: fadec9b56be3cd3d5117f7418635e4d6042a52eb
2020-07-02 13:22:29 -07:00
Jun Wu
9c9545f5c3 test-fb-hgext-pushrebase-hgsql-mfs: delete the test
Summary:
The test was about pushing to hgsql servers. Nowadays we push to Mononoke
directly and sync to hgservers in a very different code path. Delete the
test as it's no longer relevant and involves too much low level details
to be compatible with modern setups.

Reviewed By: DurhamG

Differential Revision: D22174967

fbshipit-source-id: 08006bcf59090505d6116dbc024cb9378d27cf02
2020-07-02 13:22:29 -07:00
Jun Wu
2bd81db543 test-fb-hgext-fbhistedit-rebase-interactive: enable modern configs
Summary:
This turns on modern setups including visibility, mutation and narrow-heads.
The test only needs a few revision number fixup.

Reviewed By: DurhamG

Differential Revision: D22174973

fbshipit-source-id: 47470971fb361ac0bec45b932ad3fb66950f0d1c
2020-07-02 13:22:29 -07:00
Jun Wu
5434e42738 test-copy-move-merge: enable modern configs
Summary:
This turns on modern setups including visibility, mutation and narrow-heads.
The test only needs a revision number fixup.

Reviewed By: DurhamG

Differential Revision: D22174989

fbshipit-source-id: 3cef08e4e2214c9ffabc34fdd6e35cd02101172c
2020-07-02 13:22:29 -07:00
Jun Wu
04ddfd3c37 test-convert-hg-sink: turn on visibility and narrow-heads
Summary: Assign bookmarks explicitly to make commits visible.

Reviewed By: DurhamG

Differential Revision: D22174982

fbshipit-source-id: 5310e2f3fd53145c903b824043b0f6591517a156
2020-07-02 13:22:29 -07:00
Jun Wu
d9c8fdb3eb test-verify: turn on visibility and narrow-heads
Summary: It seems the test just passes with the change.

Reviewed By: DurhamG

Differential Revision: D22174979

fbshipit-source-id: 88b940e54fc7a5f71ca839cb583cf219fd3f30b9
2020-07-02 13:22:29 -07:00
Jun Wu
073ad87437 test-strip-cross: turn on visibility and narrow-heads
Summary:
This test works fine except for stripping rev 0 (the first commit).  Since
strip is not used in production for end-users and is likely going away, and we
are unlikely to strip the first commit of a repo on server-side, I just removed
the problematic test case to make it compatible.

Reviewed By: DurhamG

Differential Revision: D22174993

fbshipit-source-id: f362c6209852015558924151b58dc06b16c95df0
2020-07-02 13:22:29 -07:00
Jun Wu
f1afaa9ae4 tests: enable narrow-heads for more Python tests
Summary: Migrate some Python tests to use narrow-heads.

Reviewed By: DurhamG

Differential Revision: D22130167

fbshipit-source-id: 5cc71711b5bc674daed5040909e7cd141b7a9cfc
2020-07-02 13:22:29 -07:00
Jun Wu
5c567018c8 tests: enable narrow-heads for more tests
Summary: Migrate more tests to use narrow-heads.

Reviewed By: DurhamG

Differential Revision: D22130171

fbshipit-source-id: 92a1b1dc237a7f66b4430ef680d1a2f60e715778
2020-07-02 13:22:28 -07:00
Jun Wu
6fd6454502 tests: enable narrow-heads for test-fb-hgext-treemanifest-*
Reviewed By: DurhamG

Differential Revision: D22130166

fbshipit-source-id: dcff8bd6263c5cb6dd1f19320969f1d202b5da67
2020-07-02 13:22:28 -07:00
Jun Wu
b1809628fe tests: enable narrow-heads for test-d* to test-fb-hgext*
Summary: Most of them need extra server-side bookmarks for deciding phases.

Reviewed By: DurhamG

Differential Revision: D22130169

fbshipit-source-id: 2a4d3301fbf9dd8b8003fc319bda1180c6196ce8
2020-07-02 13:22:28 -07:00
Jun Wu
3b722c6ebc template: remove obsfate, instabilities, succsandmarkers
Summary: Those concepts use obsstore. They become no-ops with mutation. Remove them.

Reviewed By: DurhamG

Differential Revision: D22320091

fbshipit-source-id: eb9a8196b6ec8c8164a0d04cb7258c41cd5de596
2020-07-02 13:22:28 -07:00
Jun Wu
55dd359c34 repo: narrow-heads repos are not publishing
Summary:
Usually server-side repos are not using narrow-heads. In some cases they do
(ex. push / pull without using the ssh protocol in tests). When a narrow-heads
repo receives a push without a bookmark, the repo will not update its visible
heads, causing the pushed commits to be invisible during discovery.

Change it so when narrow-heads repo receives pushes, update visible heads so
commits are visible during discovery. This is mostly for test compatibility
and does not affect production use-cases.

Reviewed By: DurhamG

Differential Revision: D22320090

fbshipit-source-id: 14625b9db0dbea6327af33bbd352a866cb10e397
2020-07-02 13:22:28 -07:00
Arun Kulshreshtha
b9d35dc854 edenapi_types: add ToJson trait
Summary: Add a `ToJson` trait as a counterpart to the `FromJson` trait introduced in the last diff. The primary use case for this will be to allow recording and replaying the data fetches that occur during Mercurial operations. A JSON representation was chosen so that the format will be directly compatible with tools like `make_req` used in EdenAPI integration tests.

Reviewed By: markbt

Differential Revision: D22344599

fbshipit-source-id: 52c888bde93a8e86b6dd76cb862337f716b007eb
2020-07-02 13:11:48 -07:00
Arun Kulshreshtha
58984e1378 edenapi_types: add FromJson trait
Summary:
Add a `FromJson` trait to `edenapi_types` and use it instead of the parsing functions when parsing requests from JSON.

Some design commentary:

I've created a custom trait rather than using `TryFrom<serde_json::Value>` for two reasons:

- From a design standpoint, I'd like users to have to explicitly opt-in to this functionality by importing this `FromJson` trait, since this is different from the usual way of deserializing with serde, and it might cause confusion.
- From an implementation standpoint, it turns out that using `TryFrom` in a trait bound causes difficulties with type inference in some situations (in particular, around the associated `Error` type), which necessitates type annotations, making the API less ergonomic to use.

Why not just use `serde::Deserialize` directly?

- The representation here doesn't actually directly match the structure of the underlying type. Instead, the JSON representation has been designed to be easy for humans to edit when writing tests. (For example, keys are replaced with simply arrays, and hashes are represented as hex rather than as byte arrays.)

- In this case, we'd like to work with `serde_json::Value` rather than going directly between bytes and structs, since this allows for a level of dynamism that will be useful later in the stack.

Reviewed By: markbt

Differential Revision: D22320576

fbshipit-source-id: 64b6bed42e1ec599a0da61ae5d55feb7c90101a4
2020-07-02 13:11:48 -07:00
Mark Thomas
dba11deb2d scs_server: implement commit_list_descendant_bookmarks
Summary:
Implement `commit_list_descendant_bookmarks` by iterating over all bookmarks and
checking if the target commit is an ancestor of the bookmark's current target.

Reviewed By: mitrandir77

Differential Revision: D22357988

fbshipit-source-id: e1b1d7387742ba7133370f52c4d36c0b1a77f4e3
2020-07-02 12:58:16 -07:00
Zeyi (Rice) Fan
21c11f4921 make FuseRequest high priority & TreePrefetchLease low priority
Summary:
This is much better than having `ObjectFetchContext` itself owns a copy of `ImportPriority`. We can actually customize how different fetch context manages these priority.

We set all FUSE requests to a higher priority and prefetch requests to a lower priority

Reviewed By: xavierd

Differential Revision: D22342802

fbshipit-source-id: b9c1d0f2ddbc7a5e5d619bc2c2222e5df0e702af
2020-07-02 12:00:45 -07:00
Zeyi (Rice) Fan
07452335fb use ObjectFetchContext for priority
Summary: This commit switch from explicitly specifying `ImportPriority` into passing priorities from `ObjectFetchContext`.

Reviewed By: xavierd

Differential Revision: D21872720

fbshipit-source-id: 26055eff21cab4ce6370e96ac3acbac2fd6af3f0
2020-07-02 12:00:45 -07:00
Zeyi (Rice) Fan
d10b17a031 make prefetch in TreeInode low priority
Summary:
This commit makes prefetch requests derived in `TreeInode` assigned to lower priority.

Since prefetch is not running under the original request's context, we create a new `ObjectFetchContext` specifically for this case, and attach it to `prefetchLease` since it is guaranteed to be alive during the span of prefetch.

Reviewed By: chadaustin

Differential Revision: D21872719

fbshipit-source-id: d1495b06031980d7d1c21ecf2a0b47e83fc9672d
2020-07-02 12:00:45 -07:00
Zeyi (Rice) Fan
5030f0be35 add ImportPriority to ObjectFetchContext
Summary:
This commit adds `ImportPriority` to `ObjectFetchContext`. By doing so we can tweak priority for a request at different stage.

This commit also provides a default implementation for the virtual methods in `ObjectFetchContext` so we can create one to carry `ImportPriority` in some specific cases.

Reviewed By: chadaustin

Differential Revision: D21872718

fbshipit-source-id: 6e8cfd84959b368e6fe69fda2baf0debf7a88295
2020-07-02 12:00:45 -07:00
Zeyi (Rice) Fan
99f8448e99 adding ObjectFetchContext to readdir
Summary: This commit makes `readdir()` to correct send its `RequestContext`  to `EdenDispatcher` method so underlying code can know the current request being processed is from FUSE.

Reviewed By: xavierd

Differential Revision: D21821949

fbshipit-source-id: f41ba912fedbfc040e3c9267aad25e7f33f8e912
2020-07-02 12:00:45 -07:00
Mark Thomas
309e4539ab mononoke_api: bypass cache for old bookmark location in move_bookmark
Summary:
Currently the `move_bookmark` API needs to get the old bookmark location in order
to move the bookmark.  We'll fix that in general later, but for now we need to
make sure the value we use doesn't come from an out-of-date cache (e.g. the
warm_bookmarks_cache), as it may prevent the move from working.

Reviewed By: krallin

Differential Revision: D22358467

fbshipit-source-id: 4d46a6be717644b24663318326fdcd81249481c9
2020-07-02 11:01:04 -07:00
svcscm
62806759fe Updating submodules
Summary:
GitHub commits:

db4aa27e31
5850ce9f95

Reviewed By: wittgenst

fbshipit-source-id: 6ba7123eea69918ecde46c1502ff8ff8c63267d2
2020-07-02 10:58:30 -07:00
Durham Goode
c91dc5ac70 py3: fix merge._resolvetrivial
Summary: It depends on items() returning a list.

Reviewed By: quark-zju

Differential Revision: D22339205

fbshipit-source-id: 4691c981ba612ec9a00752cbcc57bf2272b59007
2020-07-02 09:05:13 -07:00
Durham Goode
968e57735d configs: use zstd compression for remote config request
Summary:
The remote configs are non-minified json, so using some compression
should reduce their size tremendously.

Reviewed By: quark-zju

Differential Revision: D22336251

fbshipit-source-id: 77282aae57a8e37e3462f6379dedebd2978a4d0e
2020-07-02 09:02:15 -07:00
Ailin Zhang
7c603e51f8 make fetch heavy threshold configurable
Summary: This diff made fetch threshold configurable, so we can change it later as repository size grows.

Reviewed By: fanzeyi

Differential Revision: D22337850

fbshipit-source-id: 4b46420cb4e7164a3f1080279d67fa5f90549cd8
2020-07-02 08:44:02 -07:00
Ailin Zhang
36fd61dbaa send fetch heavy events to Scuba
Summary: This diff updated `ObjectStore` to send a `FetchHeavy` event to Scuba when the number of fetching requests of a process has reached 2000.

Reviewed By: fanzeyi

Differential Revision: D22292104

fbshipit-source-id: b7ac48412868216ea960c8681a5fb71c587952bc
2020-07-02 07:57:15 -07:00
Mark Thomas
6d5bce25c6 mononoke_api: implement pagination for all bookmarks
Summary:
Bookmark requests that are truncated because the requested limit is reached now return a `continue_after` value, containing the last bookmark that was processed.

Callers can make a subsequent request with the same parameters, but `after` set to the value received in `continue_after` to continue their request where it left off.

Reviewed By: krallin

Differential Revision: D22338301

fbshipit-source-id: 81e398bee444e0960e65dc3b4cdbbe877aff926d
2020-07-02 07:53:12 -07:00
Mark Thomas
344c8edda4 add commit_list_descendant_bookmarks
Summary:
Add `commit_list_descendant_bookmarks` which will list all bookmarks that are
descendants of a particular commit.

We will also use this opportunity to complete the implementation of pagination
for regular bookmark listing, so add the appropriate fields to the
`repo_list_bookmarks` request and response structs.

Reviewed By: StanislavGlebik

Differential Revision: D22338300

fbshipit-source-id: defd019795c2a2ac9e5573d58de187c10848397f
2020-07-02 07:53:12 -07:00
Mark Thomas
3e4e59baef bookmarks: add 'pagination' filter to 'list'
Summary:
Add a new parameter, `pagination`, to the `list` method of the `Bookmarks` trait.

This restricts the returned bookmarks to those lexicographically after the
given bookmark name (exclusive).  This can be use to implement pagination:
callers can provide the last bookmark in the previous page to fetch the
next page of bookmarks.

Reviewed By: krallin

Differential Revision: D22333943

fbshipit-source-id: 686df545020d936095e29ae5fee24258511f4083
2020-07-02 07:53:12 -07:00
Mark Thomas
742eb6f829 bookmarks: rework Bookmarks traits
Summary:
Rework the bookmarks traits:

* Split out log functions into a separate `BookmarkUpdateLog` trait.  The cache doesn't care about these methods.

* Simplify `list` down to a single method with appropriate filtering parameters.  We want to add more filtering types, and adding more methods for each possible combination will be messier.

* The `Bookmarks` and `BookmarkUpdateLog` traits become `attributes` on `BlobRepo`, rather than being a named member.

Reorganise the bookmarks crate to separate out the bookmarks log and transactions into their own modules.

Reviewed By: krallin

Differential Revision: D22307781

fbshipit-source-id: 4fe514df8b7ef92ed3def80b21a16e196d916c64
2020-07-02 07:53:12 -07:00
Mark Thomas
64610d46c2 bookmarks: escape LIKE patterns used for bookmark prefixes
Summary:
The LIKE pattern used by bookmark prefixes needs to be escaped, otherwise
users looking for bookmarks containing `\`, `_` or `%` will get the
wrong results.

Reviewed By: krallin

Differential Revision: D22336716

fbshipit-source-id: 99b0ad6097f096358e66042752e4d153359935be
2020-07-02 07:53:11 -07:00
Jun Wu
f15a21db2f color: do not show EnableVtMode warning if color is set to always
Summary:
If the user specifies `always`, they know what they are doing and do not
pollute their terminal with the EnableVtMode warnings.

Reviewed By: markbt

Differential Revision: D22343823

fbshipit-source-id: 7dcff43c3fa1163a061a2173d57799d2cd2ef05b
2020-07-02 07:13:23 -07:00
Kostia Balytskyi
b134a2f5bb blobstore_healer: fix how replication lag is monitored
Summary: We were monitoring the wrong lag so far.

Reviewed By: farnz

Differential Revision: D22356455

fbshipit-source-id: abe41a4154c2a8d53befed4760e2e9544797c845
2020-07-02 06:18:35 -07:00
Stanislau Hlebik
f6d06a266a mononoke: check conflicts correctly when doing bulk adds in transaction
Summary:
`bulk_add()` method was checking for conflicts correctly i.e. it wouldn't fail
if we try to insert the same mapping twice.
`bulk_add_git_mapping_in_transaction` wasn't doing this check i.e. it would
fail.

This caused us a few problems and this diff fixes them - now
`bulk_add_git_mapping_in_transaction` would do the same checks as bulk_add was
doing previously.

There is another important change in behaviour: if we try to insert two entries, one of them
has a conflict another don't then previously we'd insert the second entry.
Now we don't insert any, arguably that's a preferred behaviour.

Reviewed By: krallin

Differential Revision: D22332001

fbshipit-source-id: 86fff8c23c43eeca0fb36b01b10cdaa73b3ce4ab
2020-07-02 05:31:29 -07:00
Liubov Dmitrieva
6f8fc0e3a2 implement command to list workspaces
Summary:
New Commit Cloud command to list user's own worspaces in Commit Cloud

hg cloud ls

Reviewed By: markbt

Differential Revision: D22308127

fbshipit-source-id: 756c419a9bb3d6f50ddd5b4dd344cd35b9a02d2d
2020-07-02 04:49:06 -07:00
svcscm
298919794c Updating submodules
Summary:
GitHub commits:

091602b300

Reviewed By: wittgenst

fbshipit-source-id: 50ab43759eef8360768e5b5776f696c7a63485ad
2020-07-02 04:26:55 -07:00
Lukas Piatkowski
505d2e8a63 eden/scm: move third-party facebook crates dependencies to a separate folder
Summary:
Having fbsource's thrid-party crates in a separate folder will make it easier to provide a different manifest for the third-party crates for OSS builds.

In order to not trigger any network calls during internal FB builds you cannot define dependencies with "git" urls. Even if they are being patched up in the same manifest Cargo will still make a call to github. That is why internall the Cargo.toml files in third-party folder will contain dependency definitions that will use "path" dependencies only.

In the next diff a separate set of Cargo.toml files will be provided inside `fbcode/eden/oss` that ShipIt will replace for GitHub with the proper "git" url dependencies.

Note: Why even bother with "git" and "patch" section and not use only "path" dependencies? In OSS eden builds depending on other project - like fbthrift or rust-shed. They use regular "git" dependencies. So if you try to compile Eden that uses "path" dependencies with fbthrift/rust-shed that uses "git" dependencies you will get errors from Cargo saying that e.g. rust-shed's fbthrift_ext depends on "fbthrift" (from git), but the structure X implements trait Y from "fbthrift" (from path) which is not the same as trait Y from "fbthrift" (from git).

Reviewed By: quark-zju

Differential Revision: D22335430

fbshipit-source-id: 9b73f4f0bc09fa79b02488dc9b4640b97f90683c
2020-07-02 03:35:02 -07:00
Arun Kulshreshtha
f45a06be6d edenapi_types: implement quickcheck::Arbitrary for request types
Summary: Add `Arbitrary` implementations for `DataRequest`, `HistoryRequest`, and `CompleteTreeRequest` so that these types can be used in quickcheck tests.

Reviewed By: quark-zju

Differential Revision: D22344600

fbshipit-source-id: c7fcbcd4648ab45f8dde00cc4bb3c1c4d203c4e1
2020-07-01 23:03:00 -07:00
Arun Kulshreshtha
e669ef9131 edenapi_types: use #[quickcheck] attribute instead of quickcheck! macro
Summary: The `#[quickcheck]` attribute is a more modern way of defining quickcheck predicates (as opposed to the older `quickcheck!` macro). Update all existing usages in the crate.

Reviewed By: quark-zju

Differential Revision: D22344601

fbshipit-source-id: b7ee4d317a64bed5fcd8b35f90f2544f6b024410
2020-07-01 23:03:00 -07:00
Arun Kulshreshtha
0f5fa921ed edenapi_types: use Context instead of anyhow! in json module
Summary: Replace use of `.ok_or_else(|| anyhow!(...))?` with `.context(...)?`, which is a cleaner way to express the same thing.

Reviewed By: quark-zju

Differential Revision: D22323298

fbshipit-source-id: 9fc1a8183a54ee0c4f30f7497b98005a18a06468
2020-07-01 23:03:00 -07:00
Arun Kulshreshtha
ddff9b1ecf edenapi_types: move JSON parsing code from make_req to edenapi_types
Summary:
Move JSON request parsing code out of `make_req` into `edenapi_types` so it can be reused in the EdenAPI test CLI (added later in the stack).

Note to reviewer: This diff look bigger than it actually is; it is mostly just cut-and-paste (recorded as a copy to preserve history).

Reviewed By: quark-zju

Differential Revision: D22305693

fbshipit-source-id: 13248fb29b2fbb705203f889f3d2fb3c1c760bed
2020-07-01 23:03:00 -07:00
Arun Kulshreshtha
632596e947 edenapi: make JSON representation of DataRequest into an object
Summary:
EdenAPI's `make_req` tools allows developers to create ad-hoc CBOR request payloads for debugging purposes (e.g., for use with `curl`). The tool generates requests from human-created JSON, which are particularly useful in Mercurial and Mononoke's integration tests.

Later in this stack, the use of this JSON format will be extended beyond just this one tool. As such, it is important that the representation be sufficiently extensible so accommodate future changes to the request structs. In the case of the JSON representation of `DataRequest`, this means changing from an array to a single-attribute object, so that additional fields can potentially be added in the future.

Reviewed By: quark-zju

Differential Revision: D22319314

fbshipit-source-id: 5931bc7ab01ca48ceab5ffd1c9177dd3035b643c
2020-07-01 23:03:00 -07:00
Arun Kulshreshtha
765a5b840a edenapi: use autocargo for everything under lib/edenapi
Summary: Use autocargo for all EdenAPI code.

Reviewed By: quark-zju

Differential Revision: D22344400

fbshipit-source-id: 522e82fcc76792ed01ca2cdbfa169c23be5bf38f
2020-07-01 23:03:00 -07:00
svcscm
b354c3609b Updating submodules
Summary:
GitHub commits:

9e26c16401
980a8d34b1
956fe7226c
86b7eb35f4
a46da10a41

Reviewed By: wittgenst

fbshipit-source-id: dc3d35880bba5e76cf452fe2be2ae0d1c45d7d33
2020-07-01 21:56:20 -07:00