Commit Graph

19921 Commits

Author SHA1 Message Date
Muir Manders
e955a5eda2 filemerge: add start of simple merge tool DSL
Summary:
A user wants to use a different merge tool for delete+modified conflicts. The current merge tools are not flexible or composable enough to handle this. For example, even if we added a ":delete-keep-local" tool, there is no way to specify the fallback merge tool to use for other kinds of conflicts.

In this diff I introduce a simple DSL to solve the above problem. I called it "merge tool script". It only supports the "if" function, and "isabsent" to test of the "local" or "other" file context is absent/deleted.

Reviewed By: zzl0

Differential Revision: D46163270

fbshipit-source-id: 28f2579e665dc9dd66ab816b10e0b7a758e772f5
2023-06-04 22:40:24 -07:00
Muir Manders
67ef3afc23 parser: factor out string parsing
Summary: Move string parsing code from revsetlang.py into the parser so other langs can use it.

Reviewed By: zzl0

Differential Revision: D46194125

fbshipit-source-id: cb8c829ceb52a829a831a33b4265410ef704fef9
2023-06-04 22:40:24 -07:00
Muir Manders
1fd96b9a1d merge: remove v1 merge state
Summary: v1 and v2 have both been written out forever, so I think we can drop the v1 format.

Reviewed By: zzl0

Differential Revision: D46075828

fbshipit-source-id: 373dae9cf6a53534489022716986aa85b5de5552
2023-06-04 21:54:48 -07:00
Shayne Fletcher
4ea1ce14a8 upgrade to parking_lot-0.12.1
Summary: upgrade parking_lot from version 0.11.2 to version 0.12.1

Reviewed By: zertosh, diliop, dtolnay

Differential Revision: D46398501

fbshipit-source-id: ffae0ea188abad5253f1524216dd18ce9a53a74c
2023-06-02 18:45:03 -07:00
Rajiv Sharma
0443bba2cf Migrate check_git_wc command to new Mononoke App
Summary: As in title

Reviewed By: mitrandir77

Differential Revision: D46354685

fbshipit-source-id: 2a3e7e86772c71aed18cc6c83c31767c7454d65c
2023-06-02 12:00:47 -07:00
Andres Suarez
51fc649c56 unfork and update toml-5
Reviewed By: jsgf

Differential Revision: D46369474

fbshipit-source-id: 2905c96f24f3a6963c69ed89572ce36e564443d6
2023-06-02 08:04:49 -07:00
Rajiv Sharma
11982fe8da Log create_bookmark error in gitimport
Summary:
In D41769964, we decided not to stop `gitimport` simply because `create-bookmark` fails cause that could be due to the fact that the bookmark being created already exists. If that is indeed the case, we try to resolve the bookmark and move it to the new value. However, there could be other unrelated reasons for `create_bookmark` to fail in which case we would still try to resolve the bookmark and end up in unexpected situations (e.g. Luibov trying to import `paws` repo faced this problem). This diff does two things:
- Performs an early check to determine if the bookmark exists by resolving it. If it does exist, it executes the move bookmark branch of code
- If the bookmark doesn't exist, then it creates it.

If there are any errors during bookmark creation or movement, they get logged for the user.

Reviewed By: mitrandir77

Differential Revision: D46071703

fbshipit-source-id: 446b0bc720d13b9e9314feccdb147b2bab0b7cd9
2023-06-02 06:52:56 -07:00
Jeremy Fitzhardinge
1ae4f69513 third-party/rust: update text_placeholder with https://github.com/bernardoamc/text-placeholder/pull/7
Summary: Added `fill_from_function` which makes it much more useful

Reviewed By: fanzeyi

Differential Revision: D46342378

fbshipit-source-id: 9bc8c01dcb10806a03f1eab628f9763c4e5edc81
2023-06-02 03:29:24 -07:00
Pierre Chevalier
c7c032c6fb non-oss work 67/n
Differential Revision: D46326058

fbshipit-source-id: 70db644c1badfcefddea80ce9961b9f547388192
2023-06-02 02:52:32 -07:00
Xavier Deguillard
b26ca0d334 store: properly time ObjectStore::getBlobMetadata
Summary:
Our telemetry was showing that starting mid-April the cost of
ObjectStore::getBlobmetadata went down significantly but investigating the root
cause of the improvement didn't seem to yield any results. It turns out the
root cause was that the code no longer timed the cost of the future.

Reviewed By: kmancini

Differential Revision: D46369667

fbshipit-source-id: 0c7d997c93d94498384e27b3447a9c736dcce3b2
2023-06-01 15:51:45 -07:00
Zhaolong Zhu
ccec2d4263 copytrace: exclude files that are not in the dest manifest
Summary:
the previous change broke `test-fb-ext-copytrace.t` tests, but somehow the failure didn't show up in the CI, this is to fix the that.

The issue happened when rebasing 8f852f229 (mv a b) to 046961699 (del a), and file `a` doesn't in the dest commit. The old logic is to keep b in the final commit, and my previous abort with an error that a is not in dest. I think a better way is to report a conflict and ask users to resolve it. But for now, I will keep the old behevior.

```
  @  046961699  72 seconds ago  zhaolong
  │  del a
  │
  │ o  8f852f229  107 seconds ago  zhaolong
  ├─╯  mv a b
  │
  o  7a737d7e6  2 minutes ago  zhaolong
╭─╯  add a
```

Reviewed By: sggutier

Differential Revision: D46357844

fbshipit-source-id: f5860c0037b909e2dd1a62700775dba505b88849
2023-06-01 10:32:04 -07:00
Saul Gutierrez
d430d13cb5 tests: move environment related utils to their own crate
Summary: The `lock_env` function from `scm/lib/config/loader/src/test_util.rs` was way too useful to leave in its own crate, since it could be useful for Rust unit tests that modified environment variables

Reviewed By: quark-zju

Differential Revision: D46133861

fbshipit-source-id: b32842c55bd7279490f69601fbb22201da7a0767
2023-06-01 05:52:37 -07:00
Saul Gutierrez
8de3e9d5d8 io: add an IOInput trait
Summary: Adds a trait similar to `IOOutput`, which can be used for determining whether the input object held by `IO` is in a tty

Reviewed By: zzl0

Differential Revision: D46129268

fbshipit-source-id: 2fe2b19baf85d6ba361e515cd9e017bda628cc60
2023-06-01 05:52:37 -07:00
Saul Gutierrez
50d51ee002 configfile: move steps for obtaining default user config file to its own function
Summary: This function will be used in later diffs for determining the default user config file

Reviewed By: muirdm

Differential Revision: D46005612

fbshipit-source-id: 52a0fcba6fbb3d47d24c759d3ff946301ccaa922
2023-06-01 05:52:37 -07:00
Zhaolong Zhu
6a94e0c7f8 copytrace: use dagcopytrace to trace renames for src missing files
Summary:
Currently, the `_forwardcopies` doesn't support git format repo, which
means copytrace can not find the renames on the source side.  So let's use
`dagcopytrace` for it as well. It also benefits hg format repos as well, since
`dagcopytrace` support content similarity based rename finder.

Reviewed By: quark-zju

Differential Revision: D46315431

fbshipit-source-id: 5f9ba4a9346f1dbd7e96731b9559dfe686ff028c
2023-05-31 21:42:03 -07:00
Zhaolong Zhu
f276a73c2e copytrace: add .t test for testing missing files in source side
Summary:
Currently, the `_forwardcopies` doesn't support git format repo, which
means copytrace can not find the renames on the source side. Add a test to
reproduce this issue.

Reviewed By: quark-zju

Differential Revision: D46315430

fbshipit-source-id: c25f8db6da121a7baa02045720b867c08845f169
2023-05-31 21:42:03 -07:00
Zhaolong Zhu
46b2feabcb copytrace: remove unnecessary pub(crate)
Summary: remove unnecessary pub(crate)

Reviewed By: quark-zju

Differential Revision: D46315432

fbshipit-source-id: 239150866b721818fa7816ac505ed3d56e4ae719
2023-05-31 21:42:03 -07:00
Xavier Deguillard
1c20e574cc telemetry: fix typo in counter name
Summary:
Both the getBlobMetadata and fetchBlobMetadata were publishing to the same counter, making them indistinguishable.

Created from CodeHub with https://fburl.com/edit-in-codehub

Reviewed By: genevievehelsel

Differential Revision: D46343472

fbshipit-source-id: 04e09372c21dd5cb09336259ff34a5394d10d3b4
2023-05-31 20:38:26 -07:00
Liubov Dmitrieva
45565b3045 add new ods logging for bookmark fetching from phases crate
Summary:
add new ods logging for bookmark fetching from phases crate

This would help with investigation sevs like S344849, also having this info we
can setup alerts.

Reviewed By: YousefSalama

Differential Revision: D46314640

fbshipit-source-id: 66e72da19ad4eb528c78d0783c297746f57cd4f1
2023-05-31 13:57:24 -07:00
Xavier Deguillard
61b02f8f86 Back out "linearize control flow in EdenServer::mount"
Summary:
We've seen crashes on Windows when the mount is initialized but a later step at
mount time fails. In between these 2 events, if a write to the working copy
happens the ProjectedFS code assumes that the InodeMap is initialized and thus
tries to access the rootInode. This assumption is unfortunately broken by the
call to mountFinished and the code should instead follow the regular unmount
code path to make sure that no pending notifications are inflight.

I suspect NFS and FUSE also would exhibit the same issue.

Reviewed By: kmancini

Differential Revision: D46169999

fbshipit-source-id: ed1eb17dc2f084acb7e2d88c555150b3b8c6662e
2023-05-31 12:15:03 -07:00
Pierre Chevalier
2db9d7227d Enable shard manager to load unredacted repos
Summary:
This offers a  new interface for Shard Manager managed repos to be able to load redacted content.

The new API entry point: `open_managed_repos_unredacted` is analogous to `open_repo_unredacted`, but supports a Shard Manager based workflow.

Reviewed By: RajivTS

Differential Revision: D46280863

fbshipit-source-id: e2e4aebcb08d01f3be97d11d6fe8da2822ba6f29
2023-05-31 04:50:12 -07:00
Pierre Chevalier
4c8b8f43b6 non-oss work 61/n
Differential Revision: D46228970

fbshipit-source-id: 81a0b823ef88e27c324f16de412fce0d10bf7633
2023-05-31 04:50:12 -07:00
Barys Skarabahaty
cecd3a9582 Adding a possibility to configure the blake3 key
Summary: Adding a way to configure the blake3 key for blake3 hashing

Reviewed By: chadaustin

Differential Revision: D44499469

fbshipit-source-id: 344edd91cfc867c8a49d0db9369dfe2132e86c77
2023-05-30 23:10:10 -07:00
Zhaolong Zhu
3bc3732b8d copytrace: add metrics for content similarity fallback result
Summary: I am planning to enable `copytrace.fallback-to-content-similarity=True` for hg repos, and will use this metric to monitor how useful the content similarity fallback is

Reviewed By: muirdm

Differential Revision: D46163311

fbshipit-source-id: d21b0288965f8ec4da7c6c80e37d1b557b357ea6
2023-05-30 20:32:55 -07:00
Zhaolong Zhu
19d7f8214c copytrace: make dagcopytrace instance a singleton
Summary: Our cache is in the dagcopytrace instance, so let's make it a singleton.

Reviewed By: quark-zju

Differential Revision: D46105882

fbshipit-source-id: 0984015e7b052482b27f950fc05563c432aa23b1
2023-05-30 20:32:55 -07:00
Zhaolong Zhu
d9171bcf51 copytrace: add batch move candidates cache for rename finders
Summary: Optimization for directory move caused renames.

Reviewed By: muirdm

Differential Revision: D46105883

fbshipit-source-id: 69aee540b14cde9112f6327b6c782714109cac04
2023-05-30 20:32:55 -07:00
Zhaolong Zhu
875c353d48 copytrace: add test for batch move
Summary: Add tests for dir move

Reviewed By: muirdm

Differential Revision: D46149478

fbshipit-source-id: 28d98a33ac2f1c47e407f744ecfa496e4376e4bd
2023-05-30 20:32:55 -07:00
Zhaolong Zhu
96f993dcae copytrace: add batch move detection
Summary: this function will be used in the following diff

Reviewed By: quark-zju

Differential Revision: D46105885

fbshipit-source-id: 9d02fb62a44b1e5bd18eba5236522e297fae0f64
2023-05-30 20:32:55 -07:00
Rajiv Sharma
af2314d831 Using BonsaiTagMapping in GitImport and RemoteGitImport
Summary:
This diff modifies the behavior of `gitimport` and `remote-gitimport` when importing `annotated-tags`. In addition to:
- storing the raw tag
- converting the tag to a bookmark
- storing the metadata of the tag as changeset
this diff maintains a mapping between the tag-name and the changeset representing the metadata of the tag. This mapping would be useful to answer queries about the tag metadata when queried through Git and also enable going back to Git from Mononoke without losing data.

Differential Revision: D46067809

fbshipit-source-id: e2463160cdf647b50fa4ecfe4674d079016b4adc
2023-05-30 12:20:40 -07:00
Youssef Ibrahim
79fd5fd16a mononoke_api: implement an alternative to ChangesetPathHistoryContext::history using the new commit graph
Summary: Implement an alternative to ChangesetPathHistory::history using the new commit graph. This is almost identical to the skiplist implementation except that an (incorrect) optimization was removed that was trying to minimize the number of is_ancestor queries which should no longer be needed as they are a lot faster with the new commit graph.

Differential Revision: D46109911

fbshipit-source-id: 3e8f0c36a48084116673dda3741de87e0741f1a2
2023-05-30 11:47:19 -07:00
Youssef Ibrahim
81063c4c9d mononoke_api: implement an alternative to ChangesetContext::history using the new commit graph
Summary: Implement an alternative to ChangesetContext::history using the new commit graph by using AncestorsStreamBuilder. Also made the tests not depend on the exact output history order but instead accept any topological order.

Differential Revision: D46109910

fbshipit-source-id: 31f3e219e9addf04b49fc500ad245d541f110887
2023-05-30 11:47:19 -07:00
Youssef Ibrahim
899a3aed12 commit_graph: implement AncestorsStreamBuilder
Summary:
Factors out the logic of creating an ancestors stream to a builder to allow for more customizability (required for ChangesetContext::history) and reduce code duplication.
Additionally this optimizes range_stream to only traverse changesets that are descendants of start_id. Ancestry checks are kept to a minimum (worst case O(merges)) by only checking on parents of changesets that have a skip tree parent with generation lower than the generation of start_id.

Reviewed By: liubov-dmitrieva

Differential Revision: D46276429

fbshipit-source-id: 65c4f2892c92baaab0fef4f5041f06095ec49e05
2023-05-30 11:47:19 -07:00
Egor Tkachenko
7ee654f3e8 Add facet to interact with deletion log db
Summary: Facet and related boilerplate to construct it

Differential Revision: D46153855

fbshipit-source-id: 480d69deb4ff714541970e63df775a500e2cbb33
2023-05-30 02:01:31 -07:00
Genevieve (Genna) Helsel
d3ae451c74 allow using SqliteOverlay on non-Windows
Summary: allow using `SqliteOverlay` on non-Windows, since  OverlayChecker has now been updated to work with it.

Reviewed By: chadaustin

Differential Revision: D44105360

fbshipit-source-id: 5f78e6c9b5323faeee394da4a51d7e84f8d44a2f
2023-05-29 22:38:14 -07:00
Xavier Deguillard
71eebd2356 store: fix a nullptr derefence in LocalStoreCachedBackingStore
Summary:
In the rare case where fetching both the BlobMetadata and the Blob failed, the
LocalStoreCachedBackingStore would still try to dereference the nullptr Blob,
leading to a crash.

Reviewed By: chadaustin

Differential Revision: D46211469

fbshipit-source-id: f98dd215dbda432507f3823439d354d3d662dc6f
2023-05-26 09:16:16 -07:00
Youssef Ibrahim
3febd2a8eb commit_graph: add a newadmin command for finding children of a commit
Summary: We didn't have a way to do this before so let's add it.

Reviewed By: mitrandir77

Differential Revision: D46190481

fbshipit-source-id: 84544f0226ef3a4ef4d8ac9b4bf66551e4671793
2023-05-26 04:38:42 -07:00
Jan Mazur
79a82015aa remove dependency on MononokeApp
Summary: It's not ideal that we pull mononoke_app and even instantiate MononokeApp here. This is an attempt to decouple Mononoke from Metagit.

Reviewed By: mitrandir77

Differential Revision: D45862434

fbshipit-source-id: 6eab3577743810461f22535d89cf9a00dc1cbb65
2023-05-26 04:22:23 -07:00
Youssef Ibrahim
4634db2c84 commit_graph: asyncify closures in _with methods
Summary: Most of the useful ways to use these methods requires providing an async closure so let's asyncify them.

Reviewed By: RajivTS

Differential Revision: D45607981

fbshipit-source-id: ee7b471cc5d60a793a79928f6c15c48c0da1d08e
2023-05-26 02:12:44 -07:00
Zhaolong Zhu
37913dc1a8 fbscmquery: avoid scanning the repo when revision translation is failed
Summary: Currently, the code scans the repo when revision translation is failed, which is very slow.

Reviewed By: muirdm

Differential Revision: D46200075

fbshipit-source-id: 31eaca43582b480a0b3abd7af0b27ec8d6406195
2023-05-25 18:37:50 -07:00
Zhaolong Zhu
5af2319270 copytrace: abstract generate_candidates
Summary:
We get both `added` and `deleted` files in the function, this will be
needed for directory move detection.

Reviewed By: muirdm

Differential Revision: D46105884

fbshipit-source-id: d0081f8afaf27ef4ab105ab7c58b3f5cc694debc
2023-05-25 15:58:54 -07:00
Xavier Deguillard
fdbf7eadfb store: add telemetry for LocalStore operations
Summary:
We've seen a case where a large `hg update` was taking an absurdly long time in
`ObjectStore::getTree` but the telemetry was showing us that most of the time
wasn't spent fetching trees from Mercurial! The suspicion is that most of the
time was spent in the LocalStore but with no evidence to prove it.

Let's thus add some timing telemetry to various LocalStore read requests to
fill this gap.

Reviewed By: mshroyer

Differential Revision: D46154456

fbshipit-source-id: b439ac48889ed3db71db136ff6c1cc91f48c926a
2023-05-25 15:48:11 -07:00
Genevieve (Genna) Helsel
006d26a094 extend FsckTest to test SqliteOverlay
Summary:
Extends `FsckTest` to test `SqliteOverlay`. Two major changes were made to non-test files:
1) `loadInodeInfo` was moved from `OverlayChecker` into `InodeCatalog`
2) `getAllParentInodeNumbers` was added to facilitate `OverlayChecker::readInodes`

Reviewed By: chadaustin

Differential Revision: D45447172

fbshipit-source-id: 9ad291a0c4c34f1e44e1f05c1fdbf2ac7ea7d309
2023-05-25 13:50:56 -07:00
Youssef Ibrahim
84661b9dd0 commit_graph: add a sleep in update-preloaded command to prevent exiting before writing to all inner blobstores
Summary: In the case of a multiplexed blobstore, the put operation can exit after it succeeds in one inner blobstore before finishing in all, and leave the rest running in the background. This sleep tries to prevent exiting early before they all finish.

Reviewed By: mitrandir77

Differential Revision: D46190037

fbshipit-source-id: ac2ae0d1453ebdc923170b9da7ca98df7519bf52
2023-05-25 11:00:34 -07:00
Genevieve (Genna) Helsel
e95b17e5a6 move OverlayChecker into eden/fs/inodes/overlay
Summary: This will no longer be tied to the fscatalog, so it makes more sense to live in a more general location

Reviewed By: chadaustin

Differential Revision: D44980396

fbshipit-source-id: 3cabf00d87df7eb00671c2b82b04c8f12283e394
2023-05-25 10:10:08 -07:00
Youssef Ibrahim
8427780527 commit_graph: implement fetching children of a changeset
Summary: Implements a public method changeset_children in CommitGraph that fetches the changeset ids of the children of a given changeset.

Differential Revision: D46185434

fbshipit-source-id: f0b2e2ee53c7a34fe94ee5ef1f74f6e31429d056
2023-05-25 09:32:10 -07:00
Liubov Dmitrieva
8a3498edfa used cached version of API
Summary: used cached version of API

Reviewed By: YousefSalama

Differential Revision: D46184380

fbshipit-source-id: 5deb8c455ad682afcf1f0cad822d34fdd632b88b
2023-05-25 06:40:27 -07:00
Xavier Deguillard
e73ab91be0 store: add telemetry for where Tree were obtained from
Summary:
Telemetry exist for where Blob and BlobMetadata are coming from, but not for
Tree. Let's bring the Tree telemetry up to par.

Reviewed By: genevievehelsel

Differential Revision: D46154455

fbshipit-source-id: ab3c31d55b6a91009289b3b07853fa574bbaa137
2023-05-24 17:08:42 -07:00
Katie Mancini
5dac894df8 remove debugGetScmBlobMetadata
Summary:
I migrated the last caller of debugGetScmBlobMetadata to debugGetBlobMetadata
in the last diff. Let's delete the old method.

I am not really concerned about waiting for that lower diff to rollout and all
because this is a debugging command that we only sparingly ask users to run
(As far as I know like maybe 3 times).

Reviewed By: chadaustin

Differential Revision: D45972617

fbshipit-source-id: 6d4689a225fee972f8e3a802115852043f0bade8
2023-05-24 16:41:02 -07:00
Katie Mancini
5458cd7045 migrate last debugGetScmBlobMetadata call to debugGetBlobMetadata
Summary:
I created debugGetBlobMetadata a while ago and this is the last caller of
debugGetScmBlobMetadata. Let's migrate it to the new endpoint so we can delete
the old one.

Reviewed By: chadaustin

Differential Revision: D45972618

fbshipit-source-id: cbb0977260f9a1543d3bf0a31bf7b948c2f35143
2023-05-24 16:41:02 -07:00
Katie Mancini
b981977a84 remove debugGetScmBlob
Summary:
I created debugGetBlob a while ago and there are no more callers of
debugGetScmBlob from code search, so let's delete this.

Reviewed By: chadaustin

Differential Revision: D45972619

fbshipit-source-id: 97276ec2182fbe7d9022a06e9a0df34ae4e80235
2023-05-24 16:41:02 -07:00