Commit Graph

5740 Commits

Author SHA1 Message Date
Jun Wu
38e3f82998 pydag: expose NameSet::to_parents
Summary: Expose the `NameSet::to_parents` API so it can be used in Python.

Differential Revision: D34017427

fbshipit-source-id: 9a602031eb9f709f43d89a0cd23448d9b8285021
2022-02-11 22:50:44 -08:00
Jun Wu
5cdc0e9e4d dag: add NameSet::to_parents API
Summary:
A nameset is a "flat" set without graph (parents) informatin.

In certain cases (ex. graph log, annotate) the parents information
is needed. Add an API for that.

This might potentially replace some logic in Python `graphmod.py`.

Differential Revision: D34017425

fbshipit-source-id: 38d0bccb5c455cd9ec2d60113c18207a90645de9
2022-02-11 22:50:44 -08:00
Durham Goode
d2b765c164 nativecheckout: don't remove files that didn't exist before
Summary:
When doing a clone, the nativecheckout action plan included a remove
action for every file not in the sparse checkout. This is because it didn't
account for the file not existing in the old manifest and just blindly queued
the remove.

On Linux this is pretty fast, but on Windows this is extremely slow to do for
every path. It's probably slow because of NTFS, but also because we do some path
auditing logic for Windows and Mac.

This fixes it to not do remove for a file that wasn't in the old manifest.

Reviewed By: sggutier

Differential Revision: D34188271

fbshipit-source-id: 714b2dcf98cfa9ebec287f00d7cd5397126f693e
2022-02-11 17:14:45 -08:00
Durham Goode
9a766c1fdb scmstore: add new debug tracing
Summary:
The old tracing info was noisy and hard to follow. The new one is much
more concise and provides useful insight into what is being fetched, at what
stage, and what errors happen at each stage.

Reviewed By: quark-zju

Differential Revision: D34159138

fbshipit-source-id: e1de87de9f9f64deb7e33ef76e6441bb6721f9b9
2022-02-11 17:11:31 -08:00
Durham Goode
48bf24135a scmstore: remove instrumenting from methods
Summary:
These produce a ton of hard to read trace output, with enter/exit's
everywhere. Let's delete them. In the next diff we'll add more human readable
explanations of what's going on in scmstore.

Reviewed By: quark-zju

Differential Revision: D34159139

fbshipit-source-id: 48d1175edd20d3de952f12dc8fcf7a6661469d64
2022-02-11 17:11:31 -08:00
Durham Goode
790c4809f3 scmstore: don't write lfs pointers via the contentstore
Summary:
scmstore was writing lfs pointers via the contentstore. This meant that
when we disabled the contentstore fallback, this path failed. This path is used
during unbundle (like in 'jf get') when the bundle contains an lfs pointer.

Reviewed By: quark-zju

Differential Revision: D34127065

fbshipit-source-id: d50245b78dd87e6c71d8775123fe954084289e2b
2022-02-11 15:18:13 -08:00
Saul Gutierrez
72c39ddacf init: make directory creation recursive
Summary:
When initializing a new repo in the old new repo initialization, directories would be created recursively if the target path did not exist. This diff fixes the new repo initialization not doing that.

The unit test for directory creation was also reworked.

Reviewed By: quark-zju

Differential Revision: D34093210

fbshipit-source-id: 3aa0aade9bd3034448615678a005966a7da6ef5c
2022-02-11 11:22:19 -08:00
Saul Gutierrez
ad54dbb12a init: store errors in InitError instead of strings
Summary: Storing errors instead of their string representation should be much more useful for showing errors. This diff does that for `InitError`.

Reviewed By: quark-zju

Differential Revision: D34161428

fbshipit-source-id: 4f77428ef379db6b42b492bb5424a8fdd04b954a
2022-02-11 11:22:19 -08:00
Muir Manders
1884a15a19 lfs: add some error code counters
Summary:
Add some counters to track lfs transient/fatal error response codes:
- "lfs.transient_error.{method}.{code}" counts error codes we got but ended up
  succeeding after retries
- "lfs.fatal_error.{method}.{code}" counts errors we got and then ended up not
  succeeding
- "lfs.success.{method}" counts successful queries

Reviewed By: quark-zju

Differential Revision: D33962157

fbshipit-source-id: 901625eb72cac2eba57f4b2424a778f4f8ac1779
2022-02-11 09:41:10 -08:00
Durham Goode
16af96aba7 lfs: remove placeholder feature
Summary:
This feature is only used for e-discovery, and we don't need to build a
new e-discovery package from eden/scm anymore. Let's remove this feature since
the monkeypatch interferes with updating the hg status paths.

Reviewed By: quark-zju

Differential Revision: D34124434

fbshipit-source-id: b41382bc1a3cfeda82e1dd2af511244ad16a9fb8
2022-02-10 14:13:09 -08:00
Steven Troxler
09c0c04351 Convert type comments for eden
Summary:
Convert type comments to type annotations.

Produced by running:
```
python -m  libcst.tool codemod convert_type_comments.ConvertTypeComment eden
```
from a parent directory

Reviewed By: shannonzhu

Differential Revision: D34110680

fbshipit-source-id: 23081a8867b93cd349271075428176af514efa2b
2022-02-10 08:48:35 -08:00
Steven Troxler
85d70eb757 Convert type comments in scm/edenscm/hgext
Summary:
Convert type comments to type annotations in eden/scm/edenscm/hgext

Produced by running:
```
python -m  libcst.tool codemod convert_type_comments.ConvertTypeComment eden/scm/edenscm/hgext
```
from a parent directory.

Reviewed By: grievejia

Differential Revision: D34110682

fbshipit-source-id: acf8d32290411ca8673904e4f80ddd846ebceef0
2022-02-09 14:29:43 -08:00
Steven Troxler
f4abbaf4d6 Convert type comments in scm/edenscm/mercurial
Summary:
Convert type comments to type annotations in eden/scm/edenscm/mercurial

Produced by running:
```
python -m  libcst.tool codemod convert_type_comments.ConvertTypeComment eden/scm/edenscm/mercurial
```
from a parent directory.

Reviewed By: grievejia

Differential Revision: D34110681

fbshipit-source-id: 3f21aa39de9384cdf7d5b09a232d9605eadc1481
2022-02-09 14:29:43 -08:00
Muir Manders
027d05ba38 snapshot: remove existing file before writing symlink
Summary: Support the case of a snapshot modifying a symlink by removing the symlink destination before trying to create the symlink.

Reviewed By: yancouto

Differential Revision: D34097684

fbshipit-source-id: 48d4a05f61a69e686e5bd530a7ed150326a32765
2022-02-09 12:32:57 -08:00
Alex Hornby
f628e4793f remove generated thrift code and fix external CI (#26)
Summary:
Pull Request resolved: https://github.com/facebookexperimental/rust-shed/pull/26

Pull Request resolved: https://github.com/facebookexperimental/eden/pull/104

No need for the checked in generated code anymore

Successful external CI Linux run on Git Hub PR: https://github.com/facebookexperimental/eden/runs/5130405984?check_suite_focus=true

Mac PR run fails with a SSL cert issue, will look separately

Reviewed By: chadaustin, mitrandir77

Differential Revision: D33840545

fbshipit-source-id: eafc2a0e2191d438fd828adeebc2e318d319025f
2022-02-09 12:21:32 -08:00
Facebook Community Bot
20a78c4882 Re-sync with internal repository 2022-02-09 18:47:06 +00:00
Muir Manders
115656733b vfs: chmod lock files to 0666 on unix
Summary: Unlike the python repo lock files, the rust lock files are not deleted when the lock is released. That means if the locks are created as root, they will break future commands run as other users. Try to mitigate that eventuality by chmoding the lock files to 0o666 on unix.

Reviewed By: quark-zju

Differential Revision: D34016719

fbshipit-source-id: 2bd1ec9d5da0e27a12eb95da5c2a4cb3f6286137
2022-02-09 07:53:57 -08:00
Andres Suarez
68955fe35b Update async-trait to 0.1.52
Summary:
Fixes some clippy issues:
685b68b090

Reviewed By: dtolnay

Differential Revision: D34100067

fbshipit-source-id: fa49e68645b6e99d87f92a7b9ac73ba5d567fa0b
2022-02-09 02:05:17 -08:00
David Tolnay
215f6d4249 third-party/rust: Import forked terminfo crate
Summary:
The published `terminfo` crate has a WTFPL license which is forbidden from being used by Meta.

{F687594009}

However the WTFPL allows forking and relicensing, so I've gone ahead and relicensed the same code as dual MIT OR Apache 2.0 for fbsource to use. We can patch the properly licensed code in place of all transitive dependencies on the `terminfo` crate, as they have the same public Rust API.

The relevant changes: 9b4e1c6aae

Reviewed By: metajack

Differential Revision: D33190944

fbshipit-source-id: 0a9002b8799b5ceeaca3c20a211f15beec5a407f
2022-02-08 15:32:51 -08:00
Wez Furlong
fb40a9c55f Back out "Bump parking-lot to version 0.12.0"
Summary:
Original commit changeset: ce571ef50289

Original Phabricator Diff: D33985872 (c94dada73b)

Reviewed By: dtolnay

Differential Revision: D34060091

fbshipit-source-id: c1ddcdd1848a51f83dcbc9b2eec5f316bb6e44e5
2022-02-08 11:37:32 -08:00
Jan Mazur
7f1ba5f409 print headers on unrecovearable error
Summary: Sometimes we can get errors, debugging info, or advices in headers returned from services and proxies. We know some headers that we can parse, but we don't have a comprehensive list. To improve debugging we might as well just print all headers of a response on failure. As of now we don't handle that in one place (like D33918785), because technically HTTP 5xx is a valid response from the POV of http library. It's the caller's logic that need to decide if it's a failure or not. In the future we could perhaps have such debugging logic in a trait like `ResponseExt` that could have `fn debug_failed_req` to be called by multiple callers if they decide a particular request was indeed an unrecoverable failure.

Reviewed By: quark-zju

Differential Revision: D34035084

fbshipit-source-id: 8247462ed9875b6530f215fb161c311325ea83fe
2022-02-08 11:04:29 -08:00
Jae Jang
63ba8c7d8d Add bubble id to FetchSnapshotResponse and store it on client for snapshot update
Summary: Added bubble id to FetchSnapshotResponse. Also updated hg client to store latest bubble information when updating snapshot.

Reviewed By: yancouto

Differential Revision: D34042287

fbshipit-source-id: 3c206caba4fa7b833cf1f8e23123b5b6006e2890
2022-02-08 08:30:18 -08:00
Yan Soares Couto
3312f2c3cb ignore file not found when creating a snapshot
Summary:
Sometimes hg might recognise an untracked file as being added, but by the time it tries to load it, the file is gone, causing a crash.

For snapshots, it should be fine in this case to simply ignore this file. That is what this diff does. It specifically ignores `FileNotFoundError`, meaning other files still get classified as errors and we're not swallowing anything we shouldn't be.

Reviewed By: mitrandir77

Differential Revision: D34043083

fbshipit-source-id: cec7d528652c1704f3a3db707eafc0e43f24ac74
2022-02-08 07:00:28 -08:00
Yan Soares Couto
be5ddc9607 Add context to failure to create snapshots
Summary:
This just adds a `with_context` call to when we're loading files from disk, so it prints which file failed to load.

I've seen several `[Errno 2] No such file or directory (os error 2)` and they're really annoying.

To do this, I had to refactor the lambda into a function, because try blocks are not stable yet :(.

Reviewed By: markbt

Differential Revision: D34043084

fbshipit-source-id: e0fd4142d75d6b392440f7763217a9b23ee8c8a4
2022-02-08 07:00:28 -08:00
Alex Hornby
5a096bd67c move autocargo up one level (#101)
Summary:
Pull Request resolved: https://github.com/facebookexperimental/eden/pull/101

public_autocargo already existed for edenfs, but needed to move up a level so to make edenfs thrift definition visible via autocargo.

Reviewed By: markbt

Differential Revision: D32693522

fbshipit-source-id: 4c070cfa2d6294a28c17697338f0309dcab1576f
2022-02-08 06:07:52 -08:00
Rajiv Sharma
6d2c30754d morestatus: included commit details for merge conflict during rebase
Summary: Used the dirstate object to pull the current rebase related commits. If there exists a merge conflict, the commits are included in the error message (otherwise no change [e.g. non-conflicting interrupted rebase])

Reviewed By: quark-zju

Differential Revision: D34022102

fbshipit-source-id: 51bfacaf79cac5d4df4cdb99f8c92a493972ffd1
2022-02-08 04:58:44 -08:00
Alex Hornby
f354098a9d generate the eden_scm github actions (#103)
Summary:
Pull Request resolved: https://github.com/facebookexperimental/eden/pull/103

Automate maintenance of the edenscm_* github actions yamls

Add job file and name options and support for the Rust install section

Reviewed By: fanzeyi

Differential Revision: D34044422

fbshipit-source-id: 7d5f07d37bab1eff5de30a88e710dbf7479ca192
2022-02-08 03:01:23 -08:00
Jun Wu
f66036587a dag: add tests about resizing flat segments
Summary:
Now both IdDag stores have remove_flat_segment implemented, let's test the
resize feature based on remove_flat_segment.

Reviewed By: DurhamG

Differential Revision: D33821387

fbshipit-source-id: 5b92adadf8cc6989298b2637e3fb7141898c4503
2022-02-07 17:20:11 -08:00
Jun Wu
139553af9c dag: implement IndexedLogStore::remove_flat_segment
Summary:
See previous diffs for context. Implements segment deletion for
`IndexedLogStore`.

Differential Revision: D33821380

fbshipit-source-id: fe90988ec31143b3670f9277aac85b1125c4f0a5
2022-02-07 17:20:11 -08:00
Jun Wu
d46344fb83 dag: drop group from index_parent_key
Summary:
The `index_parent_key` function can derive `group` from its `child_id`
parameter. So let's drop the redundant parameter.

Reviewed By: DurhamG

Differential Revision: D33821382

fbshipit-source-id: 3a31c3fa5d4049ec36974421c21347cbdc6c473c
2022-02-07 17:20:11 -08:00
Jun Wu
f613512245 dag: move StoreId to in_process_store module
Summary:
The `StoreId` is an implementation detail of `InProcessStore`. It does
not apply to other kind of stores. Move it to a better place.

Reviewed By: yancouto

Differential Revision: D33821388

fbshipit-source-id: d7d20d9a61ebeea58e22f3f779fc9651097857df
2022-02-07 17:20:11 -08:00
Jun Wu
785a58de70 dag: move some assertions in log definition to only run once
Summary: Previously the assertions run per entry. Make them only run once.

Reviewed By: yancouto

Differential Revision: D33821389

fbshipit-source-id: 6a7380071a3f014d2034dfbc5760f94bd6d99dc2
2022-02-07 17:20:11 -08:00
Jun Wu
f084fec15b dag: implement InProcessStore::remove_flat_segment
Summary:
See previous diff for context. Implements segment deletion for
`InProcessStore`. Extra care needs to be taken for serialization.

Reviewed By: DurhamG

Differential Revision: D33821381

fbshipit-source-id: ace59345425988c2dae30b9bd55388679c36ae63
2022-02-07 17:20:10 -08:00
Jun Wu
d2d8382e75 dag: define IdDagStore::{resize,remove}_flat_segment
Summary:
See D33779403 for context. This is a step forward to implement "strip".
This diff defines the API that can be used to remove or "shrink"
a flat segment and adds some sanity checks.

The actual implementation is in upcoming diffs.
Testing will focus on "remove" first, "resize" tests will be added later.

Differential Revision: D33821386

fbshipit-source-id: 61848593e698fe7f9358b25a4cb72788637c50a5
2022-02-07 17:20:10 -08:00
Jun Wu
6589bf1187 dag: drop SegmentWithWrongHead
Summary:
See D31146158 (39f61996c7) for reasons why SegmentWithWrongHead exists.

The tl;dr is the parent->child index in the indexedlog implementation (key:
(parent, child), value: segments) can point to old segments that were replaced.

Let's ignore the "segments" from the index and just do an extra lookup
using "child" to find the up-to-date (replaced) segment. This simplifies
upcoming changes and provides better error messages in some cases.

Differential Revision: D33821383

fbshipit-source-id: 3f63ac0e3a3953d0ddcccebe2be4de2a3a3e0589
2022-02-07 17:20:10 -08:00
Jun Wu
0468989f3a dag: iter_master_flat_segments_with_parent_span => iter_flat_segments_with_parent_span
Summary:
Upcoming changes need the ability to query ranged parent->child for the entire
graph, not just the master group. Update the range query API to do that.

The only user of `iter_master_flat_segments_with_parent_span` is
`IdDag::to_first_ancestor_nth_known_universally_with_errors` and it is fine with
the change. Even better, by returning non-master segments we can answer the
`to_first_ancestor_nth` query with non-master ids. That makes the server more
flexible, as shown by the added `test_lazy_hash_on_non_master_group` test.

Differential Revision: D33821379

fbshipit-source-id: 330bd84f8535f857f6306f9ce791066688a39d48
2022-02-07 17:20:10 -08:00
Jun Wu
eb6d192f0e dag: support range deletion for IdMap
Summary: See the previous diff for context. Add `IdMapWrite::remove_range` to remove range of entries in an IdMap.

Differential Revision: D33779404

fbshipit-source-id: 24ede5949ad3c1cb0b93c3c64ca692c04b354457
2022-02-07 17:20:10 -08:00
Jun Wu
a8c32280cc dag: extend parent index key to include child
Summary:
By changing the index key it allows us to delete specific parent->child
entries in the index without affecting parent->other_child indexes.

Long story:

This is the first step to implement in-place "strip".

Having a proper way to "strip" unblocks:
- Repalce costly "rebuild_non_master" with more precisely removals without
  rebuilding anything.
- Remove problematic commits in the master group to work better with stripped
  heads server-side.
- Remove broken commits in the non-master group which lost corrosponding
  data (ex. commit message, or trees).

Previously there were a few blockers to "strip":
- IndexedLog is append-only.
- Segments must be continuous.

Now the blockers are "mostly" clear:
- IndexedLog's Index supports removal by key.
- Discontinuous segments supports punching "hole"s.

Once problem about the index removal is the parent index. Previously it
is a single key => multiple entries index. For example, parent 6 might
have children [7, 8, 9]. This makes it harder to just remove child 8
without affecting children 7 or 9. By making the index key contain
child. The index will look like (6, 7) => [7], (6, 8 ) => [8], (6. 9)
=> [9] and it's easier to remove just (6, 8 ) without affecting others.

Note the values in indexedlog is a linked list so it's in theory possible
to make indexedlog support removing just part of the values. But at this
point I'd like to implement it without changing indexedlog. We can always
go back and revisit the indexedlog fancier index removal later.

Differential Revision: D33779403

fbshipit-source-id: 4d73e7bdc66d3a8ec1482fd25f8ed5b9cc301aec
2022-02-07 17:20:10 -08:00
Saul Gutierrez
27b158897c init: integrate Rust repo initialization into the init command
Summary:
This makes use of the new Rust repo initialization added in previous commits into the `init` command in Python by replacing the initialization that was done mostly by the `localrepo` object. The codepaths for `git` and `bundle` remain active.

Another smaller addition was the option of removing `narrowheads` from the list of requirements if `experimental.narrow-heads` was explicitly disabled in the configurations.

Reviewed By: quark-zju

Differential Revision: D33901725

fbshipit-source-id: d9d976a1b844e1dd6a5e065f8ec8bf1d04fc4c03
2022-02-07 14:18:40 -08:00
Durham Goode
6917a8a5f7 revset: account for visilibity during x~-1 revset
Summary:
'x~-1' should be equivalent to 'children(X)' when there is one visible
child. Today, ~-1 doesn't take visibility into account, so if there are
invisible descendants it will error out.

This diff makes ~-1 only return visible results, which matches the children
behavior.

Reviewed By: quark-zju

Differential Revision: D33900217

fbshipit-source-id: 0b4bb015c03fa3498746e6a754a522693d0c2edf
2022-02-07 12:15:35 -08:00
Mofei Zhang
c94dada73b Bump parking-lot to version 0.12.0
Summary:
~~~Added new parking-lot version named parking_lot_0_12~~~
allow-large-files

previous versions of parking-lot are incompatible with wasm compilation targets

https://github.com/rustwasm/wasm-bindgen/issues/2160
https://github.com/Amanieu/parking_lot/pull/302

Reviewed By: dtolnay

Differential Revision: D33985872

fbshipit-source-id: ce571ef50289fe97ff89dbf72d2c7f39fbd47758
2022-02-07 05:21:28 -08:00
Facebook Community Bot
9728bf52e9 Re-sync with internal repository 2022-02-07 12:59:09 +00:00
Durham Goode
693dbc7df6 lfs: increase concurrentfetches to 4
Summary:
Prefetch code paths execute the prefetch as one huge batch, and it
turns out the lfs fetcher only fetches one file at a time. Let's increase the
parallelism

Note, checkout is less affected by this, since nativecheckout issues prefetches in smaller batches and in parallel, so we get some parallelism at a higher layer, but I do see a 15% speed up in a large file heavy repository.

Reviewed By: quark-zju

Differential Revision: D34020837

fbshipit-source-id: fb11c1aebf175b49563f48cb20f8b7c527fa3019
2022-02-04 22:25:58 -08:00
Alex Hornby
5e88a50b53 autocargo: bring fb303 repo location in sync with getdeps
Summary: It moved from facebookexperimental to facebook sometime ago

Reviewed By: fanzeyi

Differential Revision: D34009265

fbshipit-source-id: c11ca0e92d9574aa8eccd31d578b9f61a411833c
2022-02-04 14:15:48 -08:00
Jun Wu
c4334d4e7b git: normalize reference during unbundle
Summary:
Certain git bundles might contain references like "BUNDLE_HEAD" that would be
ignored by usual paths. Normalize them to visibleheads refs.

Differential Revision: D33968793

fbshipit-source-id: a3d6fc1a7d73328cddf0ebddfe7b7a021d929528
2022-02-03 15:20:56 -08:00
Yipu Miao
96c39a9540 Update eden rust files
Summary: This diff is to update the rust thrift file. Generated by cargo build.

Reviewed By: fanzeyi

Differential Revision: D33289588

fbshipit-source-id: f26a79ae60eadf2671ca0e06bcdb062fe583de17
2022-02-03 13:59:58 -08:00
Muir Manders
710f1afbce tracing: disable colors if not tty or colors config disabled
Summary: Respect our "can_color()" method to determine if tracing output should be ansified or not.

Reviewed By: DurhamG

Differential Revision: D33985125

fbshipit-source-id: c025c48e40945f9dc0c51d928f663936f582e4d2
2022-02-03 13:55:08 -08:00
Alex Hornby
f46d0814cc eden/scm/lib: add public_autocargo mapping
Summary: Add public autocargo mapping. Actual change is to eden_scm_opensource.toml and manifests/eden_scm, rest is from autocargo run.

Reviewed By: DurhamG

Differential Revision: D33896971

fbshipit-source-id: 19d75be205d9cb0179c99bd61f1be5500ba61129
2022-02-03 13:55:07 -08:00
Facebook Community Bot
88edb37132 Re-sync with internal repository 2022-02-03 21:43:02 +00:00
Alex Hornby
63109b9ccd eden/scm/lib: move rust_version target
Summary: scm/lib/version is needed by scm/lib/configparser. Move some targets so that they are visible to autocargo of scm/lib for the next diff.  Moving the target saves needing to autocargo all of scm.

Reviewed By: quark-zju

Differential Revision: D33896972

fbshipit-source-id: f055b714675e0a85aac64defacc4ca54e16fa5ce
2022-02-03 02:15:16 -08:00
CodemodService Bot
eb60f03626 Daily common/rust/cargo_from_buck/bin/autocargo
Reviewed By: krallin

Differential Revision: D33972281

fbshipit-source-id: 3e1fd5ba826bc17e9b3245a5edb7890b6f33c935
2022-02-03 01:18:05 -08:00
Carolyn Busch
bf28410902 edenapi: fix hex conversion crash
Summary: Currently the edenapi client crashes trying to convert a headname to a hex string, when the headname is a string that's not in hex form. Check if the string is in hex form before attempting to look it up as a prefix.

Reviewed By: DurhamG

Differential Revision: D32894078

fbshipit-source-id: 1d2d8145aa3136a27e4bbe674a07e2aa8bca81ef
2022-02-02 17:52:31 -08:00
Muir Manders
d68bac7192 lfs: retry fetch requests on 503 response code
Summary:
Currently a single LFS 503 during clone/checkout aborts the operation and leaves the user in a situation that is difficult to recover from. Now we treat a 503 as any other 5XX server error and retry.

The 5XX retry count/backoff can be configured via the lfs.backofftimes config knob. It defaults in the code to [1, 4, 8], which is 3 retries with rand(1), rand(4) and rand(8) sleep in between.

503 can be served from proxygen indicating there are no live lfs servers, or from x2pagentd in cases of network failure.

Reviewed By: quark-zju

Differential Revision: D33804601

fbshipit-source-id: cbb3514051843d3517bb676cf7c83dce057e42b9
2022-02-02 17:37:32 -08:00
Saul Gutierrez
56742d0027 initrepo: add python binding for Rust repo
Summary: Adds a Python binding for the Rust `repo` crate. For now it only exposes the static `Repo::init` method.

Reviewed By: DurhamG

Differential Revision: D33796094

fbshipit-source-id: 9cf49bcdd24bbabead79902e64a49ec8dc6fac55
2022-02-02 17:31:43 -08:00
Saul Gutierrez
6c9f70da66 repo: add method for repo initialization
Summary:
Adds a method that will be used in the future for replacing the repo initialization done in Python.

More specifically, this method creates and populates the files required for a new repo in the same way as we currently do via the constructor of `localrepo` with the extensions and configs we have enabled by default.

Reviewed By: DurhamG

Differential Revision: D33901712

fbshipit-source-id: 0d12732f5c9d36fb115a97e53b356a88fd609b6c
2022-02-02 17:31:43 -08:00
Jun Wu
1c8e6aa573 git: support debugbundle
Summary: This command is used by jf to find commits in a bundle.

Reviewed By: DurhamG

Differential Revision: D33948840

fbshipit-source-id: 2b7b8c09cb2b30c192a00e83437d079010be5a41
2022-02-02 14:25:38 -08:00
Yan Soares Couto
80517b6fda Default master-fastpath to True
Summary:
## Context
See D33771940 (d274a0f575)

## This diff
I move `pull.master-fastpath` to True.

Reviewed By: quark-zju

Differential Revision: D33892184

fbshipit-source-id: 105c587da85287915ce3963315d568bf65b798c9
2022-02-02 11:01:34 -08:00
Muir Manders
7bbc2e1517 lfs: only log hash mismatch if size matches expected
Summary: In production we do currently see some cases of lfs_read_hash_mismatch. I realized it may be due to missing chunks rather than bad chunk integrity. The new "skip_hash_on_read" path is still able to detect size mismatch, so tweak things to only log mismatch message if the sizes match.

Reviewed By: quark-zju

Differential Revision: D33928640

fbshipit-source-id: 5b63db2b05bc7b53779cdb5ae0041a7d22bb3962
2022-02-02 09:50:03 -08:00
Yan Soares Couto
2c677eb49c Default exchange.httpcommitlookup to True
Summary:
## Context
See D33771940 (d274a0f575)

## This diff
I move `exchange.httpcommitlookup` to True.

Reviewed By: quark-zju

Differential Revision: D33892116

fbshipit-source-id: 48a4b5b9effaeca0df62b56c173cb53ad231c497
2022-02-02 08:42:48 -08:00
Yan Soares Couto
4e25a7ef08 Default httpbookmarks to True
Summary:
## Context
See D33771940

## This diff
I move `pull.httpbookmarks` and `infinitepush.httpbookmarks` to True.

Reviewed By: quark-zju

Differential Revision: D33891956

fbshipit-source-id: cbc3d92aa7ac7ac9edf90b4f99ab9804e9d750be
2022-02-02 07:43:38 -08:00
Yan Soares Couto
d274a0f575 Default pull.httpmutations to true
Summary:
## Context

There are some config options that have been rolled out to 100% of users for some time, but they're still not used in tests by default, which makes our coverage worse.

The best way to change the behaviour of hg regarding config is to change the default value on `configitems.py`, and then (eventually, when this rolls out everywhere), delete them from the configerator overrides config.

The other option would be to leave them in configerator (or in eden/scm/lib/configparser/src/fb/mod.rs), but then they would also need to be replicated in all tests that want to use it, or at the very least duplicated in the base hgrc for integration tests, which is very prone to errors (or simply forgetting to do it, the state we are in today).

Notice that this still allows us to set them to false via configerator quickly, if necessary.

## This diff

This diff defaults `pull.httpmutations` to true: It was previously always set to true on configparser/src/fb/mod.rs

Reviewed By: quark-zju

Differential Revision: D33771940

fbshipit-source-id: 987db9cfaff720645eac4ad65edf560f875b3655
2022-02-02 07:43:38 -08:00
Jun Wu
472a1148b1 git: support bundle and unbundle
Summary: Delegate to git's own bundle implementation for bundle and unbundle.

Reviewed By: DurhamG

Differential Revision: D33895676

fbshipit-source-id: 3d17160c6f83c2f43ff41bcf3f74c00aecded630
2022-02-01 16:14:02 -08:00
Jun Wu
26d921b8c1 git: support older git by avoiding init -b
Summary:
The flag is not supported by git older than 2.28. There are still places
running older git. Change the command from `git init -b x` to
`git -c init.defaultBranch=x init` so it becomes compatibile with older
git.

We don't use the default branch. So set it to `_unused_branch` to
make it more explicit.

Reviewed By: DurhamG

Differential Revision: D33851129

fbshipit-source-id: 93534362b02353728d72d0ebef31c6a25737398f
2022-02-01 16:14:01 -08:00
Jun Wu
ed6b21bbce git: disable treemanifest prefetch
Summary:
With a local git store all trees are local. There is no need to prefetch.
Forcing prefetch trees would cause errors, since the protocol is incompatible.

Reviewed By: DurhamG

Differential Revision: D33837781

fbshipit-source-id: c41281ac96aff1501bd056901213b912a77f6c31
2022-02-01 16:14:01 -08:00
Jun Wu
d2e54864af git: update smartlog.repos config
Summary: It is a left-over from D33518822 (b897a19073).

Reviewed By: DurhamG

Differential Revision: D33837780

fbshipit-source-id: f42fb7f9099ab4cb388c7bd8f038839fd6976cbf
2022-02-01 16:14:01 -08:00
Jun Wu
0c883afe41 git: support clone --git
Summary:
Previously `clone` has to use git URLs with `git+` prefix in scheme such as
`git+ssh://...`, `git+file://...`, `git+https://...`. That was the same
behavior as the `hg-git` extension but is inconvenient for scp-like URLs
like `git@github.com:git/git.git` (provided by GitHub). That URL needs to be
translated to ssh form `git+ssh://git@github.com/git/git.git` manually,
which can be annoying.

Let's use a `--git` flag so `clone --git` can use the scp-like URL directly.
The `--git` flag also makes `clone` more consistent with `init`.

Reviewed By: DurhamG

Differential Revision: D33837782

fbshipit-source-id: 7fe11be6cf63932bde1f400cdfc2fc8120f5067a
2022-02-01 16:14:01 -08:00
Durham Goode
f88bf230ca rage: remove pack output from hg rage
Summary:
Packs are long dead. Let's remove the logging for hg rage. This logic
also doesn't work on Windows, due to depending on cmd commands, resulting in
some errors in the hg rage since we now use powershell.

Differential Revision: D33925868

fbshipit-source-id: c071f8f5685dbf7a9174cad31a0a8cd99a84cf88
2022-02-01 16:06:57 -08:00
Jun Wu
09d7d3785f eden: avoid slow linkrev scan paths
Summary: See the added comments and D33926639.

Differential Revision: D33927449

fbshipit-source-id: d60ff5410a6f878e3d0c85f7ad47072687e8e285
2022-02-01 15:37:47 -08:00
Muir Manders
17c76c1e4e nativecheckout: fix "-C" to always clear mergestate
Summary: There was an edge case where native checkout "up --clean" wasn't clearing the merge state if the working tree was clean. Add a check so we use fallback non-native code path if there is an active merge state.

Reviewed By: quark-zju

Differential Revision: D33893832

fbshipit-source-id: a0e4a8d23818a1e40582cede0afee62ac53cb680
2022-02-01 15:18:20 -08:00
Saul Gutierrez
a237ae47e7 repo: move the repo module to its own crate
Summary: Since `Repo` will soon contain more logic on its own not related to `clidispatcher`, we are moving its own crate.

Reviewed By: DurhamG

Differential Revision: D33883867

fbshipit-source-id: 94414e1e2692962bf491cc2abb4e1d3799cbcc03
2022-02-01 11:32:42 -08:00
Jan Mazur
c9e0a0bc1b remove x-x2pagentd-ws-over-h1=1 header
Summary: Due to a bug in the x2pagent with websocket connection polling in h2 we had to force h1. Now it's fixed and the new version of the agent with the fix should be rolled out to (almost - it's corp) 100% of users.

Reviewed By: quark-zju

Differential Revision: D33842574

fbshipit-source-id: aaf5c66143806a900253120a9f750e37ebe55b80
2022-02-01 03:13:47 -08:00
Muir Manders
dd78f808fb sigtrace: allow manually sigtracing long running commands
Summary: On Windows we apparently can't send a USR1 to trigger a trace on an eden helper process. This commit makes it possible to touch a file to trigger a particular long running process to write out its sigtrace.

Reviewed By: DurhamG

Differential Revision: D33904056

fbshipit-source-id: d42f6c071bcfaa8dfc352f65df31bdc5ea201823
2022-01-31 17:30:13 -08:00
Muir Manders
c7761fb8cd sparse: accept relative sparse profile names
Summary: Make "sparse enable" normalize profile names. It is not clear it wants a canonical path from the root of the repo, so now it makes some attempts to convert the given path(s) to that form. It now works with paths relative from cwd, and normalizes the path separator.

Reviewed By: quark-zju

Differential Revision: D33851998

fbshipit-source-id: 91a3d59b91266bb8941408d23d5021fa1f9e7025
2022-01-31 12:21:43 -08:00
Alex Hornby
2613ad5742 remove generated code for hgclientconf
Summary:
Point the thrift-types at the rust target, no need for the checked in generated code anymore

Saves generation, gives quicker signal on thrift changes (no codegen step), and stops thrift lib vs generated thrift code mismatches

Reviewed By: yancouto

Differential Revision: D33843588

fbshipit-source-id: 856faeb16dc0a2a42f2f07fd3005079a01b6e7bd
2022-01-31 10:34:32 -08:00
Andres Suarez
20675f6fc9 Update indexmap to 1.8.0
Summary: Non-breaking changes. https://github.com/bluss/indexmap/blob/1.8.0/RELEASES.rst

Reviewed By: Imxset21

Differential Revision: D33887730

fbshipit-source-id: 60e016f1f4429407b1f8232f3e5eefe5f89965fd
2022-01-31 07:48:41 -08:00
Muir Manders
06eb4d9977 pipe "fb" cargo feature down to clientinfo
Summary: I moved some stuff around and am having trouble getting everything to be happy. One thing I forgot was to actually pipe the "fb" feature down from the entry point crates (hgcommands and backingstore).

Reviewed By: xavierd

Differential Revision: D33852732

fbshipit-source-id: aabdd145452e66435e130fcbab0ba8a80f9d943a
2022-01-31 02:09:09 -08:00
Muir Manders
57d6b8787e nativecheckout: add total to "recording" progress bar
Summary: Replace "recording" spinner with a true progress bar.

Reviewed By: quark-zju

Differential Revision: D33796497

fbshipit-source-id: 2399b20af3c65ec91031b417413010d36d7a6101
2022-01-31 02:09:07 -08:00
Alex Hornby
b39d4ea2d1 updated expected config hash
Summary: Update the expected hash

Reviewed By: HarveyHunt

Differential Revision: D33843474

fbshipit-source-id: 4a14744e1d6ed15042b12f62f45634fa75591ffe
2022-01-31 02:09:07 -08:00
Jan Mazur
6d01fa3d6c make test-cats.t and test-advice-headers.t not flaky
Summary:
Sometimes kill was exiting with non-zero exit code. Lets make the test machinery kill ncat.

Also during stress runs, we couldn't reuse ports. Let's just randomly allocate them. The test suite does the same thing. but in a different range.

Reviewed By: HarveyHunt

Differential Revision: D33818127

fbshipit-source-id: d875495d590bf8247374ba59396e7032acbdd351
2022-01-28 04:19:00 -08:00
Jan Mazur
8c3c531f2c connect directly to mononoke - remove hgcli
Summary:
I'm:

* adjusting some leftover tests in `fbcode/eden/scm/tests` to speak to mononoke directly
* deleting hgcli from dependencies in `fbcode/eden/mononoke/tests/integration`
* deleting hgcli from dependencies in `fbcode/eden/scm/tests`
* deleting some tests from `fbcode/eden/hg-server/tests`
* removing hgcli usage form scripts like `library.sh` `dummyssh` and `unittestify.pt`

I'm leaving (for now):
* hgcli code and buck rule
* mononoke hgcli codepath

```
➜  tests pwd
/home/mzr/fbcode/eden/scm/tests
➜  tests grep -Ri hgcli
➜  tests cd ~/fbcode/eden/mononoke/tests/integration
➜  integration grep -Ri hgcli | wc
      0       0       0
```

Reviewed By: HarveyHunt

Differential Revision: D33765124

fbshipit-source-id: 46798452f74d59c873449fc6b00445c9558d3ae1
2022-01-28 01:21:32 -08:00
Muir Manders
847a1651f0 rage: limit x2pagent log size on Windows
Summary:
Use the _tail() function which can limit size and has some other smarts.

Also, tweak some progress spinner messages to show up better. "collecting information" was too long and you only saw "collecting". The sub-section spinners "collecting {something}" were also too long and you also only saw "collecting".

Reviewed By: quark-zju

Differential Revision: D33830014

fbshipit-source-id: cb3693c7cb52ac2d060a2da400c9f21ba03f9048
2022-01-27 16:10:16 -08:00
Muir Manders
66a23e61e5 edenapi: let hg-http handle cert config
Summary:
This gets rid of some config boilerplate and allows for centralized cert checks in the http client object.

I left the subtle cert validation checks as-is. We will revisit these later.

Reviewed By: DurhamG

Differential Revision: D33717226

fbshipit-source-id: e9787f7068e5988b83d7c18b60d15ac6eae3c72c
2022-01-27 13:32:13 -08:00
Muir Manders
8dafbc0d42 revisionstore: let hg-http handle lfs cert config
Summary: This gets rid of some config boilerplate and allows for centralized cert checks in the http client object.

Reviewed By: DurhamG

Differential Revision: D33717225

fbshipit-source-id: c3baf271ac0bc56b1c21eab3fd363bccd7dcff37
2022-01-27 13:32:13 -08:00
Muir Manders
47913196d5 hg-http: support configuring cert settings
Summary: This allows TLS config to flow through the http client object rather than being manually configured by each http user.

Reviewed By: DurhamG

Differential Revision: D33717227

fbshipit-source-id: e8f2a49b81f5d30aaedce3517bad8630fb770dfc
2022-01-27 13:32:12 -08:00
Muir Manders
ffbcc29ebf hg-http: migrate rest of HgHttpConfig
Summary:
Move the remaining hg_http::HgHttpConfig fields into http_client::Config. hg_http::http_config() now can derive these config fields from the hg config.

This change cause HTTP requests via EdenFS to:
1. Support x2pagentd unix socket proxy
2. Set X-Client-Info header
3. Respect the http.verbose hg config item to dump request details

Previously the above config items were only set by hg.

Reviewed By: quark-zju

Differential Revision: D33148772

fbshipit-source-id: 4359399aa2d915a2e833fa9ebf8c8860372f0810
2022-01-27 13:32:12 -08:00
Muir Manders
a144039f9b backingstore: conditionalize use of "fb" feature
Summary: Don't set the "fb" cargo feature via cmake unless we are in fb build.

Reviewed By: quark-zju

Differential Revision: D33781554

fbshipit-source-id: 39c61dc1631ebfd8a2c858b5e214c6a09ce0b005
2022-01-27 13:32:12 -08:00
Alex Hornby
e7156fd760 remove unused fb303 dependency
Summary: fb303_core is used. fb303 is not, so remove it

Reviewed By: HarveyHunt

Differential Revision: D33815465

fbshipit-source-id: 149ef2cc48e2eed17fc859707300db555ea5b6b4
2022-01-27 09:41:40 -08:00
Jan Mazur
d4bdd05d60 print testtmp dir before test run only if keep-tmpdir passed
Summary:
This is especially useful when there is long-running test or a test that's stuck. With this path it will be possible to at least look at logs or configs that are being used in the test environment.

I also considered outputting it by default if isatty() == True, but some folks might not like that and it might be too noisy.

Reviewed By: yancouto

Differential Revision: D33792360

fbshipit-source-id: 5c440c3be63242791d92b0decb523c70f4527ee9
2022-01-27 03:23:04 -08:00
Muir Manders
a0dd74742e Back out "print testtmp dir before test run"
Summary:
This breaks the output of mercurial integration tests, e.g.:

```
test-run-tests.t ...
 --- test-run-tests.t
+++ test-run-tests.t.err
@@ -59,6 +59,7 @@

   $ touch test-empty.t
   $ rt
+  testtmp dir: /private/var/folders/w7/8zttmkbx5v94zqfmg4p9kh30000xbj/T/hgtests.1m__f94n/child1/test-empty.t
```

Original commit changeset: 6d131266acfe

Original Phabricator Diff: D33751471 (b1b8a5218a)

Reviewed By: DurhamG

Differential Revision: D33771814

fbshipit-source-id: 7abbba4a5667af29138dfeca9aebb20e162142f6
2022-01-25 12:23:51 -08:00
Andres Suarez
edd73c3cb8 Update tokio to 1.15
Reviewed By: Imxset21

Differential Revision: D33759920

fbshipit-source-id: 7fd02dc93a7c1c564454a1fece3ff4b12dba3d3e
2022-01-25 02:05:10 -08:00
Jun Wu
ad5a787741 match: distinguish gitignore files and directories
Summary:
Fixes a XXX comment. I think it was there because there is a code path that
incorrectly uses matchfn instead of visitdir for directories. But now tests
are passing. The problematic code path seems to have been resolved by
D17099990 (b7680284ca), which tests `kind == dirkind` and then uses `dirignore` or `ignore`
correctly.

Reviewed By: DurhamG

Differential Revision: D33702579

fbshipit-source-id: a06d6a661a552996d90cbe96dd75cb3d4ec5c4d9
2022-01-24 20:24:22 -08:00
Jun Wu
d975e1f900 git: fix auto pull commit hashes
Summary:
Auto pulling commit hashes does not actually work. Fix it and add a test.

Note: This pulls the commit hash as a "draft" if it's based off selected
branches. If we have "public visibleheads" we might use that instead.

Reviewed By: DurhamG

Differential Revision: D33698406

fbshipit-source-id: eac5e4a5536e2c639dee6c7f8b295388b33d26f7
2022-01-24 20:24:22 -08:00
Jun Wu
d7e594deb9 git: drop unused parameter in initgitbare
Summary:
The `url` parameter is no longer used since we maintain git remote config in
hgrc and no longer calls `git remote add`.

Reviewed By: DurhamG

Differential Revision: D33698407

fbshipit-source-id: db891b4370d1b5a42783930c5bda540c799f7211
2022-01-24 20:24:22 -08:00
Jun Wu
49fe26baa2 tests: move git env reset to git.sh
Summary:
It seems there are other tests using `run-tests.py` that do not want the git
env reset behavior. Let's move the env reset logic to a separate optional script.

Reviewed By: DurhamG

Differential Revision: D33752371

fbshipit-source-id: 9465f251ec410595652f1335d1017974876953d7
2022-01-24 17:23:34 -08:00
Jan Mazur
b1b8a5218a print testtmp dir before test run
Summary: This is especially useful when there is long-running test or a test that's stuck. With this path it will be possible to at least look at logs or configs that are being used in the test environment.

Reviewed By: DurhamG

Differential Revision: D33751471

fbshipit-source-id: 6d131266acfef04b0aedad09eae8e5165175705f
2022-01-24 16:15:35 -08:00
Muir Manders
cb62f8d38b addremove: handle symlink shadowing of removed files
Summary: Previously if a removed file was shadowed through a new symlink, addremove would get confused and try to add back the removed file. Fix by "auditing" removed files before we categorize them as "forgotten" (forgotten files are added by addremove).

Reviewed By: DurhamG

Differential Revision: D33748793

fbshipit-source-id: e118baab18b7117d08d87ffb9119530ecdb21acc
2022-01-24 15:15:53 -08:00
Durham Goode
2e29fdfd5d merge: remove O(repo) behavior in _filesindirs
Summary:
When a directory is replaced with a file, Mercurial does some extra
checks. Those extra checks invoke merge._filesindirs() which currently iterates
over every file in the manifest.

This diff uses manifest.matches() to do it way faster.

Reviewed By: danielocfb

Differential Revision: D33722132

fbshipit-source-id: 229e0273f07af71728109db132c6ec72d6ac3dd9
2022-01-24 14:07:30 -08:00
Durham Goode
c34c1c8497 edenapi: cache converted pem files
Summary:
On Windows we're reconverting certs for every request. Let's cache
them.

I thought about trying to store the cache on the client or something, but it
seemed more straight forward and fairly safe to just have a global cache.

Reviewed By: quark-zju

Differential Revision: D33690775

fbshipit-source-id: d3216f5403ea2912f3e49a37aad154d22922487c
2022-01-24 13:44:31 -08:00
Jun Wu
cbd69ed217 test-git-tags: fix for osx
Summary:
There is a name collision `a` vs `A` on OSX. Fix it by reusing the lower-case
name.

Reviewed By: DurhamG

Differential Revision: D33749197

fbshipit-source-id: 27249632f5cc5621b47208ad95c68ca62269be85
2022-01-24 13:44:30 -08:00
Jun Wu
89edf0b94d doctest: fix doctest for osx
Summary: This wasn't discovered because OSX `normpath` path isn't exercised on Linux.

Reviewed By: DurhamG

Differential Revision: D33746541

fbshipit-source-id: 86ccd82477d049208f932eda744a94c03c0d1d02
2022-01-24 13:32:33 -08:00