Commit Graph

1106 Commits

Author SHA1 Message Date
Jun Wu
8d66ade1de hgcommits: add API to get commit text in batch
Summary: The API will be used in Python bindings to avoid running Python in background threads.

Reviewed By: sfilipco

Differential Revision: D25513055

fbshipit-source-id: a108b55115271a256c0d43e0ff7b82c0b209be81
2020-12-14 19:10:44 -08:00
generatedunixname89002005325677
9c5eaa7c81 Daily arc lint --take RUSTFMT
Reviewed By: zertosh

Differential Revision: D25527687

fbshipit-source-id: 25c98dcb607ec7f469099b68f432f3412ca9988e
2020-12-14 03:56:02 -08:00
Arun Kulshreshtha
f98273063a auth: make auth_for_url validate certs
Summary: Make the auth crate validate the user's certificate before returning it. This way we can catch invalid certs before trying to use them.

Reviewed By: sfilipco

Differential Revision: D25454687

fbshipit-source-id: ad253fb433310570c20f33dbd0d0bf11df21e966
2020-12-11 18:55:03 -08:00
Arun Kulshreshtha
a8a9fa3246 auth: add x509 module for validating certs
Summary: Add a new module that can parse X.509 certificates and detect common issues (e.g., the certificate is missing, corrupt, or expired). This should allow us to provide better UX around certificate errors.

Reviewed By: sfilipco

Differential Revision: D25440548

fbshipit-source-id: b7785fd17fa85f812fd38de09e79420f4e256065
2020-12-11 18:55:03 -08:00
Jun Wu
ec7c0659c9 dag: relax trait bounds for AbstractNameDag
Summary: This makes it more flexible.

Reviewed By: kulshrax

Differential Revision: D24467604

fbshipit-source-id: 63023cf0dde2fb7eac592ac79008e4b7a62340c1
2020-12-11 17:03:37 -08:00
Xavier Deguillard
49de273c77 revisionstore: handle redacted blob in LFS
Summary:
When a blob is redacted server side, the http code 410 is returned.
Unfortunately, that HTTP code wasn't supported and caused Mercurial to crash.
To fix this, we just need to store a placeholder when receiving this HTTP code
and simply return it up the stack when reading it.

Reviewed By: DurhamG

Differential Revision: D25433001

fbshipit-source-id: 66ec365fa2643bcf29e38b114ae1fc92aeaf1a7b
2020-12-11 12:28:38 -08:00
Durham Goode
d7af02abb1 revisionstore: fix paths for local indexedlog
Summary:
There was a bug with local-data indexedlog storage where it
wasn't applying the appropriate suffix, so tree data was being stored in
.hg/store/indexedlogdatastore just like file data. Let's fix that and add a
test.

Reviewed By: quark-zju

Differential Revision: D25469917

fbshipit-source-id: 731252f924f9a8014867fc077a7ef10ac9870170
2020-12-11 06:58:27 -08:00
Jun Wu
dcf4957619 dag: make parent function async
Summary: Make the parent function used by various graph building functions async.

Reviewed By: sfilipco

Differential Revision: D25353612

fbshipit-source-id: 31f173dc82f0cce6022cc2caae78369fdc821c8f
2020-12-10 12:37:36 -08:00
Jun Wu
bffffb2415 dag: remove IdMapBuildParents
Summary:
It is no longer needed for building segments (replaced by "prepared flat
segments"). Remove it.

Reviewed By: sfilipco

Differential Revision: D25353613

fbshipit-source-id: aede9e33c3217a61b5b14aae5b128d8953bc578e
2020-12-10 12:37:36 -08:00
Jun Wu
ad6f25addc dag: make IdConvert async
Summary: Make IdConvert async and migrate all its users.

Reviewed By: sfilipco

Differential Revision: D25350915

fbshipit-source-id: f05c89a43418f1180bf0ffa573ae2cdb87162c76
2020-12-10 12:37:35 -08:00
Jun Wu
24badee0d0 dag: make IdMapAssignHead async
Summary: This will make it easier to make IdConvert async.

Reviewed By: sfilipco

Differential Revision: D25350912

fbshipit-source-id: fbaf638b16a9cf468b7530b19d699b7996ddc4f1
2020-12-10 12:37:35 -08:00
Jun Wu
f30934ab4f dag: require Send and Sync on Persist::Lock
Summary: This will make async migrating easier.

Reviewed By: sfilipco

Differential Revision: D25350913

fbshipit-source-id: f33bdc0023ae0cc49601504b811991ea6813ff9e
2020-12-10 12:37:35 -08:00
Jun Wu
abc97bb6fe dag: make write_sparse_idmap async
Summary: This will make it easier to make IdConvert async.

Reviewed By: sfilipco

Differential Revision: D25350914

fbshipit-source-id: 9f2957731f13a28fdfab834de19763b8afcf8ffa
2020-12-10 12:37:35 -08:00
Jun Wu
461fa77fd7 dag: make Set::flatten async
Summary: This will make it easier to make IdConvert async.

Reviewed By: sfilipco

Differential Revision: D25345239

fbshipit-source-id: 684a0843ae32270aa9b537ef9a2b17a28c027e51
2020-12-10 12:37:34 -08:00
Jun Wu
53bdae78d9 dag: make ToIdSet async
Summary: This will make it easier to make IdConvert async.

Reviewed By: sfilipco

Differential Revision: D25345232

fbshipit-source-id: b8967ea51a6141a95070006a289dd724522f8e18
2020-12-10 12:37:34 -08:00
Jun Wu
f854d2e03e dag: make DagAlgorithm async
Summary:
Update DagAlgorithm and all its users to async. This makes it easier to make
IdConvert async.

Reviewed By: sfilipco

Differential Revision: D25345236

fbshipit-source-id: d6cf76723356bd0eb81822843b2e581de1e3290a
2020-12-10 12:37:34 -08:00
Jun Wu
6c02a90386 dag: make MetaSet accept async evaluate and contains
Summary:
Make it possible to use async functions in MetaSet functions.
It will be used when DagAlgorithm becomes async.

Reviewed By: sfilipco

Differential Revision: D25345229

fbshipit-source-id: 0469d572b56df21fbdbdfae4178377e572adbcda
2020-12-10 12:37:34 -08:00
Jun Wu
0ac5bcef79 mutationstore: make calculate_obsolete async
Summary: This will make it easier if the `Set` passed in requires async.

Reviewed By: sfilipco

Differential Revision: D25345230

fbshipit-source-id: a327d4e5d425b7eb5296b2fbe25c446492aa9ea7
2020-12-10 12:37:34 -08:00
Jun Wu
a03e8f4c55 dag: make DagPersistent and DagAddHeads async
Summary: This makes it easier to make DagAlgorithm async.

Reviewed By: sfilipco

Differential Revision: D25345234

fbshipit-source-id: 5ca4bac38f5aac4c6611146a87f423a244f1f5a2
2020-12-10 12:37:33 -08:00
Jun Wu
6d9e8eb249 hgcommits: make traits async
Summary: This will make it easier to use `.await` if part of `dag` becomes async.

Reviewed By: sfilipco

Differential Revision: D25345237

fbshipit-source-id: 7f07cdaa9c2e0468667638066611fabe3a3f7f28
2020-12-10 12:37:33 -08:00
Jun Wu
e2542490e8 dag: change &impl Trait to &dyn Trait
Summary: `impl Trait` does not work with `async_trait`.

Reviewed By: sfilipco

Differential Revision: D25345238

fbshipit-source-id: e7890dbaeb162d44e072ea4428d045004608719b
2020-12-10 12:37:33 -08:00
Jun Wu
da6b03768a dag: require Send + Sync on AbstractNameDag
Summary: This makes it easier to migrate to async.

Reviewed By: sfilipco

Differential Revision: D25345228

fbshipit-source-id: e819f0de5f805377a6977325216ef11b14d68c1d
2020-12-10 12:37:33 -08:00
Jun Wu
053cd9d8a3 dag: mark a few things as Sync or Send
Summary:
Marking IdConvert Sync makes it possible to be used as a trait object with async-trait.
See https://docs.rs/async-trait/0.1.41/async_trait/#dyn-traits

`dag` uses a lot `dyn DagAlgorithm`. In the future when async is used more, the
trait object will be required to be Send or Sync. Just require it on the trait
to make our life easier.

Marking `IdDagStore` as Send + Sync makes async migration easier.

Reviewed By: sfilipco

Differential Revision: D25345231

fbshipit-source-id: 45b96057907cbe2a1d38fd424e7d4c963dd1b245
2020-12-10 12:37:32 -08:00
Jun Wu
2496281d78 dag: make PrefixLookup async
Summary: Use async function for the PrefixLookup trait.

Reviewed By: sfilipco

Differential Revision: D24840820

fbshipit-source-id: d22cac9f11b06e3127fa956e3f116cf232214125
2020-12-10 12:37:32 -08:00
Jun Wu
c0bc7800d3 dag: add some default impls
Summary: This makes the trait objects slightly easier to use.

Reviewed By: sfilipco

Differential Revision: D24840821

fbshipit-source-id: 22fcdf13b62420302b562c309874e08360d02372
2020-12-10 12:37:32 -08:00
Jun Wu
4d791fd823 dag: require PrefixLookup for IdConvert
Summary: This makes `dyn IdConvert` include `PrefixLookup`.

Reviewed By: sfilipco

Differential Revision: D24840819

fbshipit-source-id: 8d4e25c534f6e4397ec6f643eb3aa116bff12a2c
2020-12-10 12:37:32 -08:00
Jun Wu
d9128caac1 dag: delegate IdMapSnapshot impls
Summary:
In the future, when async APIs are used, Python bindings will have lifetime
issues. Make it possible to clone the IdMap so the Python bindings can be made
to work.

Reviewed By: sfilipco

Differential Revision: D24840822

fbshipit-source-id: 6aa4e369c877c428ed39d2cbea79e6943836afa8
2020-12-10 12:37:32 -08:00
Jun Wu
c6f14f02f4 dag: use Stream interface for NameSet iteration
Summary: This makes NameSet more friendly for async use-cases, interface-wise.

Reviewed By: sfilipco

Differential Revision: D24806695

fbshipit-source-id: 6e640ba2666872a9128d6460e8b53d6a0e595e56
2020-12-10 12:37:31 -08:00
Jun Wu
32176eca42 dag: add async interface for NameSet
Summary:
Change the main API of NameSet to async. Use the `nonblocking` crate to bridge
the sync and async world for compatibility. Future changes will migrate
Iterator to async Stream.

Reviewed By: sfilipco

Differential Revision: D24806696

fbshipit-source-id: f72571407a5747a4eabe096dada288656c9d426e
2020-12-10 12:37:31 -08:00
Jun Wu
1eb5244eb1 nonblocking: add a way to convert async fn to non-blocking sync fn
Summary:
This will make it easier to incrementally migrate APIs to async, and make sync
programs able to use sync interface if they'd like to. For example, the dag
crate is likely going to be async-ize. Its async APIs are only useful for
places where the dag is lazy. It's still possible to have a non-lazy dag
that a sync interface is suitable.

Reviewed By: ikostia

Differential Revision: D24777747

fbshipit-source-id: 6d56149fbd1f9b29f5fc62387cbf194800755d12
2020-12-10 12:37:31 -08:00
Jun Wu
6dd2e4c2d8 edenapi: remove unused variable
Summary: Resolve a compiler warning.

Reviewed By: sfilipco

Differential Revision: D25350917

fbshipit-source-id: 5de3f613953c9ad3e35e149cb4abaa611c63a9ae
2020-12-10 12:37:31 -08:00
Durham Goode
1712864779 metadatastore: add config to write local history to indexedlog
Summary:
We already have a config to write shared history to an indexedlog. Let's
add a similar config for local history.

Reviewed By: quark-zju

Differential Revision: D23910457

fbshipit-source-id: eca21db0350895f573a60e4e1a6b05514e70f1c7
2020-12-10 07:28:13 -08:00
Durham Goode
a5f8e0a301 revisionstore: allow persistent IndexedLogHistory
Summary:
In a future diff we'll use the indexedlog stores for local history. We
want those to exist forever, so let's move IndexedLogHgIdHistoryStore to use a
Store under the hood, and add an enum for distinguishing between the two types
at creation time.

Reviewed By: quark-zju

Differential Revision: D25429675

fbshipit-source-id: 5f2dc494e1175d4c1dc74992d3311d2e55d784ca
2020-12-10 07:28:13 -08:00
Arun Kulshreshtha
ef67b259ee edenapi: allow single cert file
Summary: Previously, the EdenAPI client (and Mercurial's `http-client`) required both a client certificate and private key to be specified individually in order to use TLS mutual authentication. However, libcurl supports having both the certificate and key concatenated together into a single PEM file. This diff makes it possible to simply specify the combined file as the `cert`, leaving the `key` unset.

Reviewed By: quark-zju

Differential Revision: D25323786

fbshipit-source-id: 1800be3ef82ec4dfa89d725f5860190172994c89
2020-12-09 15:07:19 -08:00
Arun Kulshreshtha
b45dbd49ff auth: treat empty cert paths as unset
Summary: Treat empty paths as `None`, which allows certificate and key paths to be unset via a `--config` flag (e.g. `hg debughttp --config auth.edenapi.key=`). This would normally require adding an `%unset` line to the appropriate hgrc, which adds friction to ad-hoc command line usage.

Reviewed By: quark-zju

Differential Revision: D25416531

fbshipit-source-id: 15aae78d9b3a82227278f33def45fa960aa65a98
2020-12-09 15:07:19 -08:00
Arun Kulshreshtha
23569dfc4d Regenerate all autocargo-managed Cargo.tomls
Reviewed By: singhsrb, dtolnay

Differential Revision: D25417653

fbshipit-source-id: 81057698ae457a28b7aac6f7a1b32bcec9517eff
2020-12-09 12:44:29 -08:00
Durham Goode
476c8b7b30 contentstore: add config to write local data to indexedlog
Summary:
We already have a config to write shared data to an indexedlog. Let's
add a similar config for local data.

Reviewed By: xavierd

Differential Revision: D23909569

fbshipit-source-id: 87317554beb6bef8237e6a900403701662c3c0d0
2020-12-09 07:07:40 -08:00
Durham Goode
4f611364eb revisionstore: add repair support to IndexedLogHgIdDatatore
Summary:
We temporarily dropped repair support when transitioning to using Store instead
of a raw RotateLog. Let's add that back now.

Reviewed By: xavierd

Differential Revision: D25371622

fbshipit-source-id: e28fc425a6ffb50c93540672b0df75a172ebbe9c
2020-12-09 07:07:40 -08:00
Durham Goode
925e457ae7 revisionstore: allow persistent IndexedLogData
Summary:
In a future diff we'll use the indexedlog stores for local data. We
want those to exist forever, so let's move IndexedLogHgIdDataStore to use a
Store under the hood, and add an enum for distinguishing between the two types
at creation time.

Reviewed By: xavierd

Differential Revision: D23915622

fbshipit-source-id: 296cf6dfcd53e5cf1ae7624fdccedf0a60a77f22
2020-12-09 07:07:40 -08:00
Durham Goode
b9464c4718 revisionstore: add repair functions to Store
Summary:
In a future diff we'll be moving the IndexedLogHgIdDataStore to use the Store
type (that hides the differences between IndexedLog and RotateLog). To do so, it
needs to support repairs. Let's do some minor refactoring to enable this.

Reviewed By: xavierd

Differential Revision: D25371623

fbshipit-source-id: 846cb5f8c21f1e6b550a45259cc8da24cc65b13b
2020-12-09 07:07:39 -08:00
Stefan Filip
5f6d1a2c61 edenapi: add full_idmap_clone endpoint
Summary:
The end goal is to have clients using a sparse IdMap. There is still some work
to get there though. In the mean time we can test repositories that don't use
any revlogs. The current expections for those repositories are that they have
a full idmap locally.

Reviewed By: quark-zju

Differential Revision: D25075341

fbshipit-source-id: 52ab881fc9c64d0d13944e9619c087e0d4fb547c
2020-12-08 18:30:24 -08:00
Wez Furlong
7470b6023b eden: add config and awareness for our edenfs.kext
Summary:
This commit adds a new eden configuration option that
controls whether we try to load our edenfs.kext in preference to
an alternative fuse implementation on macOS.

The majority of this diff is plumbing to convey the configuration
value through to the privhelper, which is relatively restrictive
due to its root-ness.

I've also updated watchman and mercurial to be aware of the new
filesystem type that shows up in the mount table.

Reviewed By: genevievehelsel

Differential Revision: D25065462

fbshipit-source-id: 4f35b9440654298e2706a0d0613d97eb63451999
2020-12-07 11:18:09 -08:00
Arun Kulshreshtha
3c63f41868 auth: migrate crate to autocargo
Reviewed By: singhsrb

Differential Revision: D25323754

fbshipit-source-id: 655a2f525f0f5267b6166310d8cb9598ae2f7b1b
2020-12-04 09:24:18 -08:00
Stefan Filip
f1b78f6afb thrift: update generated thrift assets
Summary: Regen

Reviewed By: singhsrb

Differential Revision: D25320884

fbshipit-source-id: bd3ce800257450b6cd32fb42a3c49e5ead01d4e3
2020-12-03 19:48:04 -08:00
Meyer Jacobs
f16ef7dafd bindings: introduce C / C++ bindings to EdenAPI for use by EdenFS
Summary:
Introduce `edenapithin` crate, which offers C bindings to EdenAPI Client.

There are two top-level `owned` types, `EdenApiClient` and `TreeEntryFetch`, which represent `Box`ed results from calls to EdenAPI's blocking client. The C / C++ code is responsible for calling the associated `free` functions for these types, as well as `OwnedString`, which is used to represent error variants of a `Result` as a string.

Most functionality is provided through functions which operate on and return references into these top-level owned types, providing access into Rust `Result`s and `Vec`s (manually-monomorphized), and EdenApi's `TreeEntry` and `TreeChildEntry`.

Additional non-pointer types are defined in the `types` module, which do not require manual memory management.

C++ bindings are not included currently, but will be introduced soon.

Reviewed By: fanzeyi

Differential Revision: D24866065

fbshipit-source-id: bb15127b84cdbc6487b2d0e1798f37ef62e5b32d
2020-12-01 19:07:25 -08:00
Meyer Jacobs
293053b774 edenapi: expose "attributes" parameter in tree request API
Summary:
Introduce a new API type, `TreeAttributes`, corresponding to the existing type `WireTreeAttributesRequest`, which exposes which optional attributes are available for fetching. An `Option<TreeAttributes>` parameter is added to the `trees` API, and if set to `None`, the client will make a request with TreeAttributes::default().

The Python bindings accept a dictionary for the attributes parameter, and any fields present will overwrite the default settings from TreeAttributes::default(). Unrecognized attributes will be silently ignored.

Reviewed By: kulshrax

Differential Revision: D25041255

fbshipit-source-id: 5c581c20aac06eeb0428fff42bfd93f6aecbb629
2020-12-01 19:07:25 -08:00
Stefan Filip
03c55e82d0 edenapi: add full_idmap_clone functionality
Summary: Adding `full_idmap_clone` to edenapi and usign that in `debugsegmentclone`.

Reviewed By: quark-zju

Differential Revision: D25139730

fbshipit-source-id: 682055f7c30a94a941acd16e2b8e61b9ea1d0aef
2020-12-01 09:59:27 -08:00
Stefan Filip
b991370c3f commands: add debugsegmentclone
Summary: Prototype for cloning a repository using segmented changelog.

Reviewed By: quark-zju

Differential Revision: D24954821

fbshipit-source-id: 5ff42a5d60b6ba3e85dd4eab151e902bd4b96243
2020-12-01 09:59:27 -08:00
Stefan Filip
8327c54db1 dag: add DagImportCloneData::import_clone_data
Summary:
This method reconstructs a dag from clone data.

At the moment we only have a clone data construction method in Mononoke. It's
the Dags job to construct and import the clone_data. We'll consolidate that at
a later time.

Reviewed By: quark-zju

Differential Revision: D24954823

fbshipit-source-id: fe92179ec80f71234fc8f1cf7709f5104aabb4fb
2020-12-01 09:59:27 -08:00
Stefan Filip
042dcbb3a8 edenapi: update clone data request to POST
Summary:
The server expects post requests. At this time we don't want to cache this data
so POST.

Reviewed By: singhsrb

Differential Revision: D24954824

fbshipit-source-id: 433672189ad97100eee7679894a894ab1d8cff7b
2020-12-01 09:59:27 -08:00