Summary:
I'd like to lower the bar to entry as much as possible to use caching in e.g.
mappings. One thing that's a little annoying to setup right now is stats. Let's
unify those.
This does mean the stats names for mappings that had them will change, but I
think that's probably OK here.
A few notes about the implementation:
- I made the stats objects singletons, because ultimately that *is* what stats
are under the hood.
- I made a macro to define the singleton because that was otherwise really
verbose.
Reviewed By: farnz
Differential Revision: D25461182
fbshipit-source-id: f30d0908c40d8e0010c546ec95a385a06d557757
Summary:
This adds support for replacing things in the small repo with the things from
the large repo. This is useful when changing the bind.
This diff slightly changes how `rsync` is called: now `--source-csid`,
`--target-csid`, `--from-dir` and `--to-dir` are all specified before the
`copy`/`remove-excessive-files` subcommands.
There's still an ability to use this within a repo, as you can pass identical
source and target repos.
Reviewed By: StanislavGlebik
Differential Revision: D25087893
fbshipit-source-id: 6e5881f80d91ef4b794a967cf9f26dd3af7f56c9
Summary:
We had a small bug in blobimport. If mononoke repo is empty and blobimport
tries to import a single revision whose rev number is 0 then it will
successfull import it but it will report that no revision were imported
i.e. it will print "didn't import any revision" to stderr and won't update the
manifold latest imported revision marker.
That was because we didn't update max_rev_and_bcs_id if rev is equal to
max_rev. This diff fixes it.
Reviewed By: ahornby
Differential Revision: D25421164
fbshipit-source-id: 639ead0ac326a14051d3a4faba568ecb797857a2
Summary: See D25396203 for discussion. 100 is better than nothing.
Reviewed By: StanislavGlebik
Differential Revision: D25460398
fbshipit-source-id: 608a2dca9c381c78daf0e7d9bcbd1a32f201030a
Summary:
I had to have two of those while I was refactoring away the old style of
doing get or fill, but now that that's gone, we can have a single one and clean
up.
Reviewed By: StanislavGlebik
Differential Revision: D25396201
fbshipit-source-id: 459e9ec7e44e8b349c585212c2758d64077e56d1
Summary:
All the code that needed it basically gone. Might as well push the compat()
calls a little further down and be done with 0.1 futures here.
Reviewed By: StanislavGlebik
Differential Revision: D25396202
fbshipit-source-id: ae85f61c03cb2c38eabbaf0d45387f9d4422b336
Summary: All the callsites are gone now.
Reviewed By: StanislavGlebik
Differential Revision: D25396205
fbshipit-source-id: 74d0595c4528dc739d254f5dc950157e087b00dd
Summary:
Like it says in the title. The upshot here as well is a lot less cloning.
While in there, I removed the "caching" module since it basically only
contained a couple of things that were all needed in the sql module anyway.
Reviewed By: StanislavGlebik
Differential Revision: D25396208
fbshipit-source-id: aa6381c78f45a94fecd04544196180d2a918f97d
Summary: We need this in Phases, so let's add it there.
Reviewed By: StanislavGlebik
Differential Revision: D25396207
fbshipit-source-id: 34174f205028b95c9aa382c343b1344265391df2
Summary:
Like it says in the title. This reduces cloning. Unlike our cached mappings,
changesets have only way of being queried, so other than cloning, it doesn't
make a huge difference.
Reviewed By: StanislavGlebik
Differential Revision: D25396206
fbshipit-source-id: 45d3ebd403142a3f1d9e3ba7de5de2bf18317165
Summary:
Like it says in the title. I need the new futures module in there later in this
stack so this makes it cleaner.
Reviewed By: StanislavGlebik
Differential Revision: D25396200
fbshipit-source-id: 0148003c83b3dd0da5142eb468cf3a6ae2f74b7a
Summary:
Like it says in the title, this updates bonsai_hg_mapping to a futures 0.3
implementation of get_or_fill.
The upshot is that this requires less cloning (in fact, no cloning at all if
the rest of the code was 0.3 here), and in this particular instance it also
lets us completely get rid of the `from_bonsai` flag we were threading through
the whole method and checking many times.
Reviewed By: StanislavGlebik
Differential Revision: D25396199
fbshipit-source-id: f8126c96aad8d982c3deb535530484bec841929f
Summary:
Like it says in the title. Let's update this to the new get_or_fill
implementation that uses 0.3 futures.
Reviewed By: StanislavGlebik
Differential Revision: D25396204
fbshipit-source-id: 06bf449a0d15bd19625acfdcbb4578948e57cde7
Summary:
Like it says in the title, this adds a futures 0.3 variant of
GetOrFillMultipleFromCacheLayers. However, I didn't just port the code as-is,
since the code as it stands wouldn't have been very idiomatic if I did.
Instead, I refactored this to be a function and a few traits. I've also kept
the old code for now, and I'll remove it once I've converted all the callistes.
The upshot of the proposed refactor here is that it should be easier to use
this without having to heavily duplicate the instantiation of the "cacher" in
places where we have multiple variants that are cached (e.g. mappings), all the
while being able to leverage the type checker. See D25334478 (13255301b0) for a discussion
on this. This new approach also makes it much easier to work with the tests for
this (you can just mutate the store and access its fields).
Reviewed By: StanislavGlebik
Differential Revision: D25396203
fbshipit-source-id: d706729a800faa4b12fcf5e608c6dee93c5a909e
Summary: Something is wrong with this which causes Unity to freak out.
Reviewed By: fanzeyi
Differential Revision: D25453230
fbshipit-source-id: 89f61fd97817403fa65071ddac022a226b775e53
Summary:
We already have a config to write shared history to an indexedlog. Let's
add a similar config for local history.
Reviewed By: quark-zju
Differential Revision: D23910457
fbshipit-source-id: eca21db0350895f573a60e4e1a6b05514e70f1c7
Summary:
In a future diff we'll use the indexedlog stores for local history. We
want those to exist forever, so let's move IndexedLogHgIdHistoryStore to use a
Store under the hood, and add an enum for distinguishing between the two types
at creation time.
Reviewed By: quark-zju
Differential Revision: D25429675
fbshipit-source-id: 5f2dc494e1175d4c1dc74992d3311d2e55d784ca
Summary: Switching to specify derived data types other than hg explicitly on the command line
Reviewed By: farnz
Differential Revision: D25367323
fbshipit-source-id: 0e0aea1aab46b43b325486ed6161ea322f7cec4b
Summary: It's useful sometimes, like in the next diff.
Reviewed By: mitrandir77
Differential Revision: D25422597
fbshipit-source-id: 0ebb5dcc349bbaacac3dddf03f19e5e092042468
Summary:
This is useful data to see how much we are over the limit. I noticed
the missing data while resolving
https://fb.workplace.com/groups/scm/permalink/3450652774984318/ as the oncall.
Reviewed By: StanislavGlebik
Differential Revision: D25414427
fbshipit-source-id: ec4bbca9c21a4bf0e675ec1cd82e4e703cd88631
Summary: Previously, the EdenAPI client (and Mercurial's `http-client`) required both a client certificate and private key to be specified individually in order to use TLS mutual authentication. However, libcurl supports having both the certificate and key concatenated together into a single PEM file. This diff makes it possible to simply specify the combined file as the `cert`, leaving the `key` unset.
Reviewed By: quark-zju
Differential Revision: D25323786
fbshipit-source-id: 1800be3ef82ec4dfa89d725f5860190172994c89
Summary: Treat empty paths as `None`, which allows certificate and key paths to be unset via a `--config` flag (e.g. `hg debughttp --config auth.edenapi.key=`). This would normally require adding an `%unset` line to the appropriate hgrc, which adds friction to ad-hoc command line usage.
Reviewed By: quark-zju
Differential Revision: D25416531
fbshipit-source-id: 15aae78d9b3a82227278f33def45fa960aa65a98
Summary:
The f64 compatibility added by D25079001 (e91552fefe) changes the `{rev}` template output.
However, the revset object is not aware of the rev mapping so the following
pattern will stop working:
ifcontains(rev, revset('.'), '@', 'o')
# rev: mapped, but not contains in `revset('.')`
Fix it by teaching `ifcontains(a, b, ...)` to disable f64 compatibility when
evaluating `a`, since `a` is not going to be printed out.
This fixes an issue in VSCode ISL that "You are here" is missing on non-master
commits.
Reviewed By: singhsrb
Differential Revision: D25404673
fbshipit-source-id: 3e53a2ce1f135f8825c195c5a3061dad0359c4b2
Summary: In addition to tracking profile activation, it would be nice to see when people deactivate profiles.
Reviewed By: kmancini
Differential Revision: D25372256
fbshipit-source-id: e6bfefe3c838465ba9ed10d622342bdf3df1c40a
Summary: It would be interesting to track which profiles are activated most often. Also, it may be helpful to track which users are using this feature and generally track usage.
Reviewed By: kmancini
Differential Revision: D25372255
fbshipit-source-id: 2107cdce2a997180547dc63ec654c8d72ced5eb4
Summary:
The code still took a dependency on Mercurial's old manifest code to parse
manifests. It turns out the manifests have a very simple format that we could
parse directly.
This avoids various copies, conversions, std::list, removes ~1k lines of code,
at the expense of adding ~100 lines of code (some of them being C++
boilerplate).
Reviewed By: fanzeyi
Differential Revision: D25385018
fbshipit-source-id: 90d4cda2b7797584bc48c086d5592a7ecaa05dfc
Summary:
We already have a config to write shared data to an indexedlog. Let's
add a similar config for local data.
Reviewed By: xavierd
Differential Revision: D23909569
fbshipit-source-id: 87317554beb6bef8237e6a900403701662c3c0d0
Summary:
We temporarily dropped repair support when transitioning to using Store instead
of a raw RotateLog. Let's add that back now.
Reviewed By: xavierd
Differential Revision: D25371622
fbshipit-source-id: e28fc425a6ffb50c93540672b0df75a172ebbe9c
Summary:
In a future diff we'll use the indexedlog stores for local data. We
want those to exist forever, so let's move IndexedLogHgIdDataStore to use a
Store under the hood, and add an enum for distinguishing between the two types
at creation time.
Reviewed By: xavierd
Differential Revision: D23915622
fbshipit-source-id: 296cf6dfcd53e5cf1ae7624fdccedf0a60a77f22
Summary:
In a future diff we'll be moving the IndexedLogHgIdDataStore to use the Store
type (that hides the differences between IndexedLog and RotateLog). To do so, it
needs to support repairs. Let's do some minor refactoring to enable this.
Reviewed By: xavierd
Differential Revision: D25371623
fbshipit-source-id: 846cb5f8c21f1e6b550a45259cc8da24cc65b13b
Summary:
Same as development branch. Without configuration changes, nothing changes for
the production codepath.
Reviewed By: quark-zju
Differential Revision: D25405026
fbshipit-source-id: aff705aa5f96814f1f1d7552454ab1d0c13afd92
Summary:
The end goal is to have clients using a sparse IdMap. There is still some work
to get there though. In the mean time we can test repositories that don't use
any revlogs. The current expections for those repositories are that they have
a full idmap locally.
Reviewed By: quark-zju
Differential Revision: D25075341
fbshipit-source-id: 52ab881fc9c64d0d13944e9619c087e0d4fb547c
Summary:
The client dag cannot currently be instantiated with a sparse idmap (aka
universal commit idmap). Is should be usable with a full idmap. To test
repositories that use segmented changelog exclusively we add the capability of
cloning the full idmap.
I currently see StreamCloneData as an experiment. I am open to suggestions
around what structure we should have for the regular long term clone endpoint.
That said, I am leaning towards converting clone_data to return
StreamCloneData. Overall, Segmented Changelog has a few knobs that influence
how big the IdMap ends up being so the code that is more flexible will be more
useful long term. To add to that, we transform data higher in the stack using
streaming and this data does similar fetching, it seems that we should have a
stream idmap exposed by clone_data.
Reviewed By: quark-zju
Differential Revision: D24966338
fbshipit-source-id: 019b363568e3191280bd5ac09fc15062711e5523
Summary:
It uses the Python iterator -> Rust stream code path, which can might
cause deadlock with async_runtme::block_on_exclusive. Avoid running
main Python logic in multiple threads.
The prefetching is only useful for lazy commit text backend, which hasn't been
rolled out yet. So disabling it won't cause pref regressions.
Note: This should not affect Phabricator Status prefetching.
Reviewed By: sfilipco
Differential Revision: D25350916
fbshipit-source-id: 49e9c4fea5ff48e1f579aa602693d38c7dd96fc7
Summary:
From wez's description (slightly edited):
a symlink direction resolves outside the repo, and the new validation logic doesn't like that. the test is doing a redirect add bind then redirect add symlink. The second add is intended to implicitly delete the first one if the config is different, but it's not smart enough to realize that that is happening.
This adds an explicit `redirect del` before we do our second `redirect add`
Reviewed By: xavierd
Differential Revision: D25405011
fbshipit-source-id: 107fd272bbe830d8b23e437286ced00460902d91
Summary:
All the bookmarks is *a lot* of bookmarks. Don't do them all at once. Also, add
some logging output so we can tell how far along we are.
Reviewed By: HarveyHunt
Differential Revision: D25397297
fbshipit-source-id: c19b99123f88e05e99bff61e2399a62d378a6671
Summary: Also added a TryShared future to futures_ext. The problem with regular Shared is that if you want to share anyhow::Result the Error part of it is not cloneable. This TryShared will work nicely when returning anyhow::Result, which most of our code does.
Reviewed By: aslpavel
Differential Revision: D25223317
fbshipit-source-id: cf21141701884317a87dc726478dcd7a5a820c73
Summary:
This commit adds a new eden configuration option that
controls whether we try to load our edenfs.kext in preference to
an alternative fuse implementation on macOS.
The majority of this diff is plumbing to convey the configuration
value through to the privhelper, which is relatively restrictive
due to its root-ness.
I've also updated watchman and mercurial to be aware of the new
filesystem type that shows up in the mount table.
Reviewed By: genevievehelsel
Differential Revision: D25065462
fbshipit-source-id: 4f35b9440654298e2706a0d0613d97eb63451999
Summary:
Like it says in the title. This is helpful to measure the number of SQL queries
we make. This required actually threading in a CoreContext, which we didn't
have before.
Reviewed By: StanislavGlebik
Differential Revision: D25336069
fbshipit-source-id: 35677c55550e95b5126de29c2a824b4eda32092c
Summary: Like it says in the title.
Reviewed By: StanislavGlebik
Differential Revision: D25336068
fbshipit-source-id: 113050215c28a28c820d938348a0a3e54c14c3ee
Summary:
Like it says in the title, this adds a caching layer around Globalrevs using
our existing `GetOrFillMultipleFromCacheLayers` abstraction.
Note: I've opted to not track dedicated metrics for this (compare to the hg
mapping to see them), since I don't believe we really ever look at them.
I'd like to do a little bit of refactoring of
`GetOrFillMultipleFromCacheLayers` to a) track them without having to ad-hoc
code it, b) convert it 0.3 futures, and c) require less ceremony to call it.
However, I'll do so in another diff.
Reviewed By: StanislavGlebik
Differential Revision: D25334478
fbshipit-source-id: 1385298b8fdf1cd081b6e509c6c3e03b3abbfa5b
Summary: This lib.rs is getting too big. Split it.
Reviewed By: StanislavGlebik
Differential Revision: D25333510
fbshipit-source-id: ea15664d2de21a24ee107162e030b7762b1d413e
Summary:
I'd like to add a caching variant for this. Might as well not have to rewrite
those methods on an ad-hoc basis.
Reviewed By: StanislavGlebik
Differential Revision: D25333461
fbshipit-source-id: 632c0307189fe15a926d808c1eeca1e3f240eb19
Summary: Like it says in the title.
Reviewed By: StanislavGlebik
Differential Revision: D25333450
fbshipit-source-id: 49ad4b1964a4dfd9f3e0108fa421d451ef905256
Summary: If response is not HTTP/1.1 or isn't Weboscket upgrade we should report error.
Reviewed By: HarveyHunt
Differential Revision: D25368729
fbshipit-source-id: 62dcd93240902924312f4de2965f58357c46c98b
Summary:
Further down the following check is executed:
priority > bestpriority
When priority are strings, this leads to wrong expectations:
$ python3
>>> a = "90"
>>> b = "100"
>>> a > b
True
Reviewed By: krallin
Differential Revision: D25334682
fbshipit-source-id: 48287d57ae6a938e9f2619babf49d073534f46d7
