Summary:
This takes some pressure off both cpu and memory
on a laptop.
Reviewed By: pkaush
Differential Revision: D20562474
fbshipit-source-id: a058c71c47f25c3a2b3c1e34a0d0caf83e642021
Summary:
These were originally generated by getdeps, but since they
have been manually modified they are no longer regeneratable via
the getdeps tooling.
Simplify and just call the getdeps build rather than replicating
the discrete project fetches: they aren't required to break out
separately it just looks nice in the CI output--they will be fetched
on demand during the build anyway.
Reviewed By: simpkins
Differential Revision: D20660208
fbshipit-source-id: b6c63333e842d2bd1ba2fb574f8f082dcefef89e
Summary:
On linux we didn't account for the `--final-install-prefix` argument
which meant that the binaries would always be rewritten to be relative to
the destdir.
This commit fixes that.
Refs: https://github.com/facebook/watchman/issues/760
(this doesn't deal with the compiled in statedir being in the scratch path though)
Reviewed By: simpkins
Differential Revision: D20659749
fbshipit-source-id: 1e8e198a58361882249c33a67f54a7d97b849257
Summary:
The GH actions defaults for git prevent it from being able
to checkout the fbthrift repo due the length of the java related
files in the fbthrift repo.
This commit tells git to use long filenames and allows the checkout
to succeed.
Reviewed By: fanzeyi
Differential Revision: D20659750
fbshipit-source-id: 060b36d312d52a0769cf2f2dd9af60f7446f94a8
Summary:
Ensure that we are referencing python3 in the paths
that we generate for the github actions workflows, and remove
some shebangs that influence how our internal linters process
the python code.
Reviewed By: fanzeyi
Differential Revision: D20659747
fbshipit-source-id: 6f300f8e91edf7701bb27babc7b1418958cf0a10
Summary:
I'm about to add another reference to our service name, so instead of
hardcoding "Eden", add a kServiceName constant. While I'm at it, I
renamed it to EdenFS.
Reviewed By: genevievehelsel
Differential Revision: D20500668
fbshipit-source-id: 4974363156ba2934a1a5cd86d9e8fd7e93d89181
Summary:
This diff - which asyncifies `prefetch_content` - isn't a big win on readabiliy,
but by switching to async (taking references) and then changing upstream functions
to references we're able to save two layers of cloning in the `warmup` functions,
which ought to reduce heap allocations
Reviewed By: farnz
Differential Revision: D20612696
fbshipit-source-id: a52d6246789e964d3b02d0bdc1bfafbded8f25dd
Summary:
Straightforward asyncification, which lets us pass down refs and save
a couple of clones.
Reviewed By: farnz
Differential Revision: D20612725
fbshipit-source-id: 83aa30e2954101faa40d9427ad267e8e42f57d46
Summary:
The api is now asyncified, and we no longer have to clone when calling
`subcommand_tail`.
The inner stream logic is still using the old streams api - untangling that code was difficult for me, so I'll try to tackle it in a later, isolated diff
Reviewed By: farnz
Differential Revision: D20611556
fbshipit-source-id: acd79f8048b5e611c316fc1986160b6395ee6fb9
Summary:
Asyncified `subcommand_backfill`. Some of the inner code is still using
old futures because I was not able to find a new-futures version of
`future_stats::Timed`, but most of the logic is using new futures /
new streams apis.
Reviewed By: farnz
Differential Revision: D20612724
fbshipit-source-id: 00e524f606fc02f4706c02347c0bd6feba46c3eb
Summary:
While failures in the Rust updater aren't expected, at least one valid case
requires requires retrying the operation in Python: old-style LFS pointers.
When these are stored in packfiles/indexedlog, only the Python code knows how
to deal with them, and thus the operation needs to be retried there.
Reviewed By: DurhamG
Differential Revision: D20603709
fbshipit-source-id: 7d24ba573f0ff540906d909f1b4440fd4d3469a6
Summary:
The ContentStore cannot deserialize LFS pointers stored in packfiles, to avoid
potential damage, let's refuse to update LFS blobs. A proper solution will be
built in a separate diff.
Reviewed By: DurhamG
Differential Revision: D20576575
fbshipit-source-id: 4e4ce6a9432157e2ce69881c0079e943ea3f3acd
Summary:
Instead of having the magic number 0x2000 all over the place, let's move the
logic to this method.
Reviewed By: DurhamG
Differential Revision: D20637749
fbshipit-source-id: bf666f8787e37e6d6c58ad8982a5679b7e3e717b
Summary:
On Unix, pretend that NTFS doesn't support symlinks. While this isn't
technically true, NTFS on Linux is only used to alleviate performance issues
with `hg update` on Windows. With the pyworker code, I'm expecting these
performance issues to disappear allowing this code to be removed.
Reviewed By: ikostia
Differential Revision: D20527976
fbshipit-source-id: 4194f4b5af065de2e293b41b9d03e9d4ab6ea006
Summary:
Move them into a VFS struct, a future step will move the VFS code into its own
crate.
Reviewed By: DurhamG
Differential Revision: D20527977
fbshipit-source-id: 3250b05840688db72e1c43c72ec6defbc7f20851
Summary:
In this diff I create a function to handle the logic for choosing a subcommand
to run, returning a Future that we execute. By using `helpers::block_execute`
we're able to extract the spinning up of the fb303 server, which is the key
to making this an async function.
Within each of the subcommands in the new function `choose_subcmd`, this change
also asyncifies the logic inside the `async` block.
Reviewed By: krallin
Differential Revision: D20590742
fbshipit-source-id: a6298e6da3827638cc23369f72ebdd66b5051768
Summary:
This diff moves blame integration tests out from the main `test-scs.t`.
The change makes `test-scs.t` be able to complete and not time out anymore.
Reviewed By: ikostia
Differential Revision: D20629281
fbshipit-source-id: 67ba047442e7216a8addd0945c94d2f932eca08a
Summary: This diff moves lookup integration tests out from the `test-scs.t`.
Reviewed By: krallin
Differential Revision: D20620537
fbshipit-source-id: a8e1020901271b0e66dd4caa43ad3eddbf887a41
Summary:
We'd like to have health checks for scs server, at the very least to block bad
pushes before they reach production.
This diff adds a few counters that are generally useful, but the two we are going to
use for health checks is internal_failure_permille/invalid_request_permille[1]. This is the simplest
healtch check, and and it has a few drawbacks. In particular, it won't "notice" any
failures if a request doesn't get called often. It's not great, but I suggest to start with
this health check and expand it later.
Note that we use internal_failure_permille instead of doing (internal_failure /
total_requests), because canaries do not allow tracking for formulas (see
D20418610). And we might use failure rate in our canaries.
Also the diff explicitly logs "0" i.e. it logs "0" failures instead of not
logging anything. That makes it easier to set up detectors/alarms because we
don't have to deal with dead detectors.
[1] health check with internal_failure_permille will have a lower threshold
invalid_request_permille because this counter should be more reliable.
However it's probably worth alarming on invalid_request_permille as well to catch cases
e.g. where there's a bug in error classification itself.
Reviewed By: markbt
Differential Revision: D20644662
fbshipit-source-id: 694015c9add0702d47faaac8d36120d10c7003e2
Summary:
Plumb the read-only mount flag into FUSE on macOS. This fixes a build
warning and enables the --read-only mount flag.
Reviewed By: wez
Differential Revision: D20634119
fbshipit-source-id: a5e68cd163e36ceb6d86fd753844718ee9a5727f
Summary:
When we deserialize commit extras they come out as a str->str dict, so
let's make sure clients add values that are also strings. We do this by type
checking a serialization time. This fixes an issue where D19942522 assumed
extras were strings and hit a type error for globalrevs, which were ints.
Apparently the code that reads globalrevs from the extras never treats it as an
int anyway, so none of the reading code needed to be updated to convert the
string back to an int.
Reviewed By: singhsrb
Differential Revision: D20631889
fbshipit-source-id: 8c8b3c9a9f3369376e08146d670f2d6321df141f
Summary:
It's going to be useful to share certain structures between client and server.
Looking ahead, the plan is to share the segment graph along with all the
algorithms implemented for it.
Reviewed By: StanislavGlebik
Differential Revision: D20550951
fbshipit-source-id: f498a6b0cba1bcdd35fc9720125b223d7e891a44
Summary:
`iter_segments_with_parent` has a few more conditions attached to it than the
name would imply. We are renaming it to give a better sense of its true
behavior.
Reviewed By: quark-zju
Differential Revision: D20547631
fbshipit-source-id: 406f46b9de5efc9e8e6a8c4bc22ab18fa5bc54bb
Summary:
The main question I had while writing the tests was whether we expect a
specific order for Segments for `iter_segments_with_parent`. `InProcessStore`
will return the segments in the order that they were inserted.
Reviewed By: quark-zju
Differential Revision: D20501401
fbshipit-source-id: 48ceb78f3191c7425c1488a3392cf3167f7e7268
Summary:
First 6 methods implemented from the IdDagStore trait for the InProcessStore.
Any suggestions welcome.
Reviewed By: quark-zju
Differential Revision: D20499228
fbshipit-source-id: cb536a3a0136077ada78934d82a25d079a5bc809
Summary:
If `HGPLAIN` is set, and the non-legacy graph renderer is in use, use the ASCII
renderer to ensure compatible output on all platforms and configurations.
Reviewed By: farnz
Differential Revision: D20622425
fbshipit-source-id: b25a8d0526652bab07059492f7adbb684b5cbee7
Summary:
At this point all future-returning `fn`s in `benchmark_filestore`
are async.
Reviewed By: krallin
Differential Revision: D20615922
fbshipit-source-id: 8b9998e4d8933e46f0c2612270faefcb061eae37
Summary:
This diff is a combination of an asyncification and a refactor:
- isolatedthe logic for setting `blob` (which gets overwritten
5 times, often blocking on futures in the process) into it's own
function `get_blob`, which I think makes it a lot easier to see
which variables depend on one another.
- push all the rest of the logic into an async function
`run_benchmark_filestore`
Reviewed By: krallin
Differential Revision: D20615743
fbshipit-source-id: 9d0918e4078b8193c59758701c5dc5e744298d1d
Summary: Fix test that was broken by D20557896
Reviewed By: krallin
Differential Revision: D20619387
fbshipit-source-id: 3fd7ee501144528b18b7162f73dcf3d251fd5f2f
Summary:
From time to time we're experiencing the blobstore healer to crash
because its SQL queries timing out. The rootcause of the problem
is that the same blob_key may show up on the queue many times repeatedly
and the query is trying to select all occurences.
But, the original intention of blobstore healer is to act on a single
put operation across all blobstores. To be able to identify which
puts in the healer queue are part of the same operation we need
some unique id that we'll use per such operation, let's call it OperationKey.
corresponding configerator change to create db column: D20557659
NOTE: This diff has to be landed and rolled out first, before D20557700 is rolled out. I'm assuming that after some time since rolling out this diff all the rows in the production db will have proper `operation_key` value set.
Reviewed By: krallin
Differential Revision: D20557702
fbshipit-source-id: 404d9fdea6796b38193292d1bbd4b8cd4b5b3eb8
Summary:
In the previous diffs, I asyncified the api of all `fn`s in
`aliasverify`, but I still had old stream logic, which was hard to
read and required many clones of parameters to handle static lifetime
requirements.
This diff switches all the internal code over to new-style streams.
In the process of doing this, I was able to remove almost all of the
clones, and also switch `get_bounded` from needing to explicitly return
a `Future` via and `async` block to being a standard `async fn` returning
a `Result`.
Reviewed By: krallin
Differential Revision: D20582402
fbshipit-source-id: a98bb31075a41e8d6897b6893cd51ea0c5a94767
Summary:
This diff asyncifies the APIs in `aliasverify` so that all
functions are either `async fns` or (in one case where there were
challenges dealing with lifetimes) functions returning new-style
`Future`.
This diff does not asyncify the old stream API uses inside of functions,
and the remaining issues (complicated lifetime handling and extra
`clone` calls) are all related to using the old stream api, but the
function signatures should all be forward-compatible with new futures.
Reviewed By: krallin
Differential Revision: D20577861
fbshipit-source-id: 79ca2ef8f91ee9a1be9e27d8bf6f083dbc0bb7cd