Summary: Integrate http hash prefix lookup into the pull operation. One unfortunate change here is that if the prefix is ambiguous, we're only able to output possible full hashes as suggestions. Previously we'd also print commit log information. To retain that we'd need to add an error option to the response and have the server send back an error message with the log information or send another request to download the extra information.
Reviewed By: andll
Differential Revision: D30716050
fbshipit-source-id: 33f8bc38b0bfe7fce4ec11cd8def7feda3b3d3da
Summary:
Remove some `EDEN_HAVE_*` config definitions that are no longer referenced
anywhere.
Reviewed By: xavierd
Differential Revision: D30797847
fbshipit-source-id: e2d9e0049cfd984c0814f1ac61ffddb1185e9904
Summary:
As title, sampling group determines the sampling rate at which an FS event is logged. The higher the sampling group the more heavily its events are dropped, thus, more frequent events are assigned to the higher sampling groups.
I ran activity recorders on a few workflows, buck build, getdepts, and vscode editing and came up with the following assignment. Note that only a subset of events are assigned to a sampling group (so events not included will not be logged) as we just start to tune the sampling rates and these events should be good for a start.
```
Group1 (1/10)
FUSE_MKDIR
FUSE_RMDIR
FUSE_CREATE
FUSE_RENAME
Group2 (1/100)
FUSE_WRITE
FUSE_LISTXATTR
FUSE_SETATTR
Group3 (1/1000)
FUSE_GETXATTR
FUSE_GETATTR
FUSE_READ
FUSE_READDIR
Group4 (1/10000)
FUSE_LOOKUP
```
For reference, here are the counts of FS events of a cold buck build. The frequencies of other workflows are similar.
```
FUSE_LOOKUP 60.09 98733
FUSE_READ 12.80 21037
FUSE_GETXATTR 8.91 14645
FUSE_FORGET 8.01 13162
FUSE_GETATTR 5.55 9116
FUSE_READDIR 3.21 5270
FUSE_LISTXATTR 0.59 969
FUSE_READLINK 0.54 892
FUSE_STATFS 0.21 338
FUSE_WRITE 0.04 64
FUSE_CREATE 0.02 28
FUSE_RENAME 0.01 23
FUSE_SETATTR 0.01 13
FUSE_UNLINK 0.00 6
FUSE_RMDIR 0.00 1
FUSE_MKDIR 0.00 1
FUSE_MKNOD 0.00 1
```
Reviewed By: xavierd
Differential Revision: D30770533
fbshipit-source-id: 90be881ddbeba2113bbb190bdb1e300a68f500a0
Summary: The new `EdenApiHandler` framework for defining EdenAPI endpoints provides a common place where responses are encoded. This diff adds automatic content compression at this point, using the received `Accept-Encoding` header from the request to determine what compression, if any, should be used. As a result, all endpoints that implement `EdenApiHandler` will get compression for free.
Reviewed By: yancouto
Differential Revision: D30553242
fbshipit-source-id: 9eda54cbf81dd1e03abec769744c96b16fad64ea
Summary:
It can sometimes be difficult to work out from the logging which commit cloud
requests came from which client repo. Previously you could often infer it from
the client identities, however if the request is proxied, the originating hostname can be
lost, and it still doesn't handle the case where the host contains multiple
repos.
This diff adds a new `ClientInfo` struct, which is included by the client
on every `get_references` and `update_references` request. This is logged
by the service, allowing us to correlate which client it came from, and what
workspace version the client had at that time.
Reviewed By: StanislavGlebik
Differential Revision: D30697839
fbshipit-source-id: 8fe2e03f0be2f443f8ae1814f083c04ba5d1805e
Summary: It was not used and hard to implement retry with
Reviewed By: yancouto
Differential Revision: D30716647
fbshipit-source-id: a90b629f7758486c9e526d1eaf3fd29da305f2e7
Summary:
D30704344 (5704ad51f6) upgraded tokio for the buck build. We need to do the same for
the non-buck build. This unbreaks hgbuild.
Also clean up some compiler warnings while I'm here.
Reviewed By: fanzeyi
Differential Revision: D30798315
fbshipit-source-id: 47005c7674d87196aab42b3ddf2194acced3bb6c
Summary:
We have two mode of deriving data: the "normal" way and using backfilling.
Backfilling is different from "normal" mode in that it derives a few commits at
once, and saves them all to blobstore at once.
Backfilling mode seemed to have helped us when we need to derive a lot of data
(e.g. backfill the whole repo). But
a) We don't know how much it helps, and we don't know if it depends on the repo
b) We don't know if it helps when we derive data for newly landed commits (i.e.
we use "backfill" mode in derived data tailer to derive data for latest public
commits)
So this diff adds a bit of logging to a separate scuba table so that we can get
an idea about things like:
1) How long does it take to derive a stack of commits?
2) Where do we spend most of the time (e.g. deriving, saving the blobs, saving
the mapping).
Reviewed By: mitrandir77
Differential Revision: D30805504
fbshipit-source-id: d82c905cafa87459990d74769a0dddcc91fac174
Summary:
It allows us to do 3 things:
1) Remove derive function
2) Add support for backfill mode so that we can compare perf with and without
it
3) Use all derived data types, and not just 3 of them
Reviewed By: krallin
Differential Revision: D30804258
fbshipit-source-id: 604723a3d845a60cfd94b4e090a121f5b5191536
Summary: This command can be useful to split a large bonsai commit into a smaller one
Reviewed By: mitrandir77
Differential Revision: D30776789
fbshipit-source-id: dc56d7c51eb0e9e0988dcba868c6008ebf488927
Summary:
While we don't really need it, creation mononoke matches fail if they are not
present. Let's just enable it here - it's not a bad thing to initialize them
Reviewed By: mitrandir77
Differential Revision: D30780463
fbshipit-source-id: c4199c6711ae7bd9641e9f51643b94d020051dbd
Summary: The code for accessing config fields had a lot of repetitive boilerplate. Let's move that to a helper function.
Reviewed By: andll
Differential Revision: D30785932
fbshipit-source-id: fb4d47337a27bd6e75eeb38d5a9d1de5b1fac6ce
Summary: Implement serverside graph endpoint for fetching the mapping of commits to commit parents for the missing segment of a commit graph. This implementation uses the find_commits_to_send method from the get_bundle_response library. What may be missing from pull and the old bundle protocol now is mutation markers.
Reviewed By: yancouto
Differential Revision: D30485672
fbshipit-source-id: ba3a30d9e482d60831cbe7a8e89f20dab947d9a1
Summary:
Since the find_commits_to_send method was added, common is already a
hashset not a vector, so it doesn't needed to be converted to a hashset.
Reviewed By: quark-zju
Differential Revision: D30622028
fbshipit-source-id: e5d1b6c60115d13c906b25142043652ba9e89d70
Summary:
The primary branch in the fb303 github repository was renamed from `master` to
`main`. Update the `fb303-source` manifest to reflect this.
The main `fb303` manifest was already updated in D30700180 (d84d56c04c). The
`fb303-source` manifest is a separate manifest that only fetches the fb303
source code without building it. This is only used by the `eden_scm`
manifest.
Reviewed By: fanzeyi
Differential Revision: D30794650
fbshipit-source-id: 0e7790f8ceb737ce2f26ab213e1f0f94c5b3f045
