Summary: This diff added "manifest" field to Mononoke API Server's `get_changeset` method, which is required to implement Eden's `BackingStore::getTreeForCommit`
Reviewed By: StanislavGlebik
Differential Revision: D13966576
fbshipit-source-id: d2444b3aab477ab76efa7baddf114d90fa7a7a77
Summary: Add a thrift method `get_tree` to Mononoke API Server
Reviewed By: StanislavGlebik
Differential Revision: D13964085
fbshipit-source-id: c0d31e4e13079bec49d480c116ebb300c7e4260f
Summary: This is needed to enable D14085659. Currently we do not have access to the core context when actually logging stuff. This change will not affect http api as Im not sure how to send the context to the middleware.
Reviewed By: StanislavGlebik
Differential Revision: D14081760
fbshipit-source-id: 6c3e8e1e7d153fbac62c842130e9caf51e5aac5a
Summary: use common funcions to parse --myrouter-port, this is a simple clean up
Reviewed By: StanislavGlebik
Differential Revision: D14084003
fbshipit-source-id: 63d6c8301e977faead62cb1c705bac372d56594e
Summary: New commits should be logged to scribe, these will be used to trigger the update for the hg clone streamfile.
Reviewed By: lukaspiatkowski
Differential Revision: D14022599
fbshipit-source-id: a8a68f12a8dc1e65663d1ccf1a5eafa54ca2daf0
Summary:
Update the `fb303.FacebookService` type to derive from the new
`fb303_core.BaseService` definition. This is in preparation to begin moving
most of the fb303 methods to `BaseService`.
Unfortunately the Rust thrift compiler does not handle service inheritance
very well, so all existing rust thrift servers need to be updated to handle
this change.
Reviewed By: wez
Differential Revision: D13683717
fbshipit-source-id: c6ba8dcbadd05d674b7a75ea531500268e4bc3f2
Summary:
Populating skiplist at startup takes up to 3 minutes when not compiling in prod mode. This allows developers to turn this off and just use an empty skiplist (this only affects historical queries, and those can still be tested using recent commits)
The reason for negating the flag is to not hide core functionality behind a flag.
Reviewed By: StanislavGlebik
Differential Revision: D13986936
fbshipit-source-id: 053ecbd27aeeaa97f03194933e1d904a3e60ffcc
Summary:
I decided against using the naming convention maybe_ancestor and maybe_descendant as I believe the name is_ancestor already implies this uncertainty.
Skiplist is used instead of bfs, this improves performance significantly when using very old commits. For now we do not update skiplist as more recent commits are in cache anyway.
Reviewed By: StanislavGlebik
Differential Revision: D13917333
fbshipit-source-id: 21b49d920ff473c953a952ee3c6d7b55565f98ac
Summary: HgFileNodeId is a stronger typed id, so it is prefered to use it instead of HgNodeHash whenever it is identifying a filenode
Reviewed By: aslpavel
Differential Revision: D13986172
fbshipit-source-id: c0334652345acb868e86c38b8c0045e9c023c176
Summary: The Copy trait means that something is so cheap to copy that you don't even need to explicitly do `.clone()` on it. As it doesn't make much sense to pass &i64 it also doesn't make much sense to pass &<Something that is Copy>, so I have removed all the occurences of passing one of ouf hashes that are Copy.
Reviewed By: fanzeyi
Differential Revision: D13974622
fbshipit-source-id: 89efc1c1e29269cc2e77dcb124964265c344f519
Summary:
Apiserver should log success and not rely on scmquery-proxy for that. Also did some refactoring:
1. cleaned up .timed methods in the thrift section
2. Moved the creation of ScubaSampleBuilder to one central location so common logging data doesnt have to be added to both http and thrift section
Reviewed By: StanislavGlebik
Differential Revision: D13895685
fbshipit-source-id: 3c6295e172f8ccaabaf38f436b304b5cb717deca
Summary:
`blobrepo_factory` is a crate that knows how to create blobrepo given
a configuration i.e. it creates blobstores, filenodes, changesets etc and
initializes blobrepo with them.
`post_commit` is a small part of blobrepo which can also be extracted from
blobrepo crate.
There are a few upsides with this approach
1) Less dependencies on Blobrepo, meaning we have to rebuild it fewer times
2) BlobRepo compilation is faster
Reviewed By: jsgf
Differential Revision: D13896334
fbshipit-source-id: 1f5701341f01fcefff4e5f9430ddc914b9496064
Summary: Add a `/getfile/` endpoint that takes a filenode and returns Mercurial file content. This is similar to the `/blob/` endpoint, except the returned content can also include copy information in the case that the file was copied. This matches the exact format in which Mercurial stores file content, so the resulting output can be directly written to disk.
Reviewed By: chadaustin
Differential Revision: D13876096
fbshipit-source-id: 0417cb1f993c4c93687e364b1647490c3a9040bc
Summary:
At From<Error>::from errors are automatically downcast into an InternalError. While we unwrap them for http this was not done for thrift, resulting in an InternalError being returned for everything. By unwrapping the error first thrift now returns more meaningful errors.
In addition I added a new errorkind for ls_v2, this is purely for internal structure as the client will still see an InvalidInput error.
Reviewed By: StanislavGlebik
Differential Revision: D13881374
fbshipit-source-id: bd2e43a0ac7e5abdbf41068c3b1ab37681f03009
Summary:
Let's split reachability index crate. The main goal is to reduce compilation
time. Now crates like revsets will only depend on traits definition but not on
the actual implementation (skiplist of genbfs).
Reviewed By: lukaspiatkowski
Differential Revision: D13878403
fbshipit-source-id: 022eca50ac4bc7416e9fe5f3104f0a9a65195b26
Summary:
Currently if a crate depends even on a single type from metaconfig then in
order to compile this trait buck first compiles metaconfig crate with all the
logic of parsing the configs.
This diff split metaconfig into two crates. The first one just holds the types for
"external consumption" by other crates. The second holds the parsing logic.
That makes builds faster
Reviewed By: jsgf, lukaspiatkowski
Differential Revision: D13877592
fbshipit-source-id: f353fb2d1737845bf1fa0de515ff8ef131020063
Summary:
The main reason to do it is to remove dependency from cmdlib to repo_client.
repo_client depends on a lot of other crates like bundle2-resolver, hooks etc.
And it means that in order to compile mononoke_admin we need to compile these
crates too. By moving open_blobrepo into blobrepo crate we are removing
unnecessary dependencies.
Also let's remove unused blobrepo type
Reviewed By: aslpavel
Differential Revision: D13848878
fbshipit-source-id: cd3d04354649cdb5b2947f08762051318725c781
Summary: This is to take the guesswork out of revisions. MononokeRevision now states explicitely whether the value is a hash or bookmark. To roll this out I will prepare an updated scmquery-proxy version, test everything on shadow. Then use the sitevar to disable mononoke usage for a short time while both apisever and scmquery-proxy are rolled out simultaneously
Reviewed By: StanislavGlebik
Differential Revision: D13817040
fbshipit-source-id: d5eee7cf9ac972fb1313a10a17bf60c4545054af
Summary: Ran rustfmt on the complete folder
Reviewed By: HarveyHunt
Differential Revision: D13803235
fbshipit-source-id: ccb41d7be8099055b8ea8dec57ce5e04b38b9461
Summary: Create the corresponding endpoint for thrift calls to get_commit_v2. For now this only supports the non optional fields.
Reviewed By: StanislavGlebik
Differential Revision: D13730602
fbshipit-source-id: fd9d845620c864bf7dade13d810e98270425ea00
Summary: Log all thrift calls to scuba. Since the params are in thrift format we log them both in json and human readable format (e.g. path is binary thus not really readable in json format). In addition both http and thrift now log the request type.
Reviewed By: StanislavGlebik
Differential Revision: D13730245
fbshipit-source-id: 3419ec067a9066e181210d184195fbd02980c1e0
Summary: `BlobRepo` is cheaply clonable and doesn't need to be `Arc`'d.
Reviewed By: StanislavGlebik
Differential Revision: D13726727
fbshipit-source-id: 0b983c7c4625f47df8a11da5781b7777cc38d72f
Summary:
Reformat several rust files with the current rustfmt, to make the linter
happy.
Reviewed By: yfeldblum
Differential Revision: D13683205
fbshipit-source-id: f7a02dae0fbe095b6acde4de380aca2acfedf39d
Summary: MyRouter and cache support will come in next diffs
Reviewed By: StanislavGlebik
Differential Revision: D13465146
fbshipit-source-id: 0ede0e875d4a49794ff71173bd0d1563abb3ff08
Summary:
We need to add python exports to our TARGETS file to enable scmquery
to create a ServiceRouter Client for mononoke-api
Reviewed By: StanislavGlebik
Differential Revision: D13651275
fbshipit-source-id: ba5a4eb3665dae1ea127f4ceb96c1ce62e1e4563
Summary: There's nothing Mercurial-specific about identifying a repo. This also outright removes some dependencies on mercurial-types.
Reviewed By: StanislavGlebik
Differential Revision: D13512616
fbshipit-source-id: 4496a93a8d4e56cd6ca319dfd8effc71e694ff3e
Summary:
We've moved to the new config structure, I forgot to make the same changes to
apiserver
Reviewed By: lukaspiatkowski
Differential Revision: D13358407
fbshipit-source-id: cec81a21518cdb3c91dabb93e220d1ba3e25d02c
Summary:
`env_logger` was enabled in D12934682 to assist with debugging production TLS
issues with the Mononoke API server VIP. It turns out that this was actually
not neccesary due to the slog_stdlog crate. As such, remove env_logger from the
API server.
Reviewed By: StanislavGlebik
Differential Revision: D13328349
fbshipit-source-id: c5ed33d5afe96f4823ce637de892188e95c647fa
Summary: Restructure the configs so that we can specify more than one blobstore
Reviewed By: lukaspiatkowski
Differential Revision: D13234286
fbshipit-source-id: a98ede17921ed6148add570288ac23636b086398
Summary: Set the proper context for sessions.
Reviewed By: liubov-dmitrieva
Differential Revision: D13258641
fbshipit-source-id: edd18d4abc8f5475e0d2ac8395dfc877b2dd5958
Summary:
Config repo proved to be tricky to understand and hard to use. Let's just use
toml files.
Reviewed By: farnz
Differential Revision: D13179926
fbshipit-source-id: 3a44ee08c37284cc4c189c74b5c369ce82651cc6
Summary:
Let's add a command that builds and reads a skiplist indexes. This indexes will
be used by getbundle wireproto request to decrease the latency and cpu usage.
Note that we are saving only the longest "jump" from the skiplist. This is done
in order to save space.
Reviewed By: jsgf
Differential Revision: D13169018
fbshipit-source-id: 4d654284b0c0d8a579444816781419ba6ad86baa
Summary:
Let's make it use the same ChangesetFethcer as getbundle already does. It will
be used in the next diffs
Reviewed By: lukaspiatkowski
Differential Revision: D13122344
fbshipit-source-id: 37eba612935a209098a245f4be0af3bc18c5787e
Summary:
Most of our revsets are already migrated, let's migrate skiplists as well since
we want to use them in getbundle requests.
Reviewed By: lukaspiatkowski
Differential Revision: D13083910
fbshipit-source-id: 4c3bc40ccff95c3231c76b9e920af5db31b80d01
Summary: Now that Tupperware is using `/health_check`, there are no users of `/status`, so remove this endpoint.
Reviewed By: StanislavGlebik
Differential Revision: D12908785
fbshipit-source-id: be9bfae9453143a6f4b26f7fc6cbc68a3f1adc5c
Summary:
Mononoke-apiserver now will parse http header in order to get "host" url.
Based on that Host url, batch request will return the link to the same host.
Requirement:
LFS protocol consists of two parts:
- "batch" request, GET request
- HG client sends ids of the file it wants to download/upload
- API server should return valid url for downloading and uploading objects
"upload/download" request.
- Actual upload/download of the file
This diff allows API Server to return the link to itself on "batch" request. So Hg client will go to API Server again for uploading and downloading files.
Reviewed By: kulshrax
Differential Revision: D12941635
fbshipit-source-id: e56453b0d6239daa3848c285a1df09a4a869f2c8
Summary: We need more insight into what's happening inside of actix and tokio in order to debug operational issues with the API server. Unfortunately, this sort of low-level debug logs from libraries are not captured by slog. Since many of the API server's dependencies use `env_logger`, calling `env_logger::init()` in main will allow us to see the debug output from those libraries by specifying the `RUST_LOG` environment variable.
Reviewed By: HarveyHunt
Differential Revision: D12934682
fbshipit-source-id: 417201e509de66c3340c900a4914ea3a03a791fe
Summary:
This augments `/tree` to yields the size and content sha1 hash for the entries.
This is important for Eden and avoids additional round trips to the server.
The content hashing is the portion that I expect some push back on,
because it doesn't appear to be cached today and the implementation
here does a simplistic fetch and hash. By doing this we hope to
squash out a potential later fetch of the entire contents when
buck build is run.
Differential Revision: D10865588
fbshipit-source-id: c020ef07b99d8a5e8b2f8f7b699bf15e750d60a5
