Summary:
WIP
Mononoke API download for lfs
support get request
curl http://127.0.0.1:8000/{repo_name}/lfs/download/{sha256}
Reviewed By: StanislavGlebik
Differential Revision: D9850413
fbshipit-source-id: 4d756679716893b2b9c8ee877433cd443df52285
Summary:
Streaming clones are a neat hack; we get to send files down to the
Mercurial client, which it then writes out as-is. Usefully, we can send no
files, and the protocol still works.
Set up the capabilities etc needed so that we send streaming clones to
Mercurial clients, even if they're rather useless so far.
Reviewed By: StanislavGlebik
Differential Revision: D9926967
fbshipit-source-id: b543e802adac38c8bc318081eec364aed87b0231
Summary:
It was broken because of D9849883.
The problem was in the following. repo-pull was created using
`hginit_treemanifest` which creates treemanifest server repo. This repo was
backfilling flat manifests, and this backfilling sends a getbunde request with
empty common heads.
The problem was fixed by using `hgclone_treemanifest`.
Reviewed By: farnz
Differential Revision: D9940386
fbshipit-source-id: 837be6fd27c8e5ee81634d223aa1a88101926961
Summary:
We've upgraded crates.io, and somehow broke Actix so that HTTP/2 is no
longer supported. For now, update the test to run with HTTP/1.1
Reviewed By: StanislavGlebik
Differential Revision: D9934777
fbshipit-source-id: c41aa5ad376dc7b07700f1d1d1b30ff9ff694f68
Summary: It will be useful outside of pushrebase library as well
Reviewed By: farnz
Differential Revision: D9789811
fbshipit-source-id: c851df8a8cce8b1c26daa09b7fe2ffa40f290160
Summary:
Profiling showed that since we are inserting objects into blobstore
sequentially it takes a lot of time for long stacks of commit. Let's do it in
parallel.
Note that we are still inserting sequentially into changesets table
Reviewed By: farnz
Differential Revision: D9683037
fbshipit-source-id: 8f9496b97eaf265d9991b94243f0f14133f463da
Summary:
Pushrebase should send back the newly created commits. This diff adds this
functionality.
Note that it fetches both pushrebased commit and current "onto" bookmark.
Normally they should be the same, however they maybe different if bookmark
suddenly moved before current pushrebase finished.
Reviewed By: lukaspiatkowski
Differential Revision: D9635433
fbshipit-source-id: 12a076cc95f55b1af49690d236cee567429aef93
Summary:
This diff fills missing parts of push-rebase implementation
- `find_closest_root` - find closest root to specified bookmark
- `find_changed_files` - find file affected by changesets between provided `ancestor` and `descendant`
- `intersect_changed_files` - rejects push rebase if any conflicts have been found
- `create_rebased_changes` - support for merges
- `do_pushrebase` - returns updated bookmark value
Reviewed By: StanislavGlebik
Differential Revision: D9458416
fbshipit-source-id: c0cb53773eba6e966f1a5928c43ebdec761a78d3
Summary:
Modify the lookup() RPC function to be able to accept either a
bookmark or commit hash. A commit hash lookup is attempted first, falling
back to a bookmark lookup if it fails.
Reviewed By: StanislavGlebik
Differential Revision: D9457349
fbshipit-source-id: 78db21c01c498b045f5781097cb12f7220a40999
Summary: Now pushrebasing stacks as well. Again, still no conflicts checks
Reviewed By: aslpavel
Differential Revision: D9359807
fbshipit-source-id: 9f6e7a05b45fb80b40faaaaa4fe2434b7a591a7c
Summary:
Initial implementation of pushrebase. At the moment it processes just one commit, but after implementing stub function it should work for every case.
Note that there is a special PushrebaseError. This is necessary to distinguish between infra error (i.e. mysql is unavailable) and expected errors like conflicts.
Reviewed By: aslpavel
Differential Revision: D9306815
fbshipit-source-id: 7c3f91b17c6270537d63e8c9dba8116f96840ece
Summary: Just failing for now, next diffs will add an actual functionality
Reviewed By: farnz
Differential Revision: D9306814
fbshipit-source-id: c515f2e742833833d73bce08dbea1ddbb7e2ae79
Summary:
First step of implementing pushrebase algorithm. Save the commits that client
has sent us. The parts that client sends us are the same as in normal push
except for the names and parameters.
Reviewed By: farnz
Differential Revision: D9304750
fbshipit-source-id: d5be6635c0cf1a14a66a5fed5ba13f344195e8bc
Summary: ugh, yet another case of a hidden dependency.
Reviewed By: StanislavGlebik
Differential Revision: D9318498
fbshipit-source-id: 5fcd25081b5033cbef9c5f137e616348c5d6ced9
Summary: Adds proper url decoding for is_ancestor, so that special characters can be encoded in the url.
Reviewed By: kulshrax
Differential Revision: D9325467
fbshipit-source-id: d3ff60e004be8d254ea6f7288188adf54ab7ff5f
Summary:
I was seeing this test fail intermittently - increasing the loop from
40 iterations (4 seconds) to 100 iterations (10 seconds) makes it reliably pass
Reviewed By: StanislavGlebik
Differential Revision: D9333501
fbshipit-source-id: 31d842e2ec9a5bfd2e6d5d31ada85a6a891ff037
Summary: Mononoke stopped logging such a fine graned logs per commit because they are noisy, the remainder of the gettreepack test should be good enough to verify that the thing in test works
Reviewed By: sunshowers
Differential Revision: D9318050
fbshipit-source-id: 141a673731ceff0092f8f3e4209efb69cf481640
Summary: - Make `Bookmakrs` work with `ChangsetId` instead of `HgChangesetId`
Reviewed By: StanislavGlebik, farnz
Differential Revision: D9297139
fbshipit-source-id: e18661793d144669354e509271044410caa3502a
Summary:
Set a panichandler by default in cmdlib::get_logger to make sure
everyone gets one set. It configures itself to exit the process so that we
don't leave it in a half-broken state.
The Mononoke server was already using a panic hook, but this replaces it with
one that prints more detail about what was going on at the time.
Reviewed By: StanislavGlebik
Differential Revision: D9234587
fbshipit-source-id: bb51790a60b1ee545a364b4b92e09ec950788684
Summary:
Alas, the diff is huge. One thing is changing Changesets to use ChangesetId.
This is actually quite straightforward. But in order to do this we need to
adapt our test fixtures to also use bonsai changesets. Modifying existing test
fixtures to work with bonsai changesets is very tricky. Besides, existing test
fixtures is a big pile of tech debt anyway, so I used this chance to get rid of
them.
Now test fixtures use `generate_new_fixtures` binary to generate an actual Rust
code that creates a BlobRepo. This Rust code creates a bonsai changeset, that
is converted to hg changeset later.
In many cases it results in the same hg hashes as in old test fixtures.
However, there are a couple of cases where the hashes are different:
1) In the case of merge we are generating different hashes because of different
changed file list (lukaspiatkowski, aslpavel, is it expected?). this is the case for test
fixtures like merge_even, merge_uneven and so on.
2) Old test fixtures used flat manifest hashes while new test fixtures are tree
manifest only.
Reviewed By: jsgf
Differential Revision: D9132296
fbshipit-source-id: 5c4effd8d56dfc0bca13c924683c19665e7bed31
Summary: This commit adds support for changeset information retriving at `/<repo>/changeset/<commit hash>`.
Reviewed By: StanislavGlebik
Differential Revision: D8880547
fbshipit-source-id: ed68c577316693e0c685c347405b5d344d1bc87e
Summary: This commit adds support for `/<repo>/tree/<treehash>` (retrieving tree content by hash) .
Reviewed By: StanislavGlebik
Differential Revision: D8870870
fbshipit-source-id: 8b3271c819e47d112a8b44097f626360a05540d1
Summary: This commit implements the `<repo>/blob/<blobhash>` API that Eden needs.
Reviewed By: StanislavGlebik
Differential Revision: D8870300
fbshipit-source-id: eca9dc434c8fb584dfba1542c5242fbee18e6619
Summary: This commit adds support to ls operation that lists files in a folder at some commit.
Reviewed By: StanislavGlebik
Differential Revision: D8729389
fbshipit-source-id: cad6d02da075e94b5269cc18052a5a3916ddac86
Summary:
Added support for queries which use bookmark names in place of node hashes. This involved:
* Creating a method `string_to_bookmark_changeset_id`, which takes a string, treats it as a bookmark, and tries to find the corresponding changeset id in the repo.
* Modifying the `is_ancestor call` in `MononokeRepoActor` to try to interpret the query strings as bookmarks if they can't be interpretted as node hashes.
* Introducing the `cloned` crate from `//common/rust` into the API server to make the above methods cleaner.
* Modifying the integration test to add a bookmark to the test repo and attempt querying using the bookmark name.
Reviewed By: fanzeyi
Differential Revision: D8976793
fbshipit-source-id: 3a2b58cac0fb80ee18fad8529bd58af5b54f85ef
Summary: There was a change in hg replycaps in D8958866, let's update the tests
Reviewed By: farnz
Differential Revision: D8966069
fbshipit-source-id: 72606943d9751b66705b36fbf39ad7dc0702627c
Summary:
Adding support to the Mononoke API server for naive is_ancestor queries using a BFS.
The API server now supports queries as:
Request URL: "{repo}/is_ancestor/{proposed_ancestor}/{proposed_descendent}"
where the arguments in the URL are:
- repo: the name of the repo to query reachability in
- proposed_ancestor: a 20 byte hex encoded string representing a node hash
- proposed_descendent: a 20 byte hex encoded string representing a node hash
Response: One of:
- the string, "true", if 'proposed_ancestor' is an ancestor of 'proposed_descendent' in 'repo'.
- the string, "false", if the above condition isn't satisfied.
- an error if the query couldn't be performed
This involved adding:
- new enum values for the MononokeRepoQuery and MononokeRepoResponse structs for 'IsAncestor' queries and responses.
- a dependency on the 'mononoke/reachabilityindex' crate.
- a 'is_ancestor' function to the MononokeRepoActor struct, which delegates queries to a GenerationNumberBFS index.
- appropriate url handling to the server object in main
- New enums to the API server ErrorKind, and appropriate down casting from the ReachabilityIndex ErrorKind.
- Integration tests which made the test repo in test-apiserver.t have a few branches, and queries the API server for reachability of pairs of commits.
Reviewed By: fanzeyi
Differential Revision: D8844221
fbshipit-source-id: 1ba102fede378688243827850ff67aabc587a748
Summary: This commit renames `/blob/` to `/raw/`. This helps users to distinguish the proposed "get file content by hash" API from the original "get file content by path".
Reviewed By: kulshrax
Differential Revision: D8869635
fbshipit-source-id: 79d9cdaeb7e4e55b3d804d4530fb17835104cc32
Summary:
Add the integration test for (at the moment) missing pushrebase functionality.
At the moment it doesn't do anything because pushrebase is not implemented yet
Reviewed By: farnz
Differential Revision: D8858185
fbshipit-source-id: c200c9fbfeb3d28d27dd5ab3c4d6bd1145fd9f0b
Summary:
So the fixtures were broken -- there was apparently a bug that would
cause p2 of root tree manifests to always be null. Fix them by regenerating
trees.
Also the directory name the tarball got extracted to had hyphens in it instead
of underscores. Fix that to make it all underscores.
I used the following commands to do this -- apologies for the hacky zsh:
```
$ for tar in *.tar.gz; do (x=${tar//.tar.gz/}; cp $x.tar.gz ~/local/tmp &&
pushd ~/local/tmp && rm -rf $x ${x//_/-} && tar xf $x.tar.gz && mv ${x//_/-} $x
&& pushd $x/.hg/store && rm 00manifesttree.i && hg backfilltree && popd && tar
cf $x.tar.gz $x && popd && cp ~/local/tmp/$x.tar.gz .); done
$ buck run scm/mononoke/tests/fixtures:regenerate -- *.tar.gz --overwrite
```
Reviewed By: farnz
Differential Revision: D8854625
fbshipit-source-id: 08be7a1e89150b70a308343b43b299a0030708b9
Summary:
This commit added three options to specify locations to SSL certificates so the API server will accept encrypted traffic.
Currently this only works in HTTP/1.1 due to a bug in HTTP/2 parsing in actix-web. Once they fixed the bug upstream we will be able to serve HTTP/2 traffic as well.
Reviewed By: jsgf
Differential Revision: D8703861
fbshipit-source-id: 0d4e68276013a8aeb6ee006e5175b8caeba767cb
Summary:
There seems to be a deadlock in the internals of the now outdated tokio_core.
After applying the modern tokio::run the deadlock is not being triggered.
Reviewed By: farnz
Differential Revision: D8783183
fbshipit-source-id: 47a7d1d8e2756ea4d40812d0b8a6c850d7f7e9f8
Summary: This will be useful for the bonsai verification tool.
Reviewed By: StanislavGlebik
Differential Revision: D8792562
fbshipit-source-id: f409d0fa042528b04462a1539fd3c2a8064a4f6e
Summary: - integration test of were failing on my devserver as `{http|https}_proxy` environment variables
Reviewed By: StanislavGlebik
Differential Revision: D8804505
fbshipit-source-id: fafea55ed80897b24ad816bc26b241e64cd8b913
Summary:
hgcli will start logging stuff as well and it will use the same session_uuid as the server.
This also includes logging the user and source hostname.
Reviewed By: farnz
Differential Revision: D8750663
fbshipit-source-id: 7ebc8b6c10b7560d985fd23e9e3f2645f3bd0a1c
Summary: Session UUID will help identify the issues on Mononoke side whenever the client encounters problems
Reviewed By: StanislavGlebik
Differential Revision: D8732396
fbshipit-source-id: 35d04b0d56be0cfc2c608f08287a2b1d236a96e3
Summary:
This diff refactors the server config repository to support storing and loading of hooks. In the new structure each repo lives in its own directory and the config file for the server is called "server.toml".
Hooks can be referenced by relative or absolute paths allowing either local or common hooks to be loaded.
Reviewed By: StanislavGlebik
Differential Revision: D8625178
fbshipit-source-id: 62c8c515a0fbbf7a38cfc68317300d8f42eb4d7a
Summary:
New parameter depth was added. Mononoke just ignores, which if correct, but can
be less efficient. However test-gettreepack.t was failing because it didn't
supply the parameter. This diff fixes it by sending a big depth to the server
P. S.
Attached task tracks the support of depth parameter on Mononoke server-side
Reviewed By: lukaspiatkowski
Differential Revision: D8712906
fbshipit-source-id: 8246344b3fa39f00eb39f2262b4aa5aa74834be2
Summary:
They were broken since 2646d41a64917d3e50f662b0b4b628ccfdbb05a8.
It added enabled_repo repository to config, however, no enabled_repo has ever been
blobimported. Since we don't really need enabled_repo, let's remove it
Reviewed By: farnz
Differential Revision: D8712891
fbshipit-source-id: 360f2c9564087d3a8411871335aa40fa81e38c0c
Summary:
Add a per-repo config flag to repos to be configed without being
enabled. Setting "enabled = false" will make Mononoke completely ignore the
repo config. If not present, "enabled" is assumed to be true.
Reviewed By: farnz
Differential Revision: D8647161
fbshipit-source-id: 2646d41a64917d3e50f662b0b4b628ccfdbb05a8
Summary:
Use tls for connection between hgcli and Mononoke server always, even for
localhost connections[1]
The setup is similar to tls setup of Eden server.
[1] This is not necessary of course, but adding an option to bypass tls
connection may result in accidental use of it in prod. However if it turns out
to be too unusable, we can add such option in the future
Reviewed By: jsgf
Differential Revision: D8644299
fbshipit-source-id: 0898e30e33b718e13a766763479f3adf9323ffe7
Summary: This commit upgraded openssl, enabled alpn for actix-web and added tokio-codec with fixes due to the upgrade.
Reviewed By: StanislavGlebik
Differential Revision: D8682673
fbshipit-source-id: 8c7cadfd6c0c7b016202f6cb038eb4951d0f9333
Summary: This commit changes the API from directly asking for NodeHash to changeset id and path of the file.
Reviewed By: StanislavGlebik
Differential Revision: D8628826
fbshipit-source-id: 1fa37cf36db0ca00530f3a60de78da1d1d232398
Summary: This commit adds /status to return ok for Tupperware health check.
Reviewed By: kulshrax
Differential Revision: D8629282
fbshipit-source-id: 8f36bf73d4d9399721c68649e6b475362770889b
