Commit Graph

32 Commits

Author SHA1 Message Date
Stanislau Hlebik
59fa2bb650 mononoke: fix test
Reviewed By: lukaspiatkowski, farnz

Differential Revision: D13024053

fbshipit-source-id: 2d7d255deed313e98895ac25933a04052b3c5276
2018-11-12 07:14:21 -08:00
Simon Farnsworth
0e82cb3711 Advertise tree-only nature in our caps string
Summary:
We are a tree-only server; advertising this in our caps should ensure
that when we're source of truth at Facebook, anyone with pre-treemanifest
versions of Mercurial gets an insta-fail on connect, rather than a weird error
when they start exchanging data with us.

Reviewed By: HarveyHunt

Differential Revision: D12927232

fbshipit-source-id: 52c15c8a0f1842b6ca023f97228277d0fd9e8e38
2018-11-06 04:03:15 -08:00
Anastasiya Zhyrkevich
c4f7fae0bb LFS test from hg client to Mononoke server, with large files through
Summary:
Test is failing, as Mononoke server lfs support is not implemented yet.
Integration test for commands from hg client to Mononoke server.

\s(re) lines are added as after auto-save, the test script is formatted, and delete spaces at the empty lines.
In order to keep such lines, \s(re) could be added
In comparison of such line, pattern \s(re) is deleted and not compared.
See to mononoke/tests/integration/third_party/hg_run_tests.py for more information about comparison of the output lines.

Reviewed By: StanislavGlebik

Differential Revision: D10089289

fbshipit-source-id: 2962e80d919c21801d08990be190f2574c48646d
2018-10-08 16:05:59 -07:00
Stanislau Hlebik
c3b5ee6854 mononoke: add (re) for lines ending with whitespace in integration tests
Summary:
Many editors remove trailing whitespaces on save. That makes modifying these
files annoying. Adding ` (re)` mitigates the issue

Reviewed By: farnz

Differential Revision: D10237590

fbshipit-source-id: 1473f35023b878f21ff22bd5a5ccb5f11884cef3
2018-10-08 09:06:26 -07:00
Simon Farnsworth
0aaa4577bc Add minimal support to trigger streaming clones in Mercurial client
Summary:
Streaming clones are a neat hack; we get to send files down to the
Mercurial client, which it then writes out as-is. Usefully, we can send no
files, and the protocol still works.

Set up the capabilities etc needed so that we send streaming clones to
Mercurial clients, even if they're rather useless so far.

Reviewed By: StanislavGlebik

Differential Revision: D9926967

fbshipit-source-id: b543e802adac38c8bc318081eec364aed87b0231
2018-09-19 06:36:56 -07:00
Stanislau Hlebik
13f45ee98e mononoke: fix tests
Summary: There was a change in hg replycaps in D8958866, let's update the tests

Reviewed By: farnz

Differential Revision: D8966069

fbshipit-source-id: 72606943d9751b66705b36fbf39ad7dc0702627c
2018-07-24 02:37:00 -07:00
Lukas Piatkowski
221d51465b hgcli/server: move session_uuid creation to hgcli and log more information
Summary:
hgcli will start logging stuff as well and it will use the same session_uuid as the server.
This also includes logging the user and source hostname.

Reviewed By: farnz

Differential Revision: D8750663

fbshipit-source-id: 7ebc8b6c10b7560d985fd23e9e3f2645f3bd0a1c
2018-07-09 15:39:09 -07:00
Lukas Piatkowski
38b66202cb server: print the session_uuid on the client side
Summary: Session UUID will help identify the issues on Mononoke side whenever the client encounters problems

Reviewed By: StanislavGlebik

Differential Revision: D8732396

fbshipit-source-id: 35d04b0d56be0cfc2c608f08287a2b1d236a96e3
2018-07-04 11:49:54 -07:00
Rain ⁣
388474e1bf rename new_blobimport to blobimport
Summary: The old blobimport is dead.

Reviewed By: farnz

Differential Revision: D8372593

fbshipit-source-id: 63595dbcd3ac6da1cad3c5ae6761ef01be6619ed
2018-06-12 15:40:10 -07:00
Lukas Piatkowski
800b605a35 bundle2-resolver: multiple pushkey parts for pushing multiple bookmarks
Summary: Being able to push multiple bookmarks in a single hg push is required for using hg push as tailing of fbsource which contains few remote bookmarks

Reviewed By: StanislavGlebik

Differential Revision: D7743737

fbshipit-source-id: ba24445762baafbaa5b3295dc8995fe871f97872
2018-04-24 11:17:06 -07:00
Lukas Piatkowski
040c1a2b4d integration tests: use newblobimport in integration tests where possible
Summary: After ceasing recomputation of NodeHashes the newblobimport is working in many of our tests as a replacement of blobimport

Reviewed By: sid0

Differential Revision: D7707684

fbshipit-source-id: e7b4391916cd4a37968afd828f456a7b49ecabf9
2018-04-20 12:17:40 -07:00
Lukas Piatkowski
d0e66cc5f7 mercurial: distinguish between NodeHash used in mercurial Revlogs and in Mononoke Blobstore
Summary:
This codemod tries not to change the existing behavior of system, only introduce new types specific to Mercurial Revlogs.
It introduces a lot of copypasta intentionally and it will be cleaned in following diffs.

Reviewed By: farnz

Differential Revision: D7367191

fbshipit-source-id: 0a915f427dff431065e903b5f6fbd3cba6bc22a7
2018-03-22 12:24:35 -07:00
Stanislau Hlebik
12c9f55b9a mononoke: add listkeys wireproto
Summary:
This wireproto method is used by remotenames to update the list of
remotebookmarks. Implementation is the same as for listkeys bundle2 part.

Reviewed By: farnz

Differential Revision: D7271597

fbshipit-source-id: 8a75a93cae0e571d86d657e1c1d718a7fa0ab4ea
2018-03-21 14:24:55 -07:00
Lukas Piatkowski
4d4d90928f integration tests: use RocksDb based blobs rather than File based for testing
Summary: just to excersise our RocksDb blobstore which might come in handy more often than file based as it should be more efficient

Reviewed By: StanislavGlebik

Differential Revision: D7290069

fbshipit-source-id: ce776cfa14e43dc45cca796ef187655ba665d177
2018-03-16 13:41:55 -07:00
Stanislau Hlebik
b35bd5dd9d mononoke: pushkey part parsing
Summary:
Pushkey part is used to send bookmark updates from hg client to the server.
This diff does all the wireproto parsing, but doesn't actually apply bookmark
updates on the server.

Also this diff "implements" branchmap method. We have no plans to support it,
but currently remotenames extension calls it. So this diff adds a fake
implementation that always returns empty response.

Reviewed By: farnz

Differential Revision: D7150973

fbshipit-source-id: 6889c02a1105127b1805ef1fafa6fbe9c2d57e7d
2018-03-16 04:21:32 -07:00
Lukas Piatkowski
d24a501e9e blobrepo: write commit to HeadStore
Summary:
This is a hacky way of getting the push/pull working. We should instead remove the commits that are no longer heads from HeadStore and add those that are the new heads.
In this diff though we add all the commits as heads, just because this does not break the client

Reviewed By: farnz

Differential Revision: D7112279

fbshipit-source-id: 036f0fd230de52e96cbf4168c2cda7c2a1c5bd89
2018-02-28 14:15:13 -08:00
Lukas Piatkowski
995e15697e bundle2-resolver: use (NodeHash, RepoPath) as id for uploaded blobs
Summary:
From upload_entry perspective NodeHash is not unique for uploaded entry, but when combined with RepoPath it is. An example are two files with the same content, but different paths.

Additionally in this diff the requirement for uploaded blobs to be unique is loosened for Manifests, because the b2xtreegroup might contain duplicates of Manifests

Reviewed By: farnz

Differential Revision: D7087863

fbshipit-source-id: 7e9c2438db037fa171f1e65b6882b445e8c09f7a
2018-02-28 11:15:01 -08:00
Lukas Piatkowski
0ceab4951c bundle2-resolver: accept more than one Filelog chunk with the same path
Summary: It seems that my assumtion that Filelog End section follow every Filelog Chunk section was wrong, because there might be multiple Filelog Chunk sections being different version of the same file from different commits.

Reviewed By: farnz

Differential Revision: D7073007

fbshipit-source-id: 3baee3fd29c77aabf4173a618509de7ff88e4de6
2018-02-27 11:14:52 -08:00
Lukas Piatkowski
37aedb6a7c bundle2-resolver: fetch missing file content from BlobRepo in DeltaCache
Summary: The assumption that deltas can be decoded using only bundle2 was wrong, we need to be able to fetch content of other files from the BlobRepo for delta application.

Reviewed By: StanislavGlebik

Differential Revision: D7085957

fbshipit-source-id: 2f6803d7f61389c5ba38b1207ede42579b9cf2e6
2018-02-27 11:14:52 -08:00
Lukas Piatkowski
6bba67fc5e bundle2-resolver: add context to errors returned from this crate
Summary: This will make it easy to track down an error without looking at stacktraces

Reviewed By: StanislavGlebik

Differential Revision: D7087301

fbshipit-source-id: c5460dae9c5c9ab43713e5db1457f5d9155b5e8e
2018-02-27 11:14:52 -08:00
Lukas Piatkowski
39d67e5da3 integration tests: add more commits to test-push-protocol
Summary: The introduced commits feature things like adding blob that should already be present, adding blob twice and having non-linear history

Reviewed By: farnz

Differential Revision: D7071447

fbshipit-source-id: e26b7792808351e2380d68b9eb3a4d7e6e859b0e
2018-02-23 12:59:11 -08:00
Lukas Piatkowski
efc38342f7 bundle2-resolver: convert FilelogDeltaed to Filelog
Summary: This code handles deltaed filelogs and resolves them into proper Filelogs

Reviewed By: farnz

Differential Revision: D7011702

fbshipit-source-id: e8dc4844657011bc1085463eedd1790b87d317dc
2018-02-22 03:45:25 -08:00
Stanislau Hlebik
11af232471 mononoke: send replychangegroup to the client
Summary:
Return a reply to a client so that it doesn't fail.
Reply consists of just one replychangegroup that tells a client that the push
has succeeded (even though currently it wasn't).

resolve() function returns future of Bytes instead of a stream. It may be
suboptimal, but should be fine for now, and it's already used by a few
wireproto methods.

Reviewed By: lukaspiatkowski

Differential Revision: D7010578

fbshipit-source-id: 9b5425b912c640d4e2bac957a02e9881813b8871
2018-02-21 11:07:19 -08:00
Stanislau Hlebik
ebf46512ef mononoke: test cleanup, part 2
Summary: Cleanup blobimporting

Reviewed By: lukaspiatkowski

Differential Revision: D7000047

fbshipit-source-id: 2657a5608c3d58df71154dd2e97c4c899c33714a
2018-02-16 03:08:40 -08:00
Stanislau Hlebik
387bb1af8c mononoke: fixed test-push-protocol.t test
Summary:
In D6988301 luk noticed that test output is incorrect. Turned out that client
asked server to fetch revision that exists only client-side.

To fix the test let's set treeonly=True and do shallow clone.
Figuring out why exactly it won't work without these flags is very
time-consuming. But that's the usual setup we expect our clients to have.
Besides test-init.t have used these flags and worked fine, so does
test-push-protocol.t

Reviewed By: lukaspiatkowski

Differential Revision: D7000045

fbshipit-source-id: 5e3a6e9fc9c0a1ed0a1e2074f5ac86533d42fa09
2018-02-16 03:08:40 -08:00
Lukas Piatkowski
329afcf975 integration test: fix mistyped line in push protocol test
Reviewed By: farnz

Differential Revision: D6998593

fbshipit-source-id: d65fa1c5ccb1558547d1e9c031e95d29048ab3ca
2018-02-15 07:26:29 -08:00
Stanislau Hlebik
48c5ac631a mononoke: fix tests
Summary:
In D6923197 we've made it so that nullid always exists. This is necessary for
pull, because client can send nullids (for example, if client has an empty
repo). That has broken revset test.

Also gettreepack capability was added, that has broken test-push-protocol.t

Reviewed By: quark-zju

Differential Revision: D6988301

fbshipit-source-id: 2deafc3489de9f50126de7212656ffc075ef3537
2018-02-14 08:37:01 -08:00
Stanislau Hlebik
0ad9a311ff mononoke: getfiles arguments parsing
Summary:
getfiles is a remotefilelog method to fetch file content. Mercurial client
calls it on-demand, usually during `hg update` command. So normal `hg pull`
with remotefilelog doesn't send filelogs back to the client.

This diff implements getfiles arguments parsing. The code is tricky, so I've
added lots of comments to explain what's going on.

Reviewed By: farnz

Differential Revision: D6913178

fbshipit-source-id: 0248993c3ac487956e0ff547996d51b75cdaee96
2018-02-13 08:54:31 -08:00
Lukas Piatkowski
dfd17c9d57 mononoke: create integration test for a basic push case to treemanifest repo
Summary: making this test work is a 2018 m1 milestone

Reviewed By: farnz

Differential Revision: D6950112

fbshipit-source-id: 798a1b8e722e96b27f0cb8267b82232d3fe0496d
2018-02-13 06:22:17 -08:00
Stanislau Hlebik
1d7668ffba mononoke: remove get_mpath and get_path and use get_name
Summary:
As we discussed before, let's add get_name() method that returns MPathElement,
and remove get_path() and get_mpath().

Except for renaming, diff also make repoconfig work with tree manifest, and
fixes linknodes creation in blobimport - previously basename was used instead
of the whole path.

Reviewed By: jsgf

Differential Revision: D6857097

fbshipit-source-id: c09f3ff40d38643bd44aee8b4488277d658cf4f6
2018-02-07 07:53:48 -08:00
Simon Farnsworth
ebafde00b0 Remove Repo trait completely
Summary:
We're never going to serve RevlogRepo in production, and we're down to
a single BlobRepo type that will have different backing stores. Remove the
unused trait, and use BlobRepo everywhere bar blobimport and repo_config
(because we previously hardcoded revlog here - we want to change to a BlobRepo
once blobimport is full-fidelity).

Reviewed By: jsgf

Differential Revision: D6596164

fbshipit-source-id: ba6e76e78c495720792cbe77ae6037f7802ec126
2018-01-15 06:37:27 -08:00
Simon Farnsworth
b38dd81782 Make known protocol check that the commit is fully present
Summary: `changeset_exists` only checks that the changeset is in the blob store. Once we start accepting pushes, we'll also want to check that the changeset is reachable from a head. Use revsets to check this. Once we have a notion of "completeness" for changesets, we'll switch `known` to use that instead - this code is still useful as a way to go from `changeset_exists` to `changeset_complete` in bulk.

Reviewed By: jsgf

Differential Revision: D6205695

fbshipit-source-id: 7e3b5c30bc5e459feb95a20913d8a04f3fda7469
2017-12-04 11:54:53 -08:00