Commit Graph

27 Commits

Author SHA1 Message Date
Jeremy Fitzhardinge
d6504de44c scm/mononoke: make Changesets::add take ChangesetInsert by value
Summary:
There's no point passing it by reference since callers don't need to
retain it, and the async implementation needs to move it into another context.

Reviewed By: farnz

Differential Revision: D7350001

fbshipit-source-id: 5947557a84621afae801dc20e3994496244e3a10
2018-03-23 13:47:54 -07:00
Stanislau Hlebik
91d795ec72 mononoke: use diesel bookmarks
Summary: Replace old_bookmarks with the new diesel implementation

Reviewed By: farnz

Differential Revision: D7271599

fbshipit-source-id: cb6433fe44a8e6f87046a3820f7c3e68725505f6
2018-03-21 14:24:51 -07:00
Siddharth Agarwal
b338897dc4 prefix ChangesetId, ManifestId and BlobHash with Hg
Summary:
Mononoke will introduce its own ChangesetId, ManifestId and BlobHash, and it
would be good to rename these before that lands.

Reviewed By: farnz

Differential Revision: D7293334

fbshipit-source-id: 7d9d5ddf1f1f45ad45f04194e4811b0f6decb3b0
2018-03-15 17:45:29 -07:00
Siddharth Agarwal
6204e214d1 create a mononoke-types crate
Summary:
These are types that are going to be used throughout Mononoke -- I'm hoping to
avoid references to current Mercurial data structures in here.

For now, the only module I've moved is part of the `path` module. The
Mercurial-specific `fsencode` bits have been kept in mercurial-types (though
maybe they should move to `mercurial`?)

In the future, this module will also include definitions for unodes, etc.

Reviewed By: jsgf

Differential Revision: D7188722

fbshipit-source-id: fc097ca12c38a787f83e35af9b8dd308f2b910ea
2018-03-08 10:57:35 -08:00
Simon Farnsworth
bb8c39183e Thread a logger into BlobRepo
Summary: I'm going to need a logger to log future-stats output to (and later trace output). Thread one through to BlobRepo

Reviewed By: StanislavGlebik

Differential Revision: D7167450

fbshipit-source-id: 4ed729e4d448b66e491cefa19380d3be9bc99091
2018-03-06 08:09:19 -08:00
Dino Wernli
fa2b14cd8b Remove the generic types from Blob and BlobNode in favor of Bytes
Summary: Replace the generic types if `Blob` and `BlobNode` with `Bytes`.

Reviewed By: lukaspiatkowski

Differential Revision: D7115361

fbshipit-source-id: 924d347377569c6d1b3b4aed14d584510598da7b
2018-03-02 09:45:04 -08:00
Stanislau Hlebik
e8b335468a mononoke: use changesets table in blobimport
Summary:
Iff all the inserts finished successfully, then it's safe to mark changesets as complete.
This diff fills up changesets store after blobimport successfully finishes.

For simplicity if --commit-limit or --skip is set then we skip filling up the changeset store.

Reviewed By: sid0

Differential Revision: D7043831

fbshipit-source-id: 8ae864b45222d52281c885a49c2dca44ba577137
2018-02-26 09:43:24 -08:00
Stanislau Hlebik
9beeaeadc5 mononoke: add repoid to BlobRepo
Summary: Changests store requires it in it's api methods. Let's pass repoid from configs

Reviewed By: farnz

Differential Revision: D7043830

fbshipit-source-id: e4e4d5852d0ca8488cabe2140555508c143ab8df
2018-02-26 09:43:24 -08:00
Stanislau Hlebik
f88f3fbb21 mononoke: use changesets table in Mononoke
Summary:
For now it does nothing. In the next diffs it will be used to tell if commit
exists in the repo or not and to speed up revsets

Reviewed By: farnz

Differential Revision: D7043828

fbshipit-source-id: 9fcc668e68ba238123a89f18ff67828848ba0cec
2018-02-26 09:43:24 -08:00
Lukas Piatkowski
4eab1b3a13 blobrepo: make get_changeset_by_changesetid return BlobChangeset rather than Box<Changeset>
Summary: BlobChangeset has some methods that are useful also outside of blobrepo crate, like getting it's node.

Reviewed By: farnz

Differential Revision: D7056035

fbshipit-source-id: 197d261fd21ab9332950d3fda401e7dab3730cb5
2018-02-23 11:22:58 -08:00
Katherine McKinley
28ce2f6211 Replace NodeHash with ChangesetId in BlobRepo
Summary:
This change removes get_changeset_by_nodeid and replaces it with
get_changeset_by_changesetid, and propagates the changes to callers.

A few places still have ChangesetId::new() because I'm not sure where
the original NodeHash comes from. If you have any pointers, I would be
happy to fix them before landing.

Reviewed By: lukaspiatkowski

Differential Revision: D7031923

fbshipit-source-id: cd00ea1d2b955538e26d7b5735aed33fe0ae0330
2018-02-21 10:42:06 -08:00
Simon Farnsworth
9e366eacd4 Introduce a LazyMemblob test blobstore
Summary:
The current Memblob store is eager; this is great for finding certain
classes of bugs (those that assume an ordering that is not guaranteed), but not
so good for exposing other classes of bugs (those that assume that the future
has done its work before it resolves).

Add a lazy variant that functions in the same way as Memblob, but that waits
until it's polled to return.

Reviewed By: StanislavGlebik

Differential Revision: D7033792

fbshipit-source-id: 4c2d8a8150d908bcb26347757f96f99e20d74fc2
2018-02-21 07:17:00 -08:00
Stanislau Hlebik
5170568488 mononoke: add many_files_dirs test fixture
Summary:
Add a fixture where there are many files and directories.
History is linear for now, but later we may add merge commtis.
It will be used to test gettreepack method.

Reviewed By: lukaspiatkowski

Differential Revision: D6818544

fbshipit-source-id: 798a1bea66eb49491f98fe5e997b172f3c927c53
2018-01-30 01:33:06 -08:00
Xiaotian Wu
92c0528b1f modify manifestid related code
Summary: nodehash -> manifestid

Reviewed By: lukaspiatkowski

Differential Revision: D6719378

fbshipit-source-id: 1ec59b33270e389da8e74b3864c37a86c9d89f81
2018-01-16 07:57:25 -08:00
Xiaotian Wu
ed94c6702c modify changesetid related code
Summary: modify the parameter type

Reviewed By: lukaspiatkowski

Differential Revision: D6695443

fbshipit-source-id: fafcdc83773cb86c08cdcf3a8d80c1c9a498eca5
2018-01-16 07:57:25 -08:00
Simon Farnsworth
ebafde00b0 Remove Repo trait completely
Summary:
We're never going to serve RevlogRepo in production, and we're down to
a single BlobRepo type that will have different backing stores. Remove the
unused trait, and use BlobRepo everywhere bar blobimport and repo_config
(because we previously hardcoded revlog here - we want to change to a BlobRepo
once blobimport is full-fidelity).

Reviewed By: jsgf

Differential Revision: D6596164

fbshipit-source-id: ba6e76e78c495720792cbe77ae6037f7802ec126
2018-01-15 06:37:27 -08:00
Simon Farnsworth
16da012250 Remove ValueIn/ValueOut from the BlobStore generic arguments.
Summary:
BlobStore is entirely generic, and puts no limits on its
implementations. Remove ValueIn and ValueOut type parameters, and insist that
all blobs are Bytes (as per production setups)

Reviewed By: StanislavGlebik

Differential Revision: D6425413

fbshipit-source-id: 455e526d8baebd0d0f1906941648acca89be4881
2017-12-04 10:22:09 -08:00
Simon Farnsworth
5519eae489 Make Bookmarks a parameterless trait
Summary:
As part of removing excess genericism, make Bookmarks a trait with no
associated types or type parameters. I will revisit error types here once I
have removed the Repo trait - these are not ideal, but enough to get things
going

Reviewed By: StanislavGlebik

Differential Revision: D6405315

fbshipit-source-id: 814c106612a061e1766f1ea9a9428a13a73bd007
2017-11-27 08:05:02 -08:00
Simon Farnsworth
16615b3749 Make Heads a parameterless trait
Summary:
As part of removing excess genericism, make Heads a trait with no
associated types or type parameters.

Reviewed By: StanislavGlebik

Differential Revision: D6352727

fbshipit-source-id: df9ef87e0e0abe43c30e7318da38d7f930c37c6e
2017-11-23 07:05:35 -08:00
Siddharth Agarwal
e771348b76 blobrepo: incorporate linknodes in blob state
Summary: Also support them in `generate_memblob_repo.py`.

Reviewed By: jsgf

Differential Revision: D6215721

fbshipit-source-id: c8b855bbc74b619bc093b3aac449a283a1ad33ec
2017-11-13 22:01:55 -08:00
Siddharth Agarwal
a6c5093cc8 blobimport: write out linknodes
Summary:
This makes it quite easy to write out linknodes.

Also regenerate linknodes for our test fixtures -- the next commit will bring
them in.

Reviewed By: jsgf

Differential Revision: D6214033

fbshipit-source-id: 3b930fe9eda45a1b7bc6f0b3f81dd8af102061fc
2017-11-13 22:01:55 -08:00
Simon Farnsworth
573f37bc83 Add several more test repositories of different shapes
Summary:
We want a few more test fixtures that cover different "shapes" of repo
- branching, merging etc - and that test things we don't necessarily expect to
  see in developer use. Add them to the repo as extra fixtures.

The repos have the following shapes (from `hg sl --all` output):
```
@  4f7f3f  simonfar
|  Replace the base
|
o  b65231  simonfar
|  Doubled
|
o  d7542c  simonfar
|  Branch 2
|
| o  168390  simonfar
| |  I think 4 is a nice number
| |
| o  1d8a90  simonfar
| |  Add one
| |
| o  3cda5c  simonfar
|/   Branch 1
|
o  15c40d  simonfar
   base
```

```
@  264f01  simonfar
|  Add 5
|
o  5d4388  simonfar
|  Add 4
|
o  fc2cef  simonfar
|  Add 3
|
o  bc7b4d  simonfar
|  Add 2
|
o  795b81  simonfar
|  Add 1
|
o  4f7f3f  simonfar
|  Replace the base
|
o  b65231  simonfar
|  Doubled
|
o  d7542c  simonfar
|  Branch 2
|
| o  168390  simonfar
| |  I think 4 is a nice number
| |
| o  1d8a90  simonfar
| |  Add one
| |
| o  3cda5c  simonfar
|/   Branch 1
|
o  15c40d  simonfar
   base
```

```
@  49f53a  simonfar
|  Three.four
|
| o  04decb  simonfar
|/   Three.three
|
o  4685e9  simonfar
|  Two.two
|
| o  c27ef5  simonfar
| |  Three.two
| |
| | o  b6a816  simonfar
| |/   Three.one
| |
| o  9e8521  simonfar
|/   Two.one
|
o  ecba69  simonfar
   One
```

```
@    babf5e  simonfar
|\   Merge
| |
| o  4f7f3f  simonfar
| |  Replace the base
| |
| o  b65231  simonfar
| |  Doubled
| |
| o  d7542c  simonfar
| |  Branch 2
| |
o |  168390  simonfar
| |  I think 4 is a nice number
| |
o |  1d8a90  simonfar
| |  Add one
| |
o |  3cda5c  simonfar
|/   Branch 1
|
o  15c40d  simonfar
   base
```

```
@    75742e  simonfar
|\   Merge two branches
| |
| o  264f01  simonfar
| |  Add 5
| |
| o  5d4388  simonfar
| |  Add 4
| |
| o  fc2cef  simonfar
| |  Add 3
| |
| o  bc7b4d  simonfar
| |  Add 2
| |
| o  795b81  simonfar
| |  Add 1
| |
| o  4f7f3f  simonfar
| |  Replace the base
| |
| o  b65231  simonfar
| |  Doubled
| |
| o  d7542c  simonfar
| |  Branch 2
| |
o |  168390  simonfar
| |  I think 4 is a nice number
| |
o |  1d8a90  simonfar
| |  Add one
| |
o |  3cda5c  simonfar
|/   Branch 1
|
o  15c40d  simonfar
   base
```

```
@  cc7f14  simonfar
|  And work
|
o    d59249  simonfar
|\   Merge
| |
o |  03b058  simonfar
| |  Add 5
| |
o |  2fa8b4  simonfar
| |  Add 4
| |
o |  0b94a2  simonfar
| |  Add 3
| |
o |  f61fdc  simonfar
| |  Add 2
| |
o |  36ff88  simonfar
| |  Add 1
| |
o |  170052  simonfar
 /   Two
|
o  33fb49  simonfar
|  Add 5
|
o  f01e18  simonfar
|  Add 4
|
o  163adc  simonfar
|  Add 3
|
o  eee492  simonfar
|  Add 2
|
o  3775a8  simonfar
|  Add 1
|
o  9d374b  simonfar
   One
```

```
@  ec27ab  simonfar
|  And remove
|
o    9c6dd4  simonfar
|\   Merge
| |
o |  03b058  simonfar
| |  Add 5
| |
o |  2fa8b4  simonfar
| |  Add 4
| |
o |  0b94a2  simonfar
| |  Add 3
| |
o |  f61fdc  simonfar
| |  Add 2
| |
o |  36ff88  simonfar
| |  Add 1
| |
o |  170052  simonfar
 /   Two
|
o  64011f  simonfar
|  Add 10
|
o  c1d537  simonfar
|  Add 9
|
o  e819f2  simonfar
|  Add 8
|
o  5a3e8d  simonfar
|  Add 7
|
o  76096a  simonfar
|  Add 6
|
o  33fb49  simonfar
|  Add 5
|
o  f01e18  simonfar
|  Add 4
|
o  163adc  simonfar
|  Add 3
|
o  eee492  simonfar
|  Add 2
|
o  3775a8  simonfar
|  Add 1
|
o  9d374b  simonfar
   One
```

Reviewed By: StanislavGlebik

Differential Revision: D5924752

fbshipit-source-id: ba18df6963c5209e3cfa888862ac22c52d6cebf0
2017-09-28 04:06:24 -07:00
Siddharth Agarwal
a79b8267ae rename Path to MPath
Summary:
`Path` has the potential to be confused with `std::path::Path`.
`MPath` is nice, concise, and clearly different from `Path`.

Reviewed By: jsgf

Differential Revision: D5895665

fbshipit-source-id: dc5ed5c3866b227d753c6d904d3c6d213c882cd7
2017-09-22 17:27:03 -07:00
Siddharth Agarwal
da22546d74 move state into a trait
Summary:
We're going to add more stores like obsmarkers and linknodes very
soon. Stuffing all of them into type parameters is going to get ugly very soon.

Instead, add a trait which represents all the state, and make `BlobRepo`
generic on that trait. Add a few implementations for common use cases like
files and RocksDB-based stores.

This allows the dependency lists for the Mononoke and Eden servers to be more
straightforward.

Reviewed By: jsgf

Differential Revision: D5850372

fbshipit-source-id: 37a0687611687b9616ebbddce70f53e1e5d3267b
2017-09-18 00:35:52 -07:00
Simon Farnsworth
3d267f64f6 mononoke: compile test fixtures into in-memory repos
Summary:
We want to be able to test against repo fixtures. Turn repo fixtures
that have been created by blobimport into in-memory BlobRepos for testing.

Reviewed By: jsgf

Differential Revision: D5834109

fbshipit-source-id: d6fc01f19986b5a6951f25c7c8bda8aa7c9dee56
2017-09-16 02:50:00 -07:00
Siddharth Agarwal
2511835a09 add a small script to regenerate test fixtures
Summary: Makes a formerly manual process automatic.

Reviewed By: jsgf

Differential Revision: D5832456

fbshipit-source-id: 0bfa01aaf08f0794a3b98ab7290d163a65650a7e
2017-09-14 13:52:23 -07:00
Jeremy Fitzhardinge
43d88961a9 mononoke: linear history blobrepo fixture
Summary: Simple linear history with 10 changesets as a file blob repo.

Reviewed By: sid0

Differential Revision: D5731223

fbshipit-source-id: bc479304d13f7a2299d3ecc382e052ad5600c46c
2017-09-11 13:08:36 -07:00