Commit Graph

34 Commits

Author SHA1 Message Date
Johan Schuijt-Li
3d1463231a disable authorization verification hook for import
Reviewed By: StanislavGlebik

Differential Revision: D13770727

fbshipit-source-id: 8830c63124bc12a40ffe624110d23a3713cad680
2019-01-23 08:00:55 -08:00
Stanislau Hlebik
bacf9f2d64 mononoke: update skiplist building
Summary:
Previously we rely on CachingChangesetFetcher to quickly fetch all commits into
memory, but CachingChangesetFetcher was deleted in D13695201. Instead let's use
`get_many()` method from Changesets trait to quickly fetch many changesets at
once into memory.

Reviewed By: lukaspiatkowski

Differential Revision: D13712783

fbshipit-source-id: 12e8fa148f7989028547ac8d374438e23b44b6d1
2019-01-21 12:58:44 -08:00
Lukas Piatkowski
ceda67a78a blobstore: extract CacheOpsUtil for direct access for blobstores
Summary: refactor CacheOps to prepare for next diff

Reviewed By: aslpavel

Differential Revision: D13487500

fbshipit-source-id: f9023938d7c2c7573a7957bf38ec6b8641441b75
2019-01-15 09:38:26 -08:00
David Budischek
1558f571c1 Add JSON printing of 'mononoke_admin bonsai-fetch', j...
Reviewed By: StanislavGlebik

Differential Revision: D13622701

fbshipit-source-id: b0b61c23966f9485b99ead7196e276efa8e1e267
2019-01-15 04:02:05 -08:00
David Budischek
7fbfb4ad1d rustfmt mononoke_admin
Reviewed By: StanislavGlebik

Differential Revision: D13634949

fbshipit-source-id: ae97ecba84aa835ad141e4895d1861ad537abfdb
2019-01-15 04:02:05 -08:00
David Budischek
d94fe79b7f Prettify printing of mononoke_admin bonsai-fetch
Summary: Instead of dumping the debug output we print the most important information: Changeset id, author, message and file changes.

Reviewed By: StanislavGlebik

Differential Revision: D13621492

fbshipit-source-id: ea0f93f58516cc759d0dc9aac14545b1827ea136
2019-01-15 04:02:05 -08:00
Pavel Aslanov
1eea4fdb09 asynchronous blobrepo constructor
Summary:
- Support for gluster blubstore
- Asynchronous blobrepo constructor

Reviewed By: StanislavGlebik

Differential Revision: D13630033

fbshipit-source-id: 3e7f5a7be4451063dca3df2f5a7f70d30a2def58
2019-01-14 09:31:58 -08:00
Pavel Aslanov
2efacff245 rustfmt effected files
Summary: Format files effected by next commit in a stack

Reviewed By: StanislavGlebik

Differential Revision: D13650639

fbshipit-source-id: d4e37acd2bcd29b291968a529543c202f6944e1a
2019-01-14 09:31:58 -08:00
Lukas Piatkowski
5155134d0d blobrepo: generalize methods creating Blobrepo over any remote blobstore
Reviewed By: aslpavel

Differential Revision: D13464953

fbshipit-source-id: cb732204bea571788e5b45cee718e8ddf496dd0d
2019-01-10 02:57:36 -08:00
Lukas Piatkowski
2f8ddd0cfa mononoke admin: rustfmt main.rs for easier reviewing of next diffs
Reviewed By: HarveyHunt

Differential Revision: D13606992

fbshipit-source-id: 08600da1c390a552f80bf8b7c160daab78d16f9a
2019-01-09 04:27:33 -08:00
Abhay Bothra
b81b8b0005 Add convert subcommand to mononoke admin tool
Summary:
Added new command which looks as:
```
admin-convert
convert between bonsai and hg changeset hashes

USAGE:
    admin convert <HASH> --from <from> --to <to>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

OPTIONS:
    -f, --from <from>    Source hash type [possible values: hg, bonsai]
    -t, --to <to>        Target hash type [possible values: hg, bonsai]

ARGS:
    <HASH>    source hash
```

Reviewed By: StanislavGlebik

Differential Revision: D13494012

fbshipit-source-id: 1e1de7e1e28638b2001f612b9a2c9edda3979dc5
2019-01-07 11:00:48 -08:00
Stanislau Hlebik
92563375c5 mononoke: name variable correctly
Summary: Their names were mixed up

Reviewed By: ikostia

Differential Revision: D13565334

fbshipit-source-id: 1b56adb582900addf2f21ae9baa6dc4ea01c57cb
2019-01-02 10:24:12 -08:00
Harvey Hunt
162dafa9d9 mononoke: admin: Remove ability to build fbpkgs
Summary:
Now that mononoke's config can be built using normal fbpkg tools,
we can remove the ability for mononoke_admin to build configuration.

Reviewed By: StanislavGlebik

Differential Revision: D13399189

fbshipit-source-id: 17aff327773b9c904916f99030a732a99aa34134
2018-12-11 03:56:02 -08:00
Lukas Piatkowski
5d9a151d85 mononoke: pass CoreContext down to blobstore
Reviewed By: jsgf

Differential Revision: D13324220

fbshipit-source-id: af7a2a650338ea66e504ea0acae1d103af10f8e3
2018-12-04 11:40:15 -08:00
Stanislau Hlebik
f07155b402 mononoke: remove mononoke configs, use toml files
Summary:
Config repo proved to be tricky to understand and hard to use. Let's just use
toml files.

Reviewed By: farnz

Differential Revision: D13179926

fbshipit-source-id: 3a44ee08c37284cc4c189c74b5c369ce82651cc6
2018-12-04 03:52:26 -08:00
Lukas Piatkowski
02e79837a4 mononoke: pass CoreContext down to bookmarks
Reviewed By: StanislavGlebik

Differential Revision: D13302943

fbshipit-source-id: 356ec3cd3c47f843a5869edb7079d4cbd0ee33aa
2018-12-04 01:16:32 -08:00
Lukas Piatkowski
14636545aa mononoke: pass CoreContext down to changesets
Reviewed By: StanislavGlebik

Differential Revision: D13277448

fbshipit-source-id: 6e9a8dac77af8ab991005d14f654e315c234fe44
2018-11-30 10:14:22 -08:00
Lukas Piatkowski
08db0a35eb mononoke: pass CoreContext down to bonsai-hg-mapping
Reviewed By: aslpavel

Differential Revision: D13277450

fbshipit-source-id: 97cfbd917b321727bb4d960c91a784787660eb5b
2018-11-30 10:14:22 -08:00
Stanislau Hlebik
af03216ee2 mononoke: admin command to build and read skiplist indexes
Summary:
Let's add a command that builds and reads a skiplist indexes. This indexes will
be used by getbundle wireproto request to decrease the latency and cpu usage.

Note that we are saving only the longest "jump" from the skiplist. This is done
in order to save space.

Reviewed By: jsgf

Differential Revision: D13169018

fbshipit-source-id: 4d654284b0c0d8a579444816781419ba6ad86baa
2018-11-29 08:19:30 -08:00
Lowik Chanussot
395b124f5d Make get_manifest_by_nodeid accept HgManifestId
Summary: Make get_manifest_by_nodeid accept HgManifestId and correct all calls to get_manifest_by_nodeid.

Reviewed By: StanislavGlebik

Differential Revision: D10298425

fbshipit-source-id: 932e2a896657575c8998e5151ae34a96c164e2b2
2018-10-11 06:50:16 -07:00
Tim Fox
debd4a5f95 Integrate hooks into mononoke write path
Summary: Integrate hook manager into the Mononoke pushrebase path

Reviewed By: lukaspiatkowski

Differential Revision: D9896005

fbshipit-source-id: ffd79b539288d95b134af97b776b7fcc3afa0ce7
2018-10-02 07:07:16 -07:00
Pavel Aslanov
e8e5131447 compare hg/mononoke changesets
Summary: This adds `--compare-commits` option for pushrebase replayer. Which also check that commits are close enough to treat them as equal for the purpose of pushrebase.

Reviewed By: StanislavGlebik

Differential Revision: D10084308

fbshipit-source-id: f1fd05173a9a7663125a89dd03b79b2deea40dc4
2018-10-02 03:52:01 -07:00
Anastasiya Zhyrkevich
efb795b14b mononoke: add json flag to mononoke admin tool
Summary: boormarks get <BOOKMARKNAME> --json

Reviewed By: StanislavGlebik

Differential Revision: D9677271

fbshipit-source-id: e57f1ab324dcedfbb18c1f07f01b2cad9db0c1e3
2018-09-06 09:23:20 -07:00
Stanislau Hlebik
0403dba05c mononoke: remove unused options
Reviewed By: lukaspiatkowski

Differential Revision: D9627883

fbshipit-source-id: b235cb272f93178c942ebf662d77ca73c3790a40
2018-09-03 04:06:14 -07:00
Sebastian Lund
2ac12dbe08 mononoke: add bookmark set support to admin tool
Summary:
Add the ability to set bookmarks using the mononoke admin tool.

Usage: `mononoke_admin --repo-id <repo-id> bookmarks set <BOOKMARK_NAME> <HG_CHANGESET_ID>`

Reviewed By: StanislavGlebik

Differential Revision: D9539550

fbshipit-source-id: 7114a6a51711eae6784eb30d820c2ce11672679c
2018-08-30 04:22:20 -07:00
Rain ⁣
2203a158e1 cmdlib: switch from BlobRepo to MononokeRepo
Summary: This is more general, and allows one to call `RepoClient` methods.

Reviewed By: farnz

Differential Revision: D9318658

fbshipit-source-id: 09b2e64bc0d423eafcb381902e03f349fc666a41
2018-08-15 23:36:19 -07:00
Rain ⁣
fb192f8f6f cmds/admin: init_cachelib before bonsai-fetch
Summary: There's a hidden dependency here -- ideally it'd be made explicit.

Reviewed By: jsgf

Differential Revision: D9307939

fbshipit-source-id: e00a201e20bb8d83be09eab10cfdd2ce03a3e22f
2018-08-13 16:52:25 -07:00
Simon Farnsworth
3733848cef Use Abomonation and cachelib to replace Asyncmemo
Summary:
Asyncmemo has two issues for our use:

1. Separate memory pool from cachelib caches.
2. Future fusion means that a `get` that should succeed will fail because there
was an earlier get still in progress.

The second is good for memoization, where the worst case from a failed get is
extra CPU work, but not so good for caching. Replace uses of Asyncmemo for
caches with a cachelib based cache

Reviewed By: StanislavGlebik

Differential Revision: D9013679

fbshipit-source-id: b85d4eec7294e0c8ee08faa671d26901b35cf1fc
2018-08-07 13:37:09 -07:00
Simon Farnsworth
53a9245288 Add mechanism to configure cachelib process size shrinker
Summary:
We'll be running in Tupperware, and want to shrink when we get too
large to avoid OOM due to caches. Configure cachelib appropriately

Reviewed By: StanislavGlebik

Differential Revision: D8900371

fbshipit-source-id: 4f1f64c2508c64e4ce2d201e0a0e86446f84ffef
2018-08-07 11:37:37 -07:00
Stanislau Hlebik
f3e5d4100a mononoke: add bonsai-fetch to admin
Summary:
Since we have bonsai changesets now, let's add a subcommand to print bonsai
changesets given an hg changeset

Reviewed By: farnz

Differential Revision: D9179315

fbshipit-source-id: fad8a7454b45608482ac9fbf0dae9be16d5cbd69
2018-08-07 06:22:01 -07:00
Lucas Waye
2b21e28790 fetch file content by bookmark
Summary: The content-fetch subcommand will try to lookup a changeset by bookmark first and then falls back to a hash.

Reviewed By: StanislavGlebik

Differential Revision: D9110531

fbshipit-source-id: 3bd0e452739fc1c8b41dc559864f73e3a1f3f782
2018-08-02 07:53:18 -07:00
Rain ⁣
6f65d1778f make the default style for most cli tools glog
Summary:
I don't like glog for interactive use at all. But keep it as the
default for blobimport, and add a flag to change it.

Reviewed By: StanislavGlebik

Differential Revision: D8909674

fbshipit-source-id: d0b9c439f72f231c95e9109e16b30e87cfaa2eed
2018-07-20 11:37:36 -07:00
Lukas Piatkowski
f728e8bb3f admin: add config fbpkg subcommand
Summary:
This command will create a PROD bookmark on the currently checked out commit of the choosen mononoke-config repo, then import it into a temp directory and finally invoke fbpkg build to create a package with that imported content.
It also calls fbpkg build inside the mononoke-config repo so that the VCS info from mercurial is persisted in fbpkg metadata.

Reviewed By: farnz

Differential Revision: D8907916

fbshipit-source-id: ee1843bf23c9bffe524d2b20c7322f84825e2663
2018-07-19 04:08:34 -07:00
Lukas Piatkowski
26b0f52cef admin: add config clone subcommand
Summary:
can be used to easily clone mononoke-config and required hgrc setup
later this subcomand will be used to blobimport the mononok-config as well

Reviewed By: StanislavGlebik

Differential Revision: D8879500

fbshipit-source-id: 1ad19441bce73f5a8c56f5aebd04f331ca49519a
2018-07-19 04:08:34 -07:00