Summary: just to excersise our RocksDb blobstore which might come in handy more often than file based as it should be more efficient
Reviewed By: StanislavGlebik
Differential Revision: D7290069
fbshipit-source-id: ce776cfa14e43dc45cca796ef187655ba665d177
Summary:
It was flaky because server may not yet be ready to accept connections when we
send first request. Sometimes delay needs to be > 1 sec.
Let's query eden server in a loop until it responds.
Reviewed By: farnz
Differential Revision: D7233069
fbshipit-source-id: 8bcb5b2b8ebdc52d2447b33e18580e50c1e27031
Summary:
Changesets table gives us a relatively fast way to get generation number. Let's
use it inside repoinfo crate.
Reviewed By: sid0
Differential Revision: D7043829
fbshipit-source-id: 82557340a7f2fd6361f4e29c1a11e38e473eeb15
Summary: Changests store requires it in it's api methods. Let's pass repoid from configs
Reviewed By: farnz
Differential Revision: D7043830
fbshipit-source-id: e4e4d5852d0ca8488cabe2140555508c143ab8df
Summary:
Use hgclone_treemanifest instead of hgcloneshallow.
Note that it changes the behaviour of test-infinitepush.t - now it sends
treegroup part too. Since we actually want to send treegroup part during
infinitepush push, that means that the test was broken and now it fixed.
Reviewed By: lukaspiatkowski
Differential Revision: D7000046
fbshipit-source-id: 61b4517f23081faab1738f44d27220ae8063e077
Summary: making this test work is a 2018 m1 milestone
Reviewed By: farnz
Differential Revision: D6950112
fbshipit-source-id: 798a1b8e722e96b27f0cb8267b82232d3fe0496d
Summary: It removes a bit of copy-pastes code
Reviewed By: sid0
Differential Revision: D6627250
fbshipit-source-id: b110dcad998a240b6da60bc3347c7c5c0370aae0
Summary:
It won't be used in prod anyway, and for local use rocksdb blob state is still
preferable
Reviewed By: sid0
Differential Revision: D6627197
fbshipit-source-id: d9ca55b0221c050e8e8e35914aff22906198874a
Summary:
Fetching file sizes is very expensive - it's one more round-trip to the
blobstore.
Add method that returns tree content, but doesn't fetch blob sizes.
Reviewed By: jsgf
Differential Revision: D6545108
fbshipit-source-id: 3d335eafa9dc367fb9e607f5c147b7e5c37cf133
Summary:
Two tests can't be run simultaneously because they use the same port. This diff
fixes it.
Reviewed By: farnz
Differential Revision: D6457855
fbshipit-source-id: 3e5f82211bcd87abc608bdeab239fe680976316d
Summary:
It maybe not installed on the system. Let's use simple python script instead.
Note: unfortunately we can't use 'python -m json.tool' because we need to sort
entries in the list
Reviewed By: jsgf
Differential Revision: D6437136
fbshipit-source-id: 0703f45c46d220046b9ef484bdf406a7f936557b
Summary:
Config file needs to have ca file in PEM format.
It is used to authenticate client.
See code comments for more details.
Reviewed By: jsgf
Differential Revision: D6323439
fbshipit-source-id: 6b6f1fd68605b263dcb33b051843e10d3f5cb38e
Summary:
Avoid using plain HTTP and use HTTPS instead.
To do this config needs to provide paths to server certificate and private key files in PEM format.
Then they will be converted to Pkcs12 archive.
This diff adds authentication of server i.e. client can check that it talks to a real server. Next diff adds authentication of a client.
Lower-level `hyper::server::Http::bind_connection()` is used instead of `hyper::server::Http::bind()` method in order to add TLS support.
See code comments for more details.
Implementation is more complicated than I expected it to be. I need to use 3 more new crates. Lmk if there is a better way to do this.
Reviewed By: jsgf
Differential Revision: D6323440
fbshipit-source-id: 544f27e6ec210ddf840212b0c0c94145980e8be3
Summary:
We have too many parameters, and cmd-line args are no longer good enough. Let's
use toml based config, with a format that's similar to mononoke server
repoconfig.
Reviewed By: farnz
Differential Revision: D6313512
fbshipit-source-id: 61379e93707a4fcbb42aa8a0392ce526dab19f2e
Summary: Also support them in `generate_memblob_repo.py`.
Reviewed By: jsgf
Differential Revision: D6215721
fbshipit-source-id: c8b855bbc74b619bc093b3aac449a283a1ad33ec
Summary: Let's not be too restrictive about urls
Reviewed By: markbt
Differential Revision: D6099460
fbshipit-source-id: 228a5b3a3cc0f09d68307a3695952543a2f5f05a
Summary: the glog drain does not swallow f.e. backtrace of error_chain errors, so it is a bit easier to debug the tool
Reviewed By: farnz
Differential Revision: D6021671
fbshipit-source-id: 32bfe01bfd77d85c37a2a446cb3e5d000763c689
Summary: Make sure we use treemanifest in the tests, and add commit with a directory
Reviewed By: farnz
Differential Revision: D5974524
fbshipit-source-id: 5ef9f110243001e1916509f53741cccb42dc8deb
Summary:
In Eden server we want to return list of file sizes for each file in the directory.
For now we are doing a separate call to the blobrepo. Since there can be lots of such calls, let's make them in parallel using cpupool
Reviewed By: lukaspiatkowski
Differential Revision: D5950452
fbshipit-source-id: 6b852a49c201ccc14ceb8f950dd61fa5cdebad0a
Summary:
Mercurial filelog entries may have metadata fields in the beginning, usually used to track copies/renames. Previously mononoke eden server returned this metadata as part of the file blob.
This diff changes it. Now `get_content()` method returns file content without metadata, and to make it consistent, both `get_content()` of the blobrepo and revlog repo do the same.
This decision certainly has it's tradeoffs, because now it's more difficult to get metadata (`get_raw_content` needs to be used).
But we'll probably change how metadata is stored anyway, that's why I think changing `get_content` method is fine.
This diff also cleans up server/src/main.rs file, because previously it had to strip metadata itself.
Also diff fixes problem in metadata parsing - it previously failed if file is less than 2 bytes
Reviewed By: farnz
Differential Revision: D5901476
fbshipit-source-id: f3ade0179710352590068c238e6a733aab68a512
Summary: Compaction can slow down blobimporting a lot. Let's add an option to postpone it till the end
Reviewed By: farnz
Differential Revision: D5882003
fbshipit-source-id: 0611a8e94b3d7331bdacf909d820526f547414a0
Summary: I'm going to move this to a common location very soon.
Reviewed By: jsgf
Differential Revision: D5850319
fbshipit-source-id: 09937659330de93cf74bf827f225418dd413865d
Summary: Method that returns content of the manifest entry in json format.
Reviewed By: sid0
Differential Revision: D5527659
fbshipit-source-id: 1832b645f69da40cbd620a6bff318e25594c5148
Summary:
Soon I'm going to add more different url handlers. And current url handling is
neither robust nor extendable. Let's use regexes to do url handling
Reviewed By: jsgf
Differential Revision: D5489445
fbshipit-source-id: f418c3f40892903b573a33e57899b47e5719a5ea
