Commit Graph

9 Commits

Author SHA1 Message Date
Stanislau Hlebik
1ca1bc0d81 mononoke: fix buffer size in compression
Summary:
We were using incorrect buffer size. That's *very* surprising that our servers
weren't continuously crashing. However, see the test plan - it really looks
like `LZ4_compressBound()` is the correct option here.

Reviewed By: farnz

Differential Revision: D9738590

fbshipit-source-id: d531f32e79ab900f40d46b7cb6dac01dff8e9cdc
2018-09-10 09:23:50 -07:00
Jun Wu
cd12c8ab3a Back out "Reuse pylz4 encoding between hg and Mononoke into a separate library"
Summary:
Backout D9124508.

This is actually more complex than it seems. It breaks non-buck build
everywhere:

- hgbuild on all platforms. POSIX platforms break because `hg archive` will
  miss `scm/common`. Windows build breaks because of symlink.
- `make local` on GitHub repo because `failure_ext` is not public. The `pylz4`
  Cargo.toml has missing dependencies.

Fixing them correctly seems non-trivial. Therefore let's backout the change to
unblock builds quickly.

The linter change is kept in case we'd like to try again in the future.

Reviewed By: simpkins

Differential Revision: D9225955

fbshipit-source-id: 4170a5f7664ac0f6aa78f3b32f61a09d65e19f63
2018-08-08 15:23:56 -07:00
Tuan Tran
e1d33078e6 Reuse pylz4 encoding between hg and Mononoke into a separate library
Summary: Moved the lz4 compression code into a separate module in `scm/common/pylz4` and redirected code referencing the former two files to the new module

Reviewed By: quark-zju, mitrandir77

Differential Revision: D9124508

fbshipit-source-id: e4796cf36d16c3a8c60314c75f26ee942d2f9e65
2018-08-08 15:23:56 -07:00
Pulkit Goyal
fc880f518b Add Cargo.toml files to crates. (#7)
Summary:
This is a series of patches which adds Cargo.toml files to all the crates and tries to build them. There is individual patch for each crate which tells whether that crate build successfully right now using cargo or not, and if not, reason behind that.

Following are the reasons why the crates don't build:

  * failure_ext and netstring crates which are internal
  * error related to tokio_io, there might be an patched version of tokio_io internally
  * actix-web depends on httparse which uses nightly features

All the build is done using rustc version `rustc 1.27.0-dev`.
Pull Request resolved: https://github.com/facebookexperimental/mononoke/pull/7

Differential Revision: D8778746

Pulled By: jsgf

fbshipit-source-id: 927a7a20b1d5c9643869b26c0eab09e90048443e
2018-07-09 19:52:27 -07:00
Christopher Kuklewicz
6cc2cd49f8 Unifying sql code for Mononoke/Diesel
Summary:
Unify all uses of Sqlite and of Mysql

This superceded D8712926

Reviewed By: farnz

Differential Revision: D8732579

fbshipit-source-id: a02cd04055a915e5f97b540d6d98e2ff2d707875
2018-07-08 09:06:22 -07:00
Stanislau Hlebik
500a138b04 mononoke: add CompressContext
Summary:
We had a memory leak because context wasn't cleaned afterwards. This diff fixes
it

Reviewed By: farnz

Differential Revision: D8236762

fbshipit-source-id: f82b061f3f541d9104d1185ed04ea21224b7d5bc
2018-06-04 04:56:11 -07:00
Stanislau Hlebik
e2efcd5396 mononoke: rename Context to DecompressContext
Summary: We are going to add CompressContext in the next diff

Reviewed By: farnz

Differential Revision: D8236761

fbshipit-source-id: 0df55b9bc5e9fd78ac8c060576513c1216641ead
2018-06-04 04:56:10 -07:00
Stanislau Hlebik
dc20cc2051 mononoke: add compress function
Summary: Will be used in remotefilelog getfiles method.

Reviewed By: jsgf

Differential Revision: D6884919

fbshipit-source-id: e8037123a4843322c29b37c6b5749444781f4fa7
2018-02-06 11:23:57 -08:00
Stanislau Hlebik
8f0916fdf0 mononoke: add pylz4 crate
Summary:
Add a separate crate that uses lz4 in the same way as python lz4 library. The
main difference is that first 4 bytes are length of the raw data in le32
format. The reason for moving it in a separate crate is to use pylz4 for
remotefilelog getfiles method.

Also removed one panic and replaced it with error.

Reviewed By: jsgf

Differential Revision: D6884918

fbshipit-source-id: 1b05381c045a1f138ab28820175289233b07a91d
2018-02-06 11:23:57 -08:00