Commit Graph

18 Commits

Author SHA1 Message Date
Andrey Chursin
eac8385eff vfs: use &[u8] instead of Bytes for content
Summary: This is most universal/flexible interface

Reviewed By: quark-zju

Differential Revision: D26410641

fbshipit-source-id: cae74675f89c0c1c05c84331c3ce0e78fd0b0b2f
2021-02-17 10:45:46 -08:00
Andrey Chursin
0ea63a1eaf checkout: test no empty dirs are left
Reviewed By: quark-zju

Differential Revision: D26406420

fbshipit-source-id: 3a9dc818a4a140f2e532c18b2abd81286cfee1eb
2021-02-17 10:45:46 -08:00
Lukas Piatkowski
87ddbe2f74 autocargo v1: update autocargo field format to allow transition to autocargo v2
Summary:
Autocargo V2 will use a more structured format for autocargo field
with the help of `cargo_toml` crate it will be easy to deserialize and handle
it.

Also the "include" field is apparently obsolete as it is used for cargo-publish (see https://doc.rust-lang.org/cargo/reference/manifest.html#the-exclude-and-include-fields). From what I know this might be often wrong, especially if someone tries to publish a package from fbcode, then the private facebook folders might be shipped. Lets just not set it and in the new system one will be able to set it explicitly via autocargo parameter on a rule.

Reviewed By: ahornby

Differential Revision: D26339606

fbshipit-source-id: 510a01a4dd80b3efe58a14553b752009d516d651
2021-02-12 23:28:25 -08:00
Andrey Chursin
0b03898f89 checkout: add genereated test cases
Summary: This diff adds auto-generated test cases to checkout code. It generates partially overlapping trees and tests transitions between them

Reviewed By: quark-zju

Differential Revision: D26384962

fbshipit-source-id: 6140bbb7ff8b87843a2235f8325f57829cdd8cae
2021-02-11 19:10:25 -08:00
Andrey Chursin
0240026f36 vfs: do not follow symlink when overwriting
Summary:
Before this diff VFS::write_regular did not handle correctly use case when file already existed as as symlink - it would write into symlink location, instead of replacing symlink with a regular file (see updated  test_symlink_overwrite that is failing on old implementation)

This diff adds O_NOFOLLOW option on unix when overwriting the file. When destination is a symlink, attempt to write fails with E_LOOP and triggers clear_conflict that removes symlink and allows retry write to succeed.

This also allows one of test cases in checkout that previously did not work

Reviewed By: quark-zju

Differential Revision: D26378893

fbshipit-source-id: 28bcdaba78db283ac7a25bb232c198d3d8f73e5d
2021-02-11 19:10:24 -08:00
Andrey Chursin
38499b36e0 checkout: introduce file system tests
Summary: This diff contains basic test setup for checkout tests - we compare transition between two trees without dirty changes

Reviewed By: quark-zju

Differential Revision: D26359502

fbshipit-source-id: ef670c944200bae1652863c91ada92c6fecce4ac
2021-02-11 19:10:24 -08:00
Andrey Chursin
fc37800267 checkout: generalize CheckoutPlan::apply for testing
Summary: Allow different type of storages for CheckoutPlan::apply

Reviewed By: quark-zju

Differential Revision: D26291577

fbshipit-source-id: a1c9ba4dbef09e844727ae32eac25c37dd01358e
2021-02-09 17:04:30 -08:00
Andrey Chursin
713af03e97 checkout: track checkout stats
Reviewed By: quark-zju

Differential Revision: D26262590

fbshipit-source-id: fcf96070d167390b60c31392720344fd66732e77
2021-02-09 17:04:30 -08:00
Andrey Chursin
14064f8582 vfs: move conflict handling from pyworker to vfs write
Summary:
Previously, `write` can fail because the destination file exists as a
directory, or the parent directory is missing. pyworker handles those cases
by calling `clear_conflicts` to remove conflicted directories and create
missing parent directories and retry `write`. Practically, for all `write`
usecases (including checkout) we always want the `clear_conflicts` behavior.
Therefore, move `clear_conflicts` to vfs `write` and make it private.

Reviewed By: quark-zju

Differential Revision: D26257829

fbshipit-source-id: 03d1da0767202edba61c47ae5654847c0ea3b33e
2021-02-09 17:04:30 -08:00
Andrey Chursin
d9232f1db3 checkout: handle more complex flag changes
Summary: Currently we pick simple approach and just delete existing file before re-creating it if there is some non-trivial flag change

Reviewed By: quark-zju

Differential Revision: D26239977

fbshipit-source-id: 167efa1bf6018e7f967ef3a9e3c8c62781486ec9
2021-02-09 17:04:29 -08:00
Andrey Chursin
468505dc5a vfs: make audit cache concurrent
Reviewed By: quark-zju

Differential Revision: D26233458

fbshipit-source-id: e06f5642f713d0788407e3cc70655ef82261e7c2
2021-02-04 13:13:45 -08:00
Andrey Chursin
c890d2a322 checkout: update exec flag
Summary: Metadata update happens in parallel with blob fetch and content update

Reviewed By: quark-zju

Differential Revision: D26213197

fbshipit-source-id: 45ca20106123e1b6cb85d1dda2f3effa0748cf20
2021-02-02 23:14:34 -08:00
Andrey Chursin
199f1cc1b9 checkout: basic implementation for file removal
Reviewed By: quark-zju

Differential Revision: D26212172

fbshipit-source-id: 2d3d8c79023b1dfd069c9cc7b759362e3bfc24ae
2021-02-02 23:14:34 -08:00
Andrey Chursin
45e4a30651 checkout: basic CheckoutPlan::apply implementation
Summary:
Basic implementation only process file updates for now.
See todos on CheckoutPlan::apply for more detail

Reviewed By: quark-zju

Differential Revision: D26209984

fbshipit-source-id: fcfbf568359d553a51ea02ea194634048d093d0e
2021-02-02 23:14:34 -08:00
Andrey Chursin
3992818136 checkout: split updates with and without content fetch
Summary: This is needed becase later during checkout list of updates that needs fetch will go into storage stream

Reviewed By: quark-zju

Differential Revision: D26209983

fbshipit-source-id: 9fb54d48c6f0afc4fb67320aafc2e981c96ab5a9
2021-02-02 23:14:33 -08:00
Andrey Chursin
6be082f3bd checkout: implement CheckoutPlan::from_diff
Reviewed By: quark-zju

Differential Revision: D26191428

fbshipit-source-id: d26dd58a1bd9a3bab27f38abbd15f0405c646294
2021-02-02 15:34:33 -08:00
Andrey Chursin
a0fd97854b checkout: data structure for CheckoutPlan
Reviewed By: quark-zju

Differential Revision: D26189792

fbshipit-source-id: 9afa1682055d088e5c6e4c166f73d6e0b836806a
2021-02-02 15:34:32 -08:00
Andrey Chursin
e921b0b5f0 checkout: create empty checkout crate
Reviewed By: quark-zju

Differential Revision: D26185951

fbshipit-source-id: 9b483c309f2b6ceed42b9eb1d1670bfe7b80be7b
2021-02-01 21:56:14 -08:00