Commit Graph

48 Commits

Author SHA1 Message Date
Jun Wu
616306543b codemod: use explicit versions in Cargo.toml
Summary:
This is done by running `fix-code.py`. Note that those strings are
semvers so they do not pin down the exact version. An API-compatiable upgrade
is still possible.

Reviewed By: ikostia

Differential Revision: D10213073

fbshipit-source-id: 82f90766fb7e02cdeb6615ae3cb7212d928ed48d
2018-11-15 18:54:06 -08:00
Mark Thomas
93a98afbe4 vlqencoding: don't require Sized for Read or Write traits
Summary:
The `VLQEncode` and `VLQDecode` traits erroneously expected the (automatic)
`Sized` marker trait for `Read` and `Write`.  This meant they couldn't be used
for trait object `Read`s or `Write`s without jumping through hoops or extra
`mut` keywords.

By not requiring `Sized` we can remove those workarounds.

Reviewed By: quark-zju

Differential Revision: D12816459

fbshipit-source-id: 16353e8fefff5738bd24a9f41c9d7d250aea56fd
2018-10-29 04:10:46 -07:00
Jun Wu
3adc813687 codemod: add copyright headers
Summary: This is just the result of running `./contrib/fix-code.py $(hg files .)`

Reviewed By: ikostia

Differential Revision: D10213075

fbshipit-source-id: 88577c9b9588a5b44fcf1fe6f0082815dfeb363a
2018-10-26 15:09:12 -07:00
Jun Wu
dafc189588 treestate: fix documentation about FilteredKeyCache
Summary: It's just a documentation fix.

Reviewed By: singhsrb

Differential Revision: D9110152

fbshipit-source-id: ce4065b7aad6fac05f4c27ef7d2569352cdd2633
2018-07-31 16:35:50 -07:00
Jun Wu
03a0270913 treestate: change getfiltered API to return all matched entries
Summary:
Case-folding could be more complex than what Mercurial currently handles.
Suppose the following paths are committed to a repo using a case-sensitive
filesystem:

  a/a/A
  a/A/a
  A/a/a

Then querying "a/a/a" with a "normpath" filter should ideally have access to
all the above paths.

Unfortunately, the API is changed to use copy instead of references, as it's
impossible to return multiple values borrowed from `&mut self`.

Changes are made on treestate Python land as well to use the new API.  This
solves issues about case-folding corner cases covered by test-eol.t and
test-casefolding.t.

Reviewed By: DurhamG

Differential Revision: D9092405

fbshipit-source-id: 49eb4511ff3c9e5400a522b37126e112c917d2d7
2018-07-31 13:49:35 -07:00
Jun Wu
e17f635422 treestate: fix perf regression on treedirstate
Summary: `calculate_aggregated_state_recursive` should be a no-op with treedirstate.

Reviewed By: DurhamG

Differential Revision: D8505551

fbshipit-source-id: 08b081944cccc0abc4f41ac2e75c8c4305bc9772
2018-06-19 00:48:56 -07:00
Jun Wu
d7c4d3a249 treestate: optimize calculate_aggregated_state_recursive
Summary:
Follow-up of the previous diff. Change the file format so aggregated_state
could be loaded without loading all entries. This would make
`calculate_aggregated_state_recursive` (and `write_delta`) more efficient
in case the node is not modified.

Reviewed By: markbt

Differential Revision: D7909169

fbshipit-source-id: d70b662c7d8c544edf81fbc7da94da9ccbee6cf0
2018-06-11 14:32:42 -07:00
Jun Wu
4819f4203d treestate: calculate aggregated_state recursively during write_delta
Summary:
This avoids a possible Rust panic during `write_delta`, because entries
could have `id` set without `aggregated_state`, by `Node::open`. This diff
fixes that by calling `calculate_aggregated_state_recursive`. The function
has to be changed to static dispatch, since dynamic dispatch only supports
one trait. Practically, this would load one-level content unnecessarily,
which might be optimized by separating loading entries vs loading aggregated
state.

Reviewed By: markbt

Differential Revision: D7909168

fbshipit-source-id: 5effe9df59ce42829a077cab89525103e211bddf
2018-06-11 14:32:42 -07:00
Jun Wu
1f83a4dc00 treestate: require visitor to provide whether it modifies a file or not
Summary:
This is subtle. If visitor changes file state, `Node.id` should be set to
`None` to mark it as "changed".

In practise, treedirstate uses visitor to rewrite mtime to -1 if mtime is
"fsnow". Those rewritten mtime all belong to "changed" nodes (because "fsnow"
can only increase, and on-disk entries cannot have "mtime == fsnow" because
they would be written to -1 during the previous write), so it's not a problem
yet.

It is safer to not depend on the fact that "visitor" can only change "changed"
nodes. On the other hand, detecting changes for all filestate fields could be
undesirably expensive. So let's make the visitor provide the "changed or not"
information. Surely the visitor knows what it does.

Reviewed By: markbt

Differential Revision: D7909167

fbshipit-source-id: 21e71302cf1db86c1330b294baddd51cc8a96026
2018-06-11 14:32:42 -07:00
Jun Wu
98db645ecd treestate: drop Tree.get_mut API
Summary: It's not used, therefore removed.

Reviewed By: markbt

Differential Revision: D7909171

fbshipit-source-id: 587a1d844ece4f2cb0c2ccd9b2d978aed69a959f
2018-06-11 14:32:42 -07:00
Jun Wu
c5b267584b treestate: migrate to rand 0.5 to fix cargo test without breaking buck
Summary:
The pull request [1] is still open, which means `quickcheck::rand` is still
private when building with `cargo`. It only works with a patched quickcheck.
We cannot revert D8234503 since that will break buck build. So there is no
choice but upgrade to rand 0.5.

[1]: https://github.com/BurntSushi/quickcheck/pull/204

Reviewed By: DurhamG

Differential Revision: D8297404

fbshipit-source-id: 19937c49ae96a39e326b1b54eb00e6e2944193c2
2018-06-06 12:54:37 -07:00
Jeremy Fitzhardinge
98be816aba rust/tp2: update rust-crates-io
Summary:
Big change here is update to rand 0.5. This is a significant API
change. quickcheck still uses rand 0.4, so for quickcheck users I changed it so
that quickcheck re-exports the rand it uses. This means that quickcheck users
are unchanged aside from using quickcheck::rand, whereas direct rand users have
been updated to use the new API.

Reviewed By: farnz

Differential Revision: D8234503

fbshipit-source-id: f9e620851b8dfcc33f22a0af26122adcd5fbde39
2018-06-01 09:32:56 -07:00
Jun Wu
ce8e166ebe treestate: add API to get directory's aggregated states
Summary:
Add an internal `get_dir` API to return aggregated states. It is exposed via
`.get('dir/')` python interface.

This is useful for implementing `hastrackeddir` of the dirstatemap class.

Reviewed By: markbt

Differential Revision: D7909173

fbshipit-source-id: 100a8f36237a6b911a4bfb4afbb4c63b98611317
2018-05-26 14:05:18 -07:00
Jun Wu
bcbd121255 treestate: split Node.write_ext into two methods
Summary:
`Node.write_ext` currently contains the logic to calculate
`aggregated_state` and write it to disk. A future diff would need the
calculation without the writing part. So let's split the method.

Reviewed By: markbt

Differential Revision: D7909177

fbshipit-source-id: e83d622e6c1eb512c6a0c3ea8c7201055aa67a21
2018-05-26 14:05:18 -07:00
Jun Wu
46ab269f99 treestate: forbid addfile with path ending with slash
Summary:
Inserting file with names like `a/b/` should be forbidden to avoid errors
later.

Reviewed By: markbt

Differential Revision: D7886809

fbshipit-source-id: c78d357542af4fdc1cea70ad5751b356d3cb308d
2018-05-26 10:50:26 -07:00
Jun Wu
ebe5ab388f treestate: protect split_key from crashing on empty key
Summary:
Not sure how this actually happens. But it happened in a test instance.
Re-run the test locally on the same commit does not reproduce it though.

Reviewed By: phillco

Differential Revision: D8169935

fbshipit-source-id: 7e71611915722d68a1bc633819b94836f63fbc3f
2018-05-26 10:50:24 -07:00
Jun Wu
7b9867ac12 crates: pin rand to 0.4 version
Summary:
`rand` 0.5 has too many breaking changes that the code is not ready to
migrate yet. So let's ping rand to 0.4. Ideally all dependencies in
Cargo.toml should avoid using "*". But for now `rand` is the only
troublemaker.

Note `rand 0.4` is a dependency of `quickcheck 0.6.2` so it's available.

Reviewed By: phillco, singhsrb

Differential Revision: D8158406

fbshipit-source-id: 417ae6807a2efc650acb8d82370964fab6531fdb
2018-05-25 09:51:19 -07:00
Jun Wu
af62797cf3 treestate: improve aggregated_state correctness and usability
Summary:
Previously, `aggregated_state` is only a union of bit flags. It makes
querying files with rare bits fast. But it cannot help with common bits.

This diff adds an `intersection` field, so both "having rare bits", and
"not having common bits" can be queried efficiently.

As we're here, use an explicit `None` to represent "need re-calculation",
instead of using `is_empty()`. This makes the code easier to reason about.
It also solves an issue in `add` that is caught by the next test.

Reviewed By: markbt

Differential Revision: D7886281

fbshipit-source-id: 4ce395883ea26ea9b33794e03c792ea157dc21d0
2018-05-23 06:12:46 -07:00
Jun Wu
acc362ae38 treestate: rename watchman_clock to metadata
Summary:
The field is a string that contains information `TreeState` itself does not
care about. It's up to the upper layer to decide how to use it, and it does
not have to be watchman clock, and might contain other fields like hostname,
ignore matcher hash, etc. So let's rename it to clarify.

Differential Revision: D7886282

fbshipit-source-id: 739a85d7a710918e0b18a9b7fe0e31b366bab447
2018-05-23 06:12:45 -07:00
Jun Wu
8d965905d1 treestate: add TreeState.path_complete and get_filtered_key
Summary: Similar to `TreeDirstate`, those methods are required.

Reviewed By: markbt

Differential Revision: D7874126

fbshipit-source-id: 6dbd6c47c7ba2ded7ea7389dfa9de4cf43db8a01
2018-05-23 06:12:45 -07:00
Jun Wu
b27143828b treestate: change Key from Vec<u8> to Box<[u8]>
Summary: This saves one `usize` per `Key`.

Differential Revision: D7861766

fbshipit-source-id: e44d6b98758966edd0f9823f2f50270ba5481b22
2018-05-23 06:12:45 -07:00
Jun Wu
395edeaea6 treestate: remove clear_filtered_keys
Summary:
The method looks like a foot-gun, and it's O(all entries) instead of
O(cached entries). Change `get_tracked_filtered_key` to take an identity of
the filter function explicitly to solve the problem.

Reviewed By: markbt

Differential Revision: D7861765

fbshipit-source-id: a57ca4a7597120a5b00c63f3f373a62e19e5a834
2018-05-23 06:12:45 -07:00
Jun Wu
e8a83ab74e treestate: use functions to filter out what to visit
Summary:
Follow up of the StateFlags change. Previously the code uses simple bitwise
operations to decide what to visit. That can not express complex conditions.
Let's use functions instead. Rustc should know how to inline those
functions.

Reviewed By: markbt

Differential Revision: D7860276

fbshipit-source-id: 71bf381e00adbb3259a1ae61dbd68fa67f02efdb
2018-05-23 06:12:45 -07:00
Jun Wu
ff2e0ccde1 treestate: add TreeState.visit
Summary: The visit function allows visiting files, filtered by StateFlags.

Reviewed By: markbt

Differential Revision: D7824117

fbshipit-source-id: 7625396cd942ca87056f322a7342979570853d37
2018-05-23 06:12:45 -07:00
Jun Wu
8ca928ced2 treestate: update aggregated_state inside Node.visit
Summary:
Node.visit might change internal file states. So aggregated_state might need
update.

Reviewed By: markbt

Differential Revision: D7824118

fbshipit-source-id: 4f935f427de4d1803524f5908466917e6163dd90
2018-05-23 06:12:45 -07:00
Jun Wu
ddc0a1bcfe treestate: use interior mutability for aggregated_state
Summary:
The upcoming diffs need to do something like:

  self.entries.as_mut().iter_mut() {
    self.aggregated_state = ...
  }

That cannot be done if accessing entries and aggregated_state both need
`&mut` borrow of `self`. So let's use interior mutability.

Reviewed By: markbt

Differential Revision: D7824116

fbshipit-source-id: 67dd317ebfbfc698e79ed6c0e96e69d3fe495a26
2018-05-23 06:12:45 -07:00
Jun Wu
225a0771b4 treestate: revise StateFlags bits
Summary:
The "added", "removed", "normal", "? (untracked)" 4 states could be simplified
to 2 bits: "EXIST_P1", "EXIST_NEXT". With "merge" considered, adding "EXIST_P2"
would be enough. This avoids some invalid states, making it easier to reason
about. It also makes Mercurial dirstate hacks like size = -1, size = -2 noting
"merge" and "otherparent" unnecessary.

With this change, the previous `state_required_all`, `state_required_any`
query parameters are not powerful enough. That would be changed to functions
in a later diff. There is a new need to select files by querying "unset" bits.
That will be addressed by D7886281.

Reviewed By: markbt

Differential Revision: D7860277

fbshipit-source-id: 15d198fbd0ffa858c8ed751d42dff73e06114c12
2018-05-23 06:12:45 -07:00
Jun Wu
0b3737c904 treestate: avoid broad borrow on self in Node.visit
Summary:
Looking at the `path` variable, it is now:

  vec![KeyRef<'a>, KeyRef<'a>, KeyRef<'a>, ...]

where `'a` is bounded to `self`. That's okay but makes it impossible to
modify `self` between `path.push` and the end of the `visit` function.

For `Node<FileStateV2>`, `self` needs to be modified, because `visitor`
might change `file`'s state, and `self.aggregated_state` needs to be updated
accordingly.

Basically, the elements of `Vec` should match the call stack, and have
different lifetimes. A nested `visit()` should have a nested lifetime on
`Vec` elements, instead of relying on borrowing `self`.

  vec![KeyRef<'a>, KeyRef<'b>, KeyRef<'c>, ...]
  #    visit(path) {
  #                let name: KeyRef<'b>; // instead of <'a>.
  #                path.push(name)
  #                visit(path) {
  #                            let name: KeyRef<'c>;
  #                            path.push(name)
  #                            visit(path) { ... }
  #                            }
  #                }

The previous diff introduces a `VecStack` struct to work in this case. Let's
use it. This also make sure all `path.pop()` happens automatically even if
panic happens.

Reviewed By: markbt

Differential Revision: D7797457

fbshipit-source-id: e1723fac3dbd7d244b69f1fc46d90bef64a29a3f
2018-05-23 06:12:45 -07:00
Jun Wu
7fa9772b67 treestate: add VecStack to support nested lifetimes matching stack frames
Summary:
The motivation is to have something like:

  // in Node.visit
  let mut vec = Vec::new();
  {
    vec.push(self.foobar()); // borrows self
    ...
    vec.pop(); // vec no longer contains "&self". But rustc won't know.
  }
  // want to mutate "self" here.

Reviewed By: markbt

Differential Revision: D7803738

fbshipit-source-id: 1bf2fc5788e7963f3144db490d044fcdb5193fad
2018-05-23 06:12:45 -07:00
Jun Wu
5cd280bf4e treestate: add TreeState.has_dir
Summary: This is needed for certain code paths. Fix `Tree.has_dir("/")` special case.

Differential Revision: D7797455

fbshipit-source-id: 5855e7ad6ef73eb07d590dd5201367b5c7f86a96
2018-05-23 06:12:45 -07:00
Jun Wu
6111ea14fd treestate: add basic tests for TreeState
Summary: Basic tests about serialization and add/remove/modify operations.

Differential Revision: D7769657

fbshipit-source-id: 767ee8fb1d813de5bc99a5d6c81978c89c51f298
2018-05-23 06:12:45 -07:00
Jun Wu
a2e7d1cfe3 treestate: implement Rand for FileStateV2
Summary: This will be used in tests.

Differential Revision: D7769655

fbshipit-source-id: 27647685848c03f56740f49361dd286abdef8e33
2018-05-10 16:40:25 -07:00
Jun Wu
8f8adff716 treestate: add getter and setter of watchman clock to TreeState
Summary: Previously they cannot be changed.

Reviewed By: markbt

Differential Revision: D7769658

fbshipit-source-id: 4548eb90a82e9bd85fadf6a6f356cca7352fff0d
2018-04-30 19:10:45 -07:00
Jun Wu
224fe91344 treestate: add write methods to TreeState
Summary:
This allows writing `TreeState` state in two ways - save as a new file, or
incrementally update an existing file.

Reviewed By: markbt

Differential Revision: D7748822

fbshipit-source-id: 472b78af6cf7ea79968460a51ec824eaa96e4973
2018-04-30 19:10:45 -07:00
Jun Wu
a34b11ec8d treestate: make Tree write methods return BlockId
Summary: They are used by the next diff.

Reviewed By: markbt

Differential Revision: D7748834

fbshipit-source-id: 9562204975d83a8dce6eb80d2677387e24f8f0a0
2018-04-30 19:10:45 -07:00
Jun Wu
0923d108be treestate: add map-like operations to TreeState
Summary:
The method names are inspired by std HashMap. The types are slightly
different due to `Tree` implementation details.

Reviewed By: markbt

Differential Revision: D7748828

fbshipit-source-id: fc24481cdf0054c8e879d760082e192e52afc7f5
2018-04-30 19:10:45 -07:00
Jun Wu
b15a1b747f treestate: add Tree.get_mut
Summary:
The `Tree` object can return an `&mut` entry easily. Let's expose the
interface. This could be useful when the caller only wants to modify part of
the file state. For example, changing `copied` without touching anything
else.

Reviewed By: markbt

Differential Revision: D7748820

fbshipit-source-id: 430fa8ee310297c61866695a692134daf519e78d
2018-04-30 19:10:45 -07:00
Jun Wu
e07e4a99a1 treestate: add a TreeState struct
Summary:
Unlike TreeDirstate, this struct does not have two trees, and uses
FileStateV2.

Reviewed By: markbt

Differential Revision: D7748826

fbshipit-source-id: e637fad64e6b3e9b2a122e26a29fd04014181d6b
2018-04-30 19:10:45 -07:00
Jun Wu
9ce759a99a treestate: expose visit filtering via Tree.visit_advanced
Reviewed By: markbt

Differential Revision: D7748830

fbshipit-source-id: f3b41531e015fef90c01773ab65a4523ee72e7df
2018-04-25 17:38:20 -07:00
Jun Wu
53e7ab2a6c treestate: add file state filtering to Node.visit
Reviewed By: markbt

Differential Revision: D7748825

fbshipit-source-id: 2395ac8cc25fb4f4d3e6bdb5770616d859fcfab0
2018-04-25 17:38:20 -07:00
Jun Wu
23506bde19 treestate: merge Node.visit and Node.visit_changed
Summary:
They are similar. Merge into one single method. The `visit` method will be
extended to support other filtering features.

Reviewed By: markbt

Differential Revision: D7748829

fbshipit-source-id: 4388291945668a684808fe384341328ffd4ad2a8
2018-04-25 17:38:20 -07:00
Jun Wu
8cbe4d45c0 treestate: add serialization for Node<FileStateV2>
Reviewed By: markbt

Differential Revision: D7748832

fbshipit-source-id: bd7c6e8fce5b512068d86e16d441564e36565459
2018-04-25 17:38:20 -07:00
Jun Wu
b3da9a0262 treestate: add a compatibility layer for Node
Summary:
Allow `Node` type to work with both versions of file states. This is the
static dispatch approach that does not introduce runtime overhead.

Reviewed By: markbt

Differential Revision: D7748831

fbshipit-source-id: 4ac0386f9f93e55af1102b97a3510c8e872444a2
2018-04-25 17:38:20 -07:00
Jun Wu
cc0390192f treestate: add serialization for FileStateV2
Reviewed By: markbt

Differential Revision: D7748821

fbshipit-source-id: 56c7d7d81c86a8db05f6db2c8f6f02993cd07989
2018-04-25 17:38:20 -07:00
Jun Wu
75d50004f4 treestate: add serialization for StateFlags
Summary: It's just a thin wrapper about writing VLQ integers.

Reviewed By: markbt

Differential Revision: D7748835

fbshipit-source-id: 53a302afe51d551e49ac341901e2767d1a044946
2018-04-25 17:38:20 -07:00
Jun Wu
b796901d0c treestate: add StateFlags to Node
Summary:
This field stores the pre-computed aggregated state that helps fast path
traversal - if a state does not match the aggregated state, we can now skip
an entire tree quickly.

Reviewed By: markbt

Differential Revision: D7748823

fbshipit-source-id: 4b81ef5b911b4a21fdd46f8845ec217a75f5af8c
2018-04-25 17:38:20 -07:00
Jun Wu
71b99ae067 treestate: define a new file state object
Summary: The new `FileState` has a state bitflags field, and a "copied" information.

Reviewed By: markbt

Differential Revision: D7748824

fbshipit-source-id: a68687764e1b0c13252cb914673f2b16fa22d4ef
2018-04-25 17:38:19 -07:00
Jun Wu
d15213f6f5 treedirstate: move non-Python part to a separate crate
Summary:
This makes it easier to modify and test the core logic without coupling with
the Python logic.

Reviewed By: markbt

Differential Revision: D7734012

fbshipit-source-id: 0d7b19198d85f6ca7314611256e9271be60070d1
2018-04-24 15:59:07 -07:00