Summary:
The windows and ubuntu builds don't have a version of rust that
supports these features, so this breaks the build.
Reviewed By: phillco, quark-zju, singhsrb
Differential Revision: D8289651
fbshipit-source-id: d08b141b4d9996e3b899ac0604225ad34f863990
Summary:
just refactoring to improve the code quality
the main improvement is that I separated TcpReceiver to a different service,
any other services can register callbacks with TcpReceiver service.
For WorkspaceSubscriberService callbacks are implemented using mpsc channel to notify the main WorkspaceSubscriberService thread and single atomic flag that allows running subscriptions to join.
Another improvement is that I added logic to run cloud sync on the first keep alive after connection errors
Reviewed By: markbt
Differential Revision: D8226109
fbshipit-source-id: 3fe513da9273b28b2262948ecdf620821e7ab313
Summary:
Without this change, code doing `index.get(...).values().collect()` might
end up with an infinite loop.
Reviewed By: DurhamG
Differential Revision: D8156510
fbshipit-source-id: 5497aa354de7d49cfc4308a025856608ce981a1e
Summary:
Previously, the index API optionally takes a root offset. This is
inconvenient for the caller since they probably need to record both
valid file length and root offsets. Since root nodes are always at
the end of the index. Let's just simplify the API to take a logical
file length instead of a root offset.
Reviewed By: DurhamG
Differential Revision: D8156512
fbshipit-source-id: 7029272a61c9990e6484bca7ebbff64e2233c6cd
Summary:
Previously, `mmap_readonly` always reads file length, and uses that for mmap
length. In many cases we do know the desired file length and it's cleaner to
not `mmap` unused bytes. So let's add a parameter to do that.
Note: The `stat` call is still needed. Since `mmap` wouldn't return an error
of the requested length is greater than the file length.
Reviewed By: DurhamG
Differential Revision: D8156523
fbshipit-source-id: 991aa28f3542eaff24387dcc6a7302122fb6962f
Summary: The function will be reused in another module.
Reviewed By: DurhamG
Differential Revision: D8156522
fbshipit-source-id: 2aff6f2e4b8fc9b5d2c000e12ac2d940f7fab407
Summary: This is just a refactor to address the naming scheme.
Reviewed By: quark-zju
Differential Revision: D8269217
fbshipit-source-id: 8c52d2c67837550e0b7dc1a45b3faf9a80319b61
Summary:
Based on review for D8214151 by quark-zju, addressing the nit here as
well.
Reviewed By: quark-zju
Differential Revision: D8267140
fbshipit-source-id: 12c3355852a49859c2b0a243fa8666105c914c73
Summary:
Adding the tests for the case when the union store has only one data
store which always returns an `Err` as `Result`. This `Err` is not of the type
`KeyError` which the union store handles differently.
Reviewed By: quark-zju
Differential Revision: D8214156
fbshipit-source-id: bd077af343086c92f46ec6a6f1551d05dd9bda09
Summary:
Adding tests for the case when the union store only has a single data
store which is completely empty.
Reviewed By: quark-zju
Differential Revision: D8214151
fbshipit-source-id: 9d8f329548a1b7e105a5dc6219067a6e292fe97c
Summary:
This commit just renames the methods to be more specific. This is
useful for later changes.
Reviewed By: quark-zju
Differential Revision: D8214153
fbshipit-source-id: e8db9148334f7cd539aca626e3798e256b9b022f
Summary:
Big change here is update to rand 0.5. This is a significant API
change. quickcheck still uses rand 0.4, so for quickcheck users I changed it so
that quickcheck re-exports the rand it uses. This means that quickcheck users
are unchanged aside from using quickcheck::rand, whereas direct rand users have
been updated to use the new API.
Reviewed By: farnz
Differential Revision: D8234503
fbshipit-source-id: f9e620851b8dfcc33f22a0af26122adcd5fbde39
Summary:
Refactor the cdatapack logic that computes the fanout table. This more
accurately computes the correct ranges to bisect for each fanout table entry.
This fixes an off-by-one error setting end_index in most buckets that caused
it to search a slightly larger bisection range than necessary.
This also fixes the code to accurately compute which buckets do not have any
nodes, and sets a (start, end) range of (1, 0) for these buckets, causing
find() to avoid having to search anything in these cases.
Reviewed By: quark-zju
Differential Revision: D8131019
fbshipit-source-id: 70d6d0f2e1d900a2df27b64f3a38f114d301be0d
Summary:
If all of the nodes in a datapack file start with the same byte value,
the cdatapack code computed the fanout table incorrectly. If this byte value
was anything other than 0x00 it would only be able to find the last node in
the pack, and would not be able to find any other nodes.
This fixes the code to compute the fanout table correctly in this case.
Reviewed By: quark-zju
Differential Revision: D8131020
fbshipit-source-id: 84e49befc5776cff96831f6120194466d9c80b35
Summary:
Added logic to control logging rate: empty messages that comes to confirm the subscription is alive, also on error logging rate when we are offline, also when we are running in standby with no active subscriptions
Also, I made a simple cross platform API, so that hg can trigger restart subscriptions in 2 lines of code. It is simple request - response API on tcp socket and json.
If a human run `hg cloud join`, hg will add subscriber file to the directory scm daemon reads subscribers from and will send the restart command, same for any `hg cloud leave` run
Another advantage is that the client (hg) can very easy check if the scm daemon is alive or not. (In 2 lines of code, cross platform, without any pid logic or other platform specific ifs)
Another advantage is that we can use it to receive some stats from the scm daemon.
I decided do not go with any watching directory logic, because changes are really rare events, and it will be better if a client (hg) will just notify the service to restart subscriptions when needed.
Also, I verified that hg and SCM Daemon use the same config options and logic related to detected home directory on different platforms and reading the token.
Reviewed By: markbt
Differential Revision: D8162237
fbshipit-source-id: 3cb48b90f5e065ce4dc7fdc7215c3ce6ad57fb9a
Summary: Mostly empty lines removed and added. A few bugfixes on excessive line splitting.
Reviewed By: quark-zju
Differential Revision: D8199128
fbshipit-source-id: 90c1616061bfd7cfbba0b75f03f89683340374d5
Summary:
Add an internal `get_dir` API to return aggregated states. It is exposed via
`.get('dir/')` python interface.
This is useful for implementing `hastrackeddir` of the dirstatemap class.
Reviewed By: markbt
Differential Revision: D7909173
fbshipit-source-id: 100a8f36237a6b911a4bfb4afbb4c63b98611317
Summary:
`Node.write_ext` currently contains the logic to calculate
`aggregated_state` and write it to disk. A future diff would need the
calculation without the writing part. So let's split the method.
Reviewed By: markbt
Differential Revision: D7909177
fbshipit-source-id: e83d622e6c1eb512c6a0c3ea8c7201055aa67a21
Summary:
Inserting file with names like `a/b/` should be forbidden to avoid errors
later.
Reviewed By: markbt
Differential Revision: D7886809
fbshipit-source-id: c78d357542af4fdc1cea70ad5751b356d3cb308d
Summary:
Not sure how this actually happens. But it happened in a test instance.
Re-run the test locally on the same commit does not reproduce it though.
Reviewed By: phillco
Differential Revision: D8169935
fbshipit-source-id: 7e71611915722d68a1bc633819b94836f63fbc3f
Summary:
Update the open_datapack() function to fail with an DATAPACK_HANDLE_CORRUPT
error if the index file contains a valid header but no index entries.
Previously open_datapack() would succeed, but it would incorrectly read past
the end of the mmapped data when trying to find entries.
The python code never generates empty pack files, so this behavior change
should not affect any legitimate pack files.
Reviewed By: quark-zju
Differential Revision: D8131018
fbshipit-source-id: a07eac9048314c9974cae0f98efaf8a4a383d966
Summary:
The command-line utility takes 2 files and compress/decompress them.
It's not meant to be formally used like `zstd`. But is handy to do simple
tests without writing Rust code.
Reviewed By: DurhamG
Differential Revision: D7596169
fbshipit-source-id: 6c21a38e21a061fab7032ff823b907b0e586bd42
Summary:
`rand` 0.5 has too many breaking changes that the code is not ready to
migrate yet. So let's ping rand to 0.4. Ideally all dependencies in
Cargo.toml should avoid using "*". But for now `rand` is the only
troublemaker.
Note `rand 0.4` is a dependency of `quickcheck 0.6.2` so it's available.
Reviewed By: phillco, singhsrb
Differential Revision: D8158406
fbshipit-source-id: 417ae6807a2efc650acb8d82370964fab6531fdb
Summary:
Using zstd dictionary as the "delta base" can result in overall better and
faster compression (than things like mdiff + zstd, or fossil_delta + zstd).
This diff adds utility functions to do delta generation and application.
It tweaks compression parameters (wlog, hlog) to optimize the "delta-ing"
usecase. It now hardcoded the "fast" strategy (level=1) to have reasonable
speed. We can add other compression levels later if needed.
Reviewed By: jsgf
Differential Revision: D7562908
fbshipit-source-id: 3334059b4abeb8923d603d055bde0bfdc854bc7b
Summary: This changes to support Scm Daemon on dev machines
Reviewed By: farnz
Differential Revision: D8139892
fbshipit-source-id: b6df53d6ce6615d24822b739d4d1705e0f572660
Summary: Scm Daemon initial implementation that currently just listen to Commit Cloud Live Notifications and trigger `hg cloud sync` on notifications
Reviewed By: markbt
Differential Revision: D8119768
fbshipit-source-id: a0d86624fe4b81b3adc89990640916d3da279b8c
Summary:
Update cdatapack_dump and cdatapack_get to check the return value from
`open_datapack()` to confirm if they actually successfully opened the file.
Previously these programs would segfault if invoked with a non-existent path.
Reviewed By: quark-zju
Differential Revision: D8131017
fbshipit-source-id: 90800de57430efd176b8e71fa84161f7b288e375
Summary:
Previously, `aggregated_state` is only a union of bit flags. It makes
querying files with rare bits fast. But it cannot help with common bits.
This diff adds an `intersection` field, so both "having rare bits", and
"not having common bits" can be queried efficiently.
As we're here, use an explicit `None` to represent "need re-calculation",
instead of using `is_empty()`. This makes the code easier to reason about.
It also solves an issue in `add` that is caught by the next test.
Reviewed By: markbt
Differential Revision: D7886281
fbshipit-source-id: 4ce395883ea26ea9b33794e03c792ea157dc21d0
Summary:
The field is a string that contains information `TreeState` itself does not
care about. It's up to the upper layer to decide how to use it, and it does
not have to be watchman clock, and might contain other fields like hostname,
ignore matcher hash, etc. So let's rename it to clarify.
Differential Revision: D7886282
fbshipit-source-id: 739a85d7a710918e0b18a9b7fe0e31b366bab447
Summary: Similar to `TreeDirstate`, those methods are required.
Reviewed By: markbt
Differential Revision: D7874126
fbshipit-source-id: 6dbd6c47c7ba2ded7ea7389dfa9de4cf43db8a01
Summary:
The method looks like a foot-gun, and it's O(all entries) instead of
O(cached entries). Change `get_tracked_filtered_key` to take an identity of
the filter function explicitly to solve the problem.
Reviewed By: markbt
Differential Revision: D7861765
fbshipit-source-id: a57ca4a7597120a5b00c63f3f373a62e19e5a834
Summary:
Follow up of the StateFlags change. Previously the code uses simple bitwise
operations to decide what to visit. That can not express complex conditions.
Let's use functions instead. Rustc should know how to inline those
functions.
Reviewed By: markbt
Differential Revision: D7860276
fbshipit-source-id: 71bf381e00adbb3259a1ae61dbd68fa67f02efdb
Summary:
The upcoming diffs need to do something like:
self.entries.as_mut().iter_mut() {
self.aggregated_state = ...
}
That cannot be done if accessing entries and aggregated_state both need
`&mut` borrow of `self`. So let's use interior mutability.
Reviewed By: markbt
Differential Revision: D7824116
fbshipit-source-id: 67dd317ebfbfc698e79ed6c0e96e69d3fe495a26
Summary:
The "added", "removed", "normal", "? (untracked)" 4 states could be simplified
to 2 bits: "EXIST_P1", "EXIST_NEXT". With "merge" considered, adding "EXIST_P2"
would be enough. This avoids some invalid states, making it easier to reason
about. It also makes Mercurial dirstate hacks like size = -1, size = -2 noting
"merge" and "otherparent" unnecessary.
With this change, the previous `state_required_all`, `state_required_any`
query parameters are not powerful enough. That would be changed to functions
in a later diff. There is a new need to select files by querying "unset" bits.
That will be addressed by D7886281.
Reviewed By: markbt
Differential Revision: D7860277
fbshipit-source-id: 15d198fbd0ffa858c8ed751d42dff73e06114c12
Summary:
Looking at the `path` variable, it is now:
vec![KeyRef<'a>, KeyRef<'a>, KeyRef<'a>, ...]
where `'a` is bounded to `self`. That's okay but makes it impossible to
modify `self` between `path.push` and the end of the `visit` function.
For `Node<FileStateV2>`, `self` needs to be modified, because `visitor`
might change `file`'s state, and `self.aggregated_state` needs to be updated
accordingly.
Basically, the elements of `Vec` should match the call stack, and have
different lifetimes. A nested `visit()` should have a nested lifetime on
`Vec` elements, instead of relying on borrowing `self`.
vec![KeyRef<'a>, KeyRef<'b>, KeyRef<'c>, ...]
# visit(path) {
# let name: KeyRef<'b>; // instead of <'a>.
# path.push(name)
# visit(path) {
# let name: KeyRef<'c>;
# path.push(name)
# visit(path) { ... }
# }
# }
The previous diff introduces a `VecStack` struct to work in this case. Let's
use it. This also make sure all `path.pop()` happens automatically even if
panic happens.
Reviewed By: markbt
Differential Revision: D7797457
fbshipit-source-id: e1723fac3dbd7d244b69f1fc46d90bef64a29a3f
Summary: This is needed for certain code paths. Fix `Tree.has_dir("/")` special case.
Differential Revision: D7797455
fbshipit-source-id: 5855e7ad6ef73eb07d590dd5201367b5c7f86a96
Summary:
watchman rust client: add tests for arrays of objects
this contains a reproducer test for broken compact arrays in combination with untagged enums
Reviewed By: sunshowers
Differential Revision: D7877495
fbshipit-source-id: 7fc3c05f1590e708a7645f0f9adbfd545e8bae42
Summary:
watchman rust client for mercurial needs
* describe responses and request with strong typing
* supports bser and json protocols
* command line transport and unix socket transport
* socket discovery (env or query command line client)
* read timeouts
Reviewed By: wez
Differential Revision: D7764586
fbshipit-source-id: 1f5725f6ce615e3e6e30395d09b5b37e0c2229d4
Summary:
The crates.io lz4 bindings only support the lz4 framed format, while
our python lz4 library produces custom framed compressed blobs. Let's add a new
wrapper around lz4-sys that handles are special framing. We can migrate to the
standard framing later.
Reviewed By: quark-zju
Differential Revision: D7855502
fbshipit-source-id: 04abb1bc784c6be7f22bcd80645d1b50debc93bd
