Summary: This just iterates over all the keys contained in the store.
Reviewed By: quark-zju
Differential Revision: D17104467
fbshipit-source-id: 34eee949cf4a1504df67c09fab7f71ef95210eed
Summary:
This moves code from AsyncMutableHistoryPack. This allows abstracting the type
of history store so the IndexedLogHistoryStore can be easily added.
Reviewed By: quark-zju
Differential Revision: D17100019
fbshipit-source-id: ca226336ae9ff05385f9218e88e853bf6e5356e3
Summary:
We've seen a handful of cases where the indexedlogdatastore becomes corrupted
which makes Mercurial unable to run properly. For now, and since we only use
the indexedlog for the hgcache, let's just remove it.
A better solution would be to harden the indexedlog code to better detect the
corruption and attempt to fix them.
Reviewed By: quark-zju
Differential Revision: D17115622
fbshipit-source-id: ee474a5df60c4414f6ea21ace7dff0f7048879c9
Summary:
I had assumed that we store p1 and p2 in the same order that they are used in
Node computation. That is incorrect. In general p1 and p2 are assumed to have
an ordering that matters and it's Node computation that is specific.
Reviewed By: quark-zju
Differential Revision: D17125743
fbshipit-source-id: 3a2673d9c243e2d2103aba0cb4fd8f536386efa7
Summary:
In some cases the finalize algorithm is used to persist data that is received
in a bundle. The process is that it constructs a store from the bundle and
goes to construct a tree with the root node received. It then goes through
finalize to generate the entries that need to be written to local storage.
Reviewed By: quark-zju
Differential Revision: D17125149
fbshipit-source-id: de5a1e922a6aebe48e238d8473177a8d3f7a9ef5
Summary:
`listdir` returns the contents a directory in the manifest. The format
is pretty simple, containing only the simple names of the files and or
directories. I don't know if this is something that eden can use because
it seems to simple. In other words, we have something but we may want
to iterate on it before we market it broadly.
Reviewed By: quark-zju
Differential Revision: D17098082
fbshipit-source-id: d6aa42c96781cf1f8b2e916fa10bb275593bdc65
Summary:
The C++ manifest implements walksubdirtrees which is used to compute the packs
that a "client" wants for a prefetch. In terms of interface the function is very
annoying and couples with storage and tree representations without being part
of any of them.
We reproduce that functionality as a means to replace the C++ implementation.
The long term goal is to do lazy fetches using an iteration style that plays
nicer with batching downloads.
This change also includes fastmanifest updates because they are required to
enable the walksubdirtrees functionality in our tests.
Reviewed By: quark-zju
Differential Revision: D17086669
fbshipit-source-id: 6c1f9fbf975814f0a2071f8d1c8e022e5ad58e29
Summary:
This makes it possible to use explicitly named arguments. It can simplify the
code a bit by moving the error handling (incorrect number of arguments) to the
derived code.
Reviewed By: xavierd
Differential Revision: D16905460
fbshipit-source-id: 93dc58d684bb52cd2f9bddc25108ef0c500f81db
Summary:
If there is no `#[args]` field set, the command does not need arguments. In
that case, just error out directly. This can simplify command implementations.
Reviewed By: xavierd
Differential Revision: D16905461
fbshipit-source-id: f4a2d03cf032f34d1b5794010774ff80eb88e1cd
Summary: This make it possible for the flags parsing to return an error.
Reviewed By: xavierd
Differential Revision: D16905462
fbshipit-source-id: 06a70ee7c72132ebb2df453a740b58e6b46dfba2
Summary:
Most commands do not care about the "command name" (aka. arg0). Only commands
sharing a similar implementation (ex. hg up/down/bottom/top) would need it.
Make it optional and make `args` not containing the command name.
Reviewed By: xavierd
Differential Revision: D16905459
fbshipit-source-id: 0a786731ebce2580e116fd50aad75062b03a1fa3
Summary: This makes the code easier to read as we add more fields to the macro state.
Reviewed By: xavierd
Differential Revision: D16905458
fbshipit-source-id: a3bc4dc1accf21cf491921469482f86aec16321b
Summary:
This makes it possible to capture output of a Python command via the Rust
hgcommands crate.
Reviewed By: xavierd
Differential Revision: D16866467
fbshipit-source-id: e64feece9b31196fabfea110015d5c45cb548046
Summary: This allows the io module to be used in non-extension code (ex. hgcommands).
Reviewed By: xavierd
Differential Revision: D16866458
fbshipit-source-id: b36b535e3804ba2b8e656508c10ca7ceb3e1fdf1
Summary: An upcoming change would like to downcast IO to stdio or PyObjects.
Reviewed By: xavierd
Differential Revision: D16866456
fbshipit-source-id: 73289aab13260ec421b979280f0b9c73383ab5d0
Summary: This makes it possible for Python to execute commands implemented in Rust.
Reviewed By: xavierd
Differential Revision: D16866457
fbshipit-source-id: e7352eed0e7b722c28974cb1123975f591ac44bc
Summary:
Expose a simple method in hgcommands to run the actual (Rust or Python) command.
This makes it easier to expose the "runcommand" API via bindings.
I have to disable chg temporarily since `hg root` with chg stops working at present.
chg will be re-enabled in a later diff.
Reviewed By: xavierd
Differential Revision: D16866461
fbshipit-source-id: 46c5df9a577d748fab302a8c7157d8aa62d6f1bd
Summary:
Previously, HgPython hardcodes the arguments to be std::env::args().
This diff makes that an configurable argument.
Reviewed By: xavierd
Differential Revision: D16866464
fbshipit-source-id: 3a37dd89793e0813732ea9b088ff22c9a4c6b125
Summary:
`Py_Finalize` is not recursive. It unconditionally destroys the Python interpreter
state. To make `hgcommands` usable in a library (ex. bindings), detect the case
where the interpreter was initialized, and skip `Py_Finalize` in that case.
Reviewed By: markbt
Differential Revision: D16866454
fbshipit-source-id: ea6bb6a83bc0b0fe7b5c2ce21158ae9d9e2a779d
Summary:
The indexedlog based datastore is being rolled out more broadly, let's add a
basic historystore indexedlog to replace the historypacks. One of its first use
will be in hg_memcache_client write path to remove some pathological cases
where hg_memcache_client can write thousands of packfiles. This in turn will
remove the need to run repack to keep the amount of packfiles in control.
The IndexedLog key is the concatenation of sha1(path) and the node. Hashing the
path should be fairly cheap and makes it easy to integrate with the IndexedLog.
One of the drawback versus the histpack will be storage space usage, as the
path is always stored per entry, while it was shared with multiple entries in
the history pack. I though about having a separate index to easily translate
the hashed path to the path, but due to the potential log rotation, we could
end-up in a case where the path isn't present at all in the store.
Reviewed By: quark-zju
Differential Revision: D16616082
fbshipit-source-id: 1e47260b479f8923cc137a39dcba54b2d074f43a
Summary:
Add a way to explicitly `fsync` data to reduce changes of data corruption in
case of hard reboots.
If `fsync` is set on a `Log`, it sets `fsync` recursively on its indexes and
checksum tables. If frequent `fsync` is an issue, it might be possible to only
`fsync` the source of truth (log and meta) and have other logic to rebuild
indexes on demand.
`fsync` on btrfs seems to be relatively cheap (comparing to ext4), so maybe
this is fine for btrfs users.
Reviewed By: markbt
Differential Revision: D16926659
fbshipit-source-id: a9de2fa352f607fe6f0b9d36047323862770f2e6
Summary:
Put OpenOptions as a field in Index. This simplifies Index struct as we add
more fields to OpenOptions.
Reviewed By: markbt
Differential Revision: D16926660
fbshipit-source-id: 0d6fad0eb1c9b38831494be74b3c440e4d827202
Summary:
Use `PathDataError` in some cases so they contain path information and might be
more useful.
Reviewed By: markbt
Differential Revision: D16876969
fbshipit-source-id: c85b576006872fdcdefb89588bccd871ace69aa6
Summary:
Define errors using failure_derive. Remove ad-hoc macro.
As we're here, also define some more-detailed errors. They'll be used in
upcoming diffs.
Reviewed By: markbt
Differential Revision: D16876971
fbshipit-source-id: 0a47a7e03876f27b5a5bb54c5f396a3816488c47
Summary:
Previously, checksum error just shows "integrity check failed". This diff
changes it to print the file path and byte range.
`#[inline(never)]` was preserved to maintain performance.
Reviewed By: markbt
Differential Revision: D16872827
fbshipit-source-id: 606a6f80a47028ce4d51e84bacbf1d90a3e9e048
Summary:
The pest codegen has some non-determinism (HashMap) that breaks buck build.
According to jsgf:
This basically indicates some kind of non-determinism in the build. They're a pain, so I'd been hoping that we'd got them all.
Yeah, pest is generating non-deterministic output, which will screw things up badly. The problem is:
https://github.com/pest-parser/pest/blob/master/generator/src/generator.rs#L92-L93
fn generate_builtin_rules() -> HashMap<&'static str, TokenStream> {
let mut builtins = HashMap::new();
is putting builtins into a `HashMap`, then:
https://github.com/pest-parser/pest/blob/master/generator/src/generator.rs#L46
rules.extend(defaults.into_iter().map(|name| builtins[name].clone()));
emitting them in hashmap order. It needs to use a `BTreeMap` to make sure they're in a consistent order.
(I didn't check whether there are other instances of this.)
Reviewed By: jsgf
Differential Revision: D17063573
fbshipit-source-id: c03adc3c6d50bd09ffbd44ca8dc7bc51d6cad28d
Summary:
The workspace was added by D8741175 mainly to make build artifacts share a same
"target" directory. D14606468 made `setup.py` write a `.cargo/config` that
specifies a sharable "target" directory. Remove the workspace since the crates
already share a "target" directory.
This makes it a bit easier to add new crates.
Reviewed By: singhsrb
Differential Revision: D17053934
fbshipit-source-id: d34781c796356b725ddce3453c1951a4d4133807
Summary:
The insert code would be unclear in what kind of issue it ran into when
inserting files. Sometimes the file we want to insert is a directory and
other times it want to traverse a directory. This change makes those
situations clear along with some other corner case behaviors.
Reviewed By: quark-zju
Differential Revision: D16775354
fbshipit-source-id: 50ab6bc52b70cc5cef013d11050eb3cdf5b160a5
Summary:
Updating the manifest implementation for remove with the intended API.
When I originally implemented remove I wasn't confident what was the
best way to implement remove. As I've gained more experience, I feel
confident that doing two iteration over the tree is a good approach
for this method. The first iteration should validate that the file
exists then the second iteration will actually traverse down updating
the nodes to mutable ephemerals.
Reviewed By: quark-zju
Differential Revision: D16775353
fbshipit-source-id: 8ebee9ca347efcb694a6d27c1eeae2c149643766
Summary:
`get_link` started as test function that broaden in scope but did not have
it's behavior updated as it started to be used more broadly.
No reason to error out when we request a path that has parent files in the
manifest.
Reviewed By: quark-zju
Differential Revision: D16775356
fbshipit-source-id: a320926100378f16d723ca204746906e79c7752e
Summary:
Add blackbox logging of syncing to or from the commit cloud workspace,
including which changes are being submitted.
Also log when obsmarker fixup happens in blackbox.
Reviewed By: quark-zju
Differential Revision: D16961281
fbshipit-source-id: 0d0f675d77ab3446198703b31eea940dae3bdd85
Summary: This makes `config|debugconfig` as a debug command. Previously it was not.
Reviewed By: singhsrb
Differential Revision: D16866460
fbshipit-source-id: f1fcf0777d2fe6c1426f557ae1999710dd722109
Summary:
Aliases with `:doc` in name are not real commands. Do not treat them as
commands.
The upstream patch https://phab.mercurial-scm.org/D5087 added other metadata
including `:help` and `:category`. We might end up having some in the future
so I blacklisted names with `:` in them, not just `:doc`.
Reviewed By: chadaustin
Differential Revision: D16955596
fbshipit-source-id: b6f3e1129b632e0ba43c800e7e6fdd2b52d3b40c
Summary:
`fs::canonicalize` has unwanted side effect - resolving symlinks.
Use `util::path::absolute` instead. This resolves a side effect where
Note on Linux, `chdir` resolves symlinks and `getcwd` (aka.
`std::env::current_dir`) will return the resolved path. On Windows
`std::env::current_dir` does not return the resolved path, at least
for the mapped drive created via the `subst` command.
Differential Revision: D16952484
fbshipit-source-id: 6969a1844020ff6b82de46416d8950ec40394159
Summary:
On Windows, some people use the "map drive" feature to map a long path (ex.
`C:\long\path\to\repo`) to a short path (ex. `Z:\`) so their tooling can
handle some long paths.
In that case, resolving symlinks by `hg root` is undesirable.
Unfortunately, the Rust stdlib does not have a Python `os.path.abspath`
equivalent. There were some attempts (ex. https://github.com/rust-lang/rust/pull/47363)
but the corner cases (ex. symlinks) have made the problem much more
complicated.
There are some 3rd-party crates. But they are not a good fit:
- https://github.com/danreeves/path-clean/ (last commit fb84930) follows the golang plan9 idea. It does not have proper support for Windows paths.
- https://github.com/vitiral/path_abs/ (latest commit 8370838) reinvents many path-related types, which is an overkill for this usecase.
This diff implements the feature "reasonably" for both Windows and Linux, with
nasty corner cases (symlink) ignored.
Differential Revision: D16952485
fbshipit-source-id: ba91f4975c2e018362e2530119765a380f103e19
Summary: The configparser crate is a better place for logic to load system and user config.
Reviewed By: sfilipco
Differential Revision: D16796396
fbshipit-source-id: c1ae5f85aa9839d89f5026279d699135dc1b442b
Summary:
Move `repo.shared_path` handling to repo initialization time and store it in
the repo structure.
This makes `repo.shared_path()` cheap if it gets used frequently.
Reviewed By: sfilipco
Differential Revision: D16796401
fbshipit-source-id: e19f3381cc87b55500ea1d27fd918ccb16a71972
Summary:
Some functions in `dispatch.rs` are about the "repo". Move them to a better
place - `repo.rs`.
The repo and config logic is coupled. A new enum `OptionalRepo` was added, to
make code easier to write - `Some(repo)` means the repo exists, and `repo` owns
the config. `None(config)` means the repo is missing, but the config is still
available.
Reviewed By: sfilipco
Differential Revision: D16796403
fbshipit-source-id: 2f4017a52296b629e990f85437b2cfdd7263b9e6
Summary:
"infer" means "try to get the repo path from command line arguments like a full path".
The enum variant is really "repo is optional". Rename to clarify.
Reviewed By: sfilipco
Differential Revision: D16796399
fbshipit-source-id: 505d2a406a83e0006200ece63d360b119548d2dd
Summary:
Change error type in clidispatch from `DispatchError` to `failure::Error`.
Pros:
- `failure` will attach a backtrace for free. (otherwise, backtrace handling is
manual)
- Wrapping other errors (ex. `io::Error`, `cliparser::Error`) is optional.
(otherwise, wrapping other errors is mandatory, and needs to be careful to
not lose information)
Cons:
- No longer able to enumerate *all* possible error types. (but can still
downcast to specific errors)
This seems to be a good tradeoff especially because of the backtrace handling - I
ran into a few issues where the location where the error happened really helped
debugging.
Since we can no longer enumerate all possible error types, the enum was changed
to individual structs to make the code shorter (ex. the struct can be downcasted
directly, instead of down-casting to the enum, then matching its variant).
The `HighLevelError` handling was simplified and moved to `hgmain`.
The new code path falls back to Python less aggressively, therefore some behaviors
were tweaked (ex. `-R` takes a bundle path).
Reviewed By: sfilipco
Differential Revision: D16796400
fbshipit-source-id: 1b17eb8d62503644b118c6b799778182bf222538
Summary:
This is more conistent with Mercurial style. And make them usable directly in
Rust code.
Reviewed By: sfilipco
Differential Revision: D16796397
fbshipit-source-id: 9016ea2b09fdf96b2b54138f5c8405caf96390f7
Summary:
This makes the "content" of the error stable. It is used in a later diff where
AmbiguousCommand gets handled by Rust directly instead of falling back to
Python dispatch.py.
Reviewed By: sfilipco
Differential Revision: D16796404
fbshipit-source-id: c439db14ec83c76c4762d3c627bfce1ea44bccf4
Summary:
In case there are many CommandDefinitions, we don't need all their flag
definitions until we decided to execute one of the commands.
Reviewed By: sfilipco
Differential Revision: D16796398
fbshipit-source-id: af205f59efd77fd7ff9eb4655d1f9167e2c350da
Summary: Convert global flags to `HgGlobalOpts` struct to make code shorter.
Reviewed By: sfilipco
Differential Revision: D16796407
fbshipit-source-id: b9d4c3dbec68c81908d439da4c353249347ca74a
Summary:
The tests are
- Mostly testing about configurations.
- Mostly depends on private functions.
ConfigSet already has a good test coverage, the tests do not provide much
value (aside from testing config override ordering, which is also covered
by hg tests). I'm going to change / remove the private functions. Remove the
tests to make changes easier.
Reviewed By: sfilipco
Differential Revision: D16796402
fbshipit-source-id: 56a8d55a0d1b0438bd2fcde5d3379d76f51dcd9d
Summary: This makes it possible to simplify the code reading global flags.
Reviewed By: sfilipco
Differential Revision: D16796405
fbshipit-source-id: eb604470d052ef84b748d1e60270cacd410fc5da
Summary:
The `Dispatcher` provides lots of features but its internal state only contains
the command table. Replace it with `CommandTable` and make the methods free
functions.
This makes function dependencies more cleaner, for example things like "locating
a repo", "getting the args" etc. won't require a `Dispatcher`.
A side effect of this change is the non-utf8 command line arguments are no longer
supported. This is already broken since our hg wrapper (accidentally) enforced
utf-8 command line. Therefore related tests are removed.
Reviewed By: sfilipco
Differential Revision: D16796395
fbshipit-source-id: a793ce7b8befe7caf62405c582fc932eb3daa099
