Summary:
The configparser tests `hg::tests::test_basic_hgplain` and
`hg::tests::test_hgplainexcept` set different values for `HGPLAIN` and
`HGPLAINEXCEPT`. Since the tests run in parallel and use the same environment,
one of the tests may fail if they run at the same time.
For these tests, create a mutex for the environment and lock it for the
duration of the test, ensuring these tests do not interfere with each other.
Reviewed By: jsgf
Differential Revision: D14615394
fbshipit-source-id: 9f123668d93223655514db2ae34b05354a6b578c
Summary:
Windows has 2 kinds of paths - the UNC path (starting with `\\?\`), and paths
most people use (ex. `C:\foo\bar`). The former is more powerful (reserved names
like `nul` can be used), and is the "canonicalized" form as seen by Rust stdlib.
The UNC paths are stricter, though. `/` is not treated as `\` automatically,
`.` and `..` are considered illegal. That is, trying to canonicalize a UNC path
with `..` or `.` will result in an error.
It's possible to get `.`, or `..` into part of an UNC path, by using the
`PathBuf::join` API provided by the Rust stdlib. That is, a legal UNC path stored
in `PathBuf` can become illegal by `join`ing a non-UNC path.
I'm not sure what's the most "clean" fix. Perhaps using two different types to
represent UNC path and non-UNC path in stdlib? But that's definitely not a trivial
(or even practical) change.
For now, just teach the config parser to "friendly try again" by stripping the
UNC prefix and re-canonicalize paths. So it can handle `.` or `..` used by
`%include`.
Reviewed By: sfilipco
Differential Revision: D14568119
fbshipit-source-id: 2a55faa945c8d03574fd56e82d946c9ef7f0138f
Summary:
The spec for both XDG and Mercurial say that when the XDG_CONFIG_HOME variable
is not set, we should default to $HOME/.config.
Windows and macOS also have a designated config folder outside of the home directory.
The `dirs` crate provides consistent access to this folder. I see no harm in looking at config
folders across all operating systems.
Reviewed By: quark-zju
Differential Revision: D14380686
fbshipit-source-id: 5e5a9cd4694aaa49fbc526f4917dc4afdaeb9842
Summary:
`listdir` makes it more expensive to detect config changes. We no longer need
it. Therefore drop the feature.
Reviewed By: markbt
Differential Revision: D13875655
fbshipit-source-id: 147adce45021c7b028aada5c40f498c2fd58c7f5
Summary:
D13875656 made a config path change that breaks tests without using HGRCPATH,
or local build runs.
Reviewed By: DurhamG
Differential Revision: D14034919
fbshipit-source-id: 80de214f1769a8f40e79dc0ab1dbba4d55f506a7
Summary:
Change system config entry point to only `/etc/mercurial/system.rc` (unix) and
`\ProgramData\Facebook\Mercurial\system.rc` (Windows) so they won't overlap
with a vanilla Mercurial installation.
Another goal of this change is to make it easier to drop the directory
`%include` feature. So detecting config changes (for example, edenfs wants to
make sure ignore rules are up-to-date) can be made cheaper by just stating
files without `listdir`.
Reviewed By: markbt
Differential Revision: D13875656
fbshipit-source-id: 314c0bf87ff086dec5b88e232edca0133356484e
Summary:
Update pest to 2.1.0.
This version has a new behaviour for parser error messages: the line feed at
the end of the line is shown in the error output.
Reviewed By: wez
Differential Revision: D13671099
fbshipit-source-id: b8d1142a44a56a0b21b3b72cf027f3f8a30f421e
Summary:
Use of `write!` requires checking for errors, however in this case, there is no
need to use `write!`, as we just want the error as a string.
Reviewed By: ikostia
Differential Revision: D13596497
fbshipit-source-id: 5892025344936936188cf3a8ca227e71eff57d55
Summary:
When I was debugging an eden importer issue with Puneet, we saw errors caused
by important extensions (ex. remotefilelog, lz4revlog) not being loaded. It
turned out that configpaser was checking the "exe dir" to decide where to
load "system configs". For example, If we run:
C:\open\fbsource\fbcode\scm\hg\build\pythonMSVC2015\python.exe eden_import_helper.py
The "exe dir" is "C:\open\fbsource\fbcode\scm\hg\build", and system config is
not there.
Instead of copying "mercurial.ini" to every possible "exe dir", this diff just
switches to a hard-coded system config path. It's now consistent with what we
do on POSIX systems.
The logic to copy "mercurial.ini" to "C:\open\fbsource\fbcode\scm\hg" or
"C:\tools\hg" become unnecessary and are removed.
Reviewed By: singhsrb
Differential Revision: D13542939
fbshipit-source-id: 5fb50d8e42d36ec6da28af29de89966628fe5549
Summary:
Add "--dry-run" for fix-code.py and use it in test-check.
This avoids license header and version = "*" issues.
Reviewed By: ikostia
Differential Revision: D10213070
fbshipit-source-id: 9fdd49ead3dfcecf292d5f42c028f20e5dde65d3
Summary:
This diff implements getBlob on top of the mercurial rust
datapack code. It adds a C++ binding on top of the rust code to
make it easier to use and hooks it up in the hg backing store.
Need to figure this out for our opensource and windows builds:
* Need to teach them how to build and link the rust code
* need to add a windows version of the methods that accept paths;
this is just a matter of adding a WCHAR version of the functions.
Reviewed By: strager
Differential Revision: D10433450
fbshipit-source-id: 45ce34fb9c383ea6018a0ca858581e0fe11ef3b5
Summary:
The `configlist` function converts a config value to a list of strings.
I have thought about using pest to parse it. However, pest might return errors
(ex. `a,",b` does not parse due to missing end quote), while the original logic
can happily parse everything (`a,",b` gets parsed into `['a', '"', 'b']`).
The code might be simplified to make it more obvious that `unwrap()` cannot
panic. But it handles so many corner cases that I'd like to port as-is for
correctness.
Reviewed By: DurhamG
Differential Revision: D9323743
fbshipit-source-id: 5f8be562b7437260b7551d87d751424558d76e8f
Summary: This is just the result of running `./contrib/fix-code.py $(hg files .)`
Reviewed By: ikostia
Differential Revision: D10213075
fbshipit-source-id: 88577c9b9588a5b44fcf1fe6f0082815dfeb363a
Summary:
Before this patch, `%include` support on Windows is:
# Works fine - UNC path: `\\?\c:\1.rc`.
%include c:\1.rc
# Works fine - UNC path: `\\?\c:\1.rc`.
%include \1.rc
# Works fine - UNC path: `\\?\c:\1.rc`.
%include c:/1.rc
# Bad - UNC path: `\\?\c:/1.rc`.
%include /1.rc
People expect `%include /1.rc` to work on Windows. Fix it by normalizing
the path in `%include` handling.
More context:
Normally, `/` and `\` can be used interchangeably on Windows. But it's not true
for UNC paths. The config parser uses `std::fs::canonicalize` to normalize
paths. The following Python script demonstrates the difference:
>>> import os
>>> open('c:\\1.rc').close()
>>> os.path.exists('\\\\?\\c:\\1.rc')
True
>>> os.path.exists('\\\\?\\c:/1.rc')
False
Reviewed By: phillco
Differential Revision: D10036882
fbshipit-source-id: fd85e0bc86d1e5776701077751ac875e71d60568
Summary:
It's cleaner for the config parser to take care of environment variable
handling.
A side effect of this change is, `$HGPROF` only affects `profiling.type`,
not `profiling:foo.type`, which is more desirable since we don't want
`profiling:foo.type` to be overridden by `$HGPROF`.
Reviewed By: markbt
Differential Revision: D9828547
fbshipit-source-id: 27be3683beee60a4eee6040ca1b4160dc1a89f73
Summary:
Preserve leading (but not tailing) new lines so the config (where `_` denotes a
space):
x_=__
__Foo
__
is parsed as `"\nFoo"`.
This is useful in template configs.
Reviewed By: ryanmce
Differential Revision: D9929764
fbshipit-source-id: e30659df94937c7c2121627f42ea425191003fb1
Summary:
Be more permissive about spaces. Namely:
- Spaces after a section name like `[foo] ` are allowed.
- Spaces in config names are allowed.
- Spaces at trailing lines are ignored and no longer insert an `\n` to the previous config.
This makes it closer to the older config parser behavior. But it's still
different on some cases, like `[foo]]`, `[foo] # bar`, `[foo]]` still do not
parse.
Benchmark shows no obvious (within 10%) slowdown. So this is probably fine.
Reviewed By: strager
Differential Revision: D9620253
fbshipit-source-id: 8489ef8e83606d0557db56e8da0a017d55ff1514
Summary: This would provide information about performance changes.
Reviewed By: singhsrb
Differential Revision: D9620252
fbshipit-source-id: 51d243b50b349c63e552bd1c43db17497025f73a
Summary:
liubov-dmitrieva encountered an issue where her home hgrc is not loaded. That's because
environment variables in HGRCPATH are not expanded. Fix it by calling
`expand_path` on the paths.
Reviewed By: phillco
Differential Revision: D9499239
fbshipit-source-id: cd4b7a26fd12f1c3148a21dbb5584bbeb3885286
Summary:
Orignally both `mercurial.ini` and `hgrc.d` were looked up in the same location
as main Mercurial executable, not in the `datadir`. See from `scmwindows.py`:
```
filename = util.executablepath()
# Use mercurial.ini found in directory with hg.exe
progrc = os.path.join(os.path.dirname(filename), "mercurial.ini")
rcpath.append(progrc)
# Use hgrc.d found in directory with hg.exe
progrcd = os.path.join(os.path.dirname(filename), "hgrc.d")
```
Reviewed By: quark-zju
Differential Revision: D9540052
fbshipit-source-id: d5921193dd14fcb46cf428aaa77d26a58aef7868
Summary:
Without this patch, all hg commands will fail with our current config:
hg: parse error: <filename>
--> 6:5
|
6 | commandexception
| ^---
|
= expected new_line
The config is:
[blackbox]
track = command
commandexception
...
Because "\r\n" was treated as the same as double "\n"s.
Reviewed By: ryanmce
Differential Revision: D9494909
fbshipit-source-id: 64ef173c69f3cf61d4e71116c581dbca72fb2c4b
Summary:
The config remapping, whitelisting features are hg specific. And is done by
using `append_filter` API exposed by `config.rs`. They are more of "extended
features". So move them to `hg.rs`.
Reviewed By: DurhamG
Differential Revision: D9323789
fbshipit-source-id: 89bc4416ee7276c2d1d4db8eba6404747cbb4ec4
Summary:
On Windows `%include` can have paths containing environment variables like
`%PROGRAMDATA%`. We already ship that kind of config files to users therefore
let's add support for that.
The change assumes `%` is not used as part of a normal path, which is probably
good enough for practical uses. If `%` does need to be legally used in a
filename, we can add escaping support later.
Reviewed By: DurhamG
Differential Revision: D9283303
fbshipit-source-id: bcc80307fe19dfc40aea88b6a0a5f69681e835fc
Summary:
Embed a snapshot of the config file at the parsing time. So applications can
have access to them, and can do things like calculating line numbers, or editing
the config files.
This is shallow copy. So it does not affect performance.
Reviewed By: DurhamG
Differential Revision: D8960872
fbshipit-source-id: e1905712dbec4b02d93a4fecc97064f0e00024c8
Summary: Otherwise there is no way to get parse errors with the last change.
Reviewed By: DurhamG
Differential Revision: D8960867
fbshipit-source-id: 48ef748096a67baa155bddf202c8ebec7ed1eeb5
Summary:
Change the API to return parse errors directly, instead of keeping them in
ConfigSet struct. This makes it easier to get errors related to one of the
"parse" calls.
Reviewed By: DurhamG
Differential Revision: D8960869
fbshipit-source-id: fbd571f264415e788c5ac44961149d1498826a6d
Summary:
Multiline value like:
[section]
foo = a
b
should be parsed as "a\nb", instead of "a \nb".
It does not affect configlist, but affects template definations.
Unfortunately in this case we had to allocate a new buffer instead of using
`Bytes::slice`. Fortunately most configs are single-line, so the performance
impact is hardly visible practically.
Reviewed By: DurhamG
Differential Revision: D8960866
fbshipit-source-id: 011e7f431d682236529ce176fe577aac6a010d91
Summary:
Switch to indexmap, which is more actively maintained than linked-hash-map.
There is no visible performance difference when parsing large config files.
Reviewed By: DurhamG
Differential Revision: D8960870
fbshipit-source-id: 8d6650e2d8b14989061dceb2081a3f93004cea76
Summary:
`home_dir` in stdlib is going to be deprecated. Therefore switch to
external crate.
Reviewed By: DurhamG
Differential Revision: D8960874
fbshipit-source-id: e123debc5c58e6a632a801dedcd9fc6834cb1f65
Summary:
[pest](https://github.com/pest-parser/pest) is an elegant Rust library for
parsing text.
A navie benchmark on a 1MB config file shows pest is about 1.5 to 2x slower.
But the better error message and cleaner code seems worth it.
Practically, in a VirtualBox VM, parsing a set of our config files takes 3-7ms.
The overhead seems to be opening too many files. Reducing it to one file makes
parsing complete in 2-4ms.
Unfortunately the buck build has issues with the elegant syntax
`#[grammar = "spec.pest"]`, because "spec.pest" cannot be located by pest_derive.
Therefore a workaround is used to generate the parser.
The motivation behind this is because I noticed multi-line value can not be
taken as a plain Bytes slice. For example:
[section]
foo = line1
line2
"foo" should be "line1\nline2", instead of "line1\n line2". It does not make a
difference on configlist. But it affects templates. Rather than making the
parser more complex, it seems better to just adopt a reasonbly fast parsing
library.
Reviewed By: DurhamG
Differential Revision: D8960876
fbshipit-source-id: 2fa04e38b706f7126008512732c9efa168f84cc7
Summary:
Previously, a line with all space characters is considered "illegal" and I
didn't handle it. It would actually be parsed as part of config name.
Let's skip them. So config files with spaces would behave sanely.
Reviewed By: DurhamG
Differential Revision: D8887370
fbshipit-source-id: e55d221d281fc58b2d2efbcb9196e7f68a78d719
Summary:
Command-line flags override config files configs. However, config files load
after parsing command-line flags in Mercurial's current logic. Therefore, a way
to make sure config files do not override command line flags is needed.
`ui.py` uses two config objects `ui._ocfg`, `ui._ucfg` and calls
`_ucfg.update(_ocfg)` after loading a config file to solve the problem.
That adds overhead updating ucfg.
With configparser's "filter" API, instead of rewriting configs afterwards,
the configs can be stopped from loading via files in the first place. So
there is no overhead maintaining two config sets and updating them.
Reviewed By: DurhamG
Differential Revision: D8960877
fbshipit-source-id: cf7b9a820911638956e123c1c93d3febeabf53c2
Summary:
This is to locate system / user config files at some fixed places.
It will replace most of `mercurial.rcutil`, and be used in native clients.
Reviewed By: DurhamG
Differential Revision: D8895791
fbshipit-source-id: 47166b943a3bd90a8aff1c15674a3da4e14bf8d3
Summary:
Implement HGPLAIN handling using the filter feature of `Options`.
This is hg-specific, therefore it's implemented in a separate `hg` module, as
an extension to `config::Options`.
The `hg` module could contain more hg related logic, like locating system
and user config files, to make it easier to use by Eden.
The plan is to have this as the single source of truth handling HGPLAIN
environment variables and migrate other places reading HGPLAIN to
use side effects caused by functions defined here. The side effects
are ideally just normal config options accessible via `ConfigSet::get` APIs,
instead of another special case (ex. `HgPlain::get(name) -> bool`).
Reviewed By: DurhamG
Differential Revision: D8895788
fbshipit-source-id: fa0ad7e7207513d947216292cbbd65530391cf11
Summary:
The feature is required by Mercurial config layer. It's used by hgweb and
some templater configs.
Reviewed By: DurhamG
Differential Revision: D8886246
fbshipit-source-id: 836fc255b821e6b6c50cf2a435837e9051e90a7d
Summary:
The Mercurial API allows setting a section whitelist when parsing configs.
Let's add such feature to the Rust config parser.
Reviewed By: StanislavGlebik
Differential Revision: D8886247
fbshipit-source-id: 981026b98962e065b536077012d7d1042d2ada91
Summary:
There are some advanced config related requirements in Mercurial:
- Drop certain configs if certain HGPLAIN features are set.
- But, do not drop HGPLAIN configs if the config is set via CLI flags.
- Remap section names.
- Whitelist sections.
This diff adds a filter function option aiming to support all of the above.
Reviewed By: StanislavGlebik
Differential Revision: D8895787
fbshipit-source-id: 1abd90974c4e4b3f7f2fb33173ad2af34e0a4a65
Summary:
It turns out that "source" is not the only "option" that the caller needs to
set. From Mercurial's existing code, namely `ui.readconfig`, the API also
needs to support whitelisting config sections, and "remap" config sections.
Instead of adding more parameters to almost all functions. Let's add an
`Options` struct that will holds those configs. For now, it only has
`source`. New fields will be added by upcoming changes.
To help existing code migrate smoothly, and satisfy the most common
use-cases where only "source" is set, a `From<impl Into<Bytes>>` trait is
implemented.
Reviewed By: StanislavGlebik
Differential Revision: D8886244
fbshipit-source-id: 90b49565de6fbbce3e8e48db8e6805154d156360
Summary:
`std::env::home_dir` got deprecated [1]. But the replacements are not in tp2
yet (meaning the buck build will fail). So let's silence the warning for now.
As we're here, also fix an incorrect comment.
[1]: https://internals.rust-lang.org/t/deprecate-or-break-fix-std-env-home-dir/7315
Reviewed By: mitrandir77
Differential Revision: D8886248
fbshipit-source-id: aca0334cbc8b710e42c5c86c952f58adcd10ba2c
Summary: Exposes important types so they can be used in other crates.
Reviewed By: mitrandir77
Differential Revision: D8790923
fbshipit-source-id: 955249219ba5d963d0529ba35f79ed4a8120140a
Summary:
Handling sections and normal config items. `%` support will be added in an
upcoming patch.
Note: regex would make the code simpler - the expression
`^([^\s=]+)\s*=\s*(.*(?:\n[\t ].*)*)\s*` can extract both config name and
multi-line values. However a naive benchmark shows it is 20x slower parsing
larger files, and it has some initialization cost. Config parsing is at such
a low level and its performance is critical. So the code does its own
parsing instead of using regex.
Reviewed By: mitrandir77
Differential Revision: D8779051
fbshipit-source-id: a2de698f0676c886737c47891a0400f187bff822
Summary:
Add functions to load a path, where the path can either be a directory, or a
file. Implement the directory traversal. Loading a file is the most complex
part and will be implemented by an upcoming diff.
Reviewed By: lukaspiatkowski
Differential Revision: D8779052
fbshipit-source-id: f25265b4b7cc5df5cc3717643c3d0ee9cf6da8a4
Summary: They will be used by the actual parser.
Reviewed By: lukaspiatkowski
Differential Revision: D8777326
fbshipit-source-id: c6cda3168a060b1d36aaf3224a5e547d0aa45530
