Summary:
This new disallowlist will let us specify config section.key's which
should not be accepted from old rc files. This will let us incrementally disable
loading of those configs from the old files, which will then let us delete them
from the old rc's and eventually delete the old rc's entirely.
This diff also removes hgrc.local and hgrc.od from the list of configs we
verify, since those are not on the list of configs that need to be removed in
this initiative.
Reviewed By: quark-zju
Differential Revision: D23065595
fbshipit-source-id: 5cd742d099efd651174cab5e87bb7cdc4bae8054
Summary:
Previously the backing store was loading configs manually. Now that
system, dynamic, user, and repo config loading are unified, let's go through
that approved path.
Reviewed By: kulshrax
Differential Revision: D22736338
fbshipit-source-id: 232023e660107a096691e9d99bf89c04c218dfbd
Summary:
The last few diffs prepared Rust for loading dynamic and repo configs.
This diff finally changes Python so that we're no longer doing any of this work
in Python.
Reviewed By: quark-zju
Differential Revision: D22712625
fbshipit-source-id: 0b71e81d79d10ea3ce7a1b31f315ada5728af9a9
Summary:
This threads the calls to load_dynamic and load_repo through the Rust
layer up to the Python bindings. This diff does 2 notable things:
1. It adds a reload API for reloading configs in place, versus creating a new
one. This will be used in localrepo.__init__ to construct a new config for the
repo while still maintaining the old pinned values from the copied ui.
2. It threads a repo path and readonly config list from Python down to the Rust
code. This allows load_dynamic and load_repo to operate on the repo path, and
allows the readonly filter to applied to all configs during reloading.
Reviewed By: quark-zju
Differential Revision: D22712623
fbshipit-source-id: a0f372f4971c5feac2f20e89a0fb3fe6d4a65d6f
Summary:
In a future diff we'll enable dynamic and repo config loading purely
from Rust. To do so we need load functions for both cases. A future diff will
call these.
The dynamicconfig loading is based off the Python equivalent in uiconfig.py
Reviewed By: quark-zju
Differential Revision: D22712624
fbshipit-source-id: ff46f6315fb80d4cd9e31d875ac60264563b12f2
Summary:
Previously load_system would skip loading if HGRCPATH was present and
then load_user would actually load the HGRCPATH. In an upcoming diff I add
load_dynamic, which happens after system but before user. The tests for
dynamicconfig depend on HGRCPATH being loaded when load_dynamic runs, so let's
move HGRCPATH loading up to load_system.
Reviewed By: quark-zju
Differential Revision: D22712627
fbshipit-source-id: 91175d9d7f85b9392ffea4af815a4facebbfe7c1
Summary:
In a future diff we'll allow an outside caller to pass an Options down
to configparsers::hg::load() so that filters can be applied during loading. Inside
hg::load() we need to use the options multiple times with different values, so
let's make Options clonable.
Reviewed By: quark-zju
Differential Revision: D22712626
fbshipit-source-id: 975145f38d35afe7d4a6c8e87071b0fb0ae74797
Summary:
A future diff will move all dynamic and repo config loading to be in
configparser. As part of this, let's simplify the repo.rs API to not pass
configs around everywhere.
Reviewed By: quark-zju
Differential Revision: D22712628
fbshipit-source-id: 79f23991aa826ce8b4f7430b45d7702efdc6b982
Summary:
Similar to the Python runbgcommand (extutil.py), this is a Rust utility that runs a
detached background process in a cross platform way.
This will be used in a later diff to run dynamicconfig generation in the
background.
Reviewed By: quark-zju
Differential Revision: D22712629
fbshipit-source-id: a317465bf03c96d977a203678e2bef13ce57cc12
Summary:
As part of moving all hg config loading and generation logic into Rust,
let's move the config generation logic from hgcommands and pyconfigparser to
configparser, unifying them at the same time.
Future diffs will move config loading in as well.
Reviewed By: quark-zju
Differential Revision: D22590208
fbshipit-source-id: d1760c404a6a5c57347df30713c20de55cfdb9a4
Summary:
A future diff will unify all config loading into configparser::hg, but
to do so we need dynamicconfig to live in configparser, so it can load
dynamicconfigs. Let's move everything in.
Reviewed By: quark-zju
Differential Revision: D22587237
fbshipit-source-id: 5613094175b6e1597aa113ee3e6d92ce7ec79f6d
Summary:
We had two spots that loaded system and user configs, one in the
pyconfigparser layer, and one in the pure rust config layer. In an upcoming diff
I'd like to move dynamicconfig loading down into the pure rust layer, so let's
unify these.
Reviewed By: quark-zju
Differential Revision: D22585554
fbshipit-source-id: 0cea7801ae1d5a3a3c12b80ee23b37f9e690e2bc
Summary:
In a future diff we'll increase the size of the rotatelog temporarily
during clones. To do so we need it to be configurable.
Reviewed By: quark-zju
Differential Revision: D23089539
fbshipit-source-id: ebfc3beaf3c0fe5b01b87d97c19455b0a24afa72
Summary:
In a future diff we'll increase the size of the rotatelog temporarily
during clones. To do so we need it to be configurable.
Reviewed By: quark-zju
Differential Revision: D23089541
fbshipit-source-id: 5010e417a83a2611283322f1dbb7023f4286f503
Summary:
from_path is an awkward constructor because it doesn't pass any other
information, like a config object. It also requires that the constructor be very
generic across all the stores. Right now it's only needed for pack files, so
let's move it to it's own trait that is limited to pack files.
This will allow us to make the indexedlog store constructors more versatile in a
later diff. Once we get rid of pack files we can delete the StoreFromPath trait
entirely.
Reviewed By: xavierd
Differential Revision: D23089542
fbshipit-source-id: ea6c50853e5d5390a029002ef5d15c74fe41fe69
Summary:
Similar to `changelog.dageval`, but provides extra functions like `public`,
`draft`, `obsolete`, etc.
Reviewed By: sfilipco
Differential Revision: D23036070
fbshipit-source-id: b985f2b338a3dce11bddf53c00c30e4887762676
Summary:
Make it work with closure and `lambda dag: dag.only(...)`.
This is useful when there is a name conflict with a local variable, like:
only = ...
cl.dageval(lambda: only(...)) # only refers to the local variable.
cl.dageval(lambda dag: dag.only(...)) # only refers to the dag operation.
Besides, drop `func_` so they are Py3 compatible.
Reviewed By: sfilipco
Differential Revision: D23036064
fbshipit-source-id: 8d32a34c51b6ba945cda5be313f9d71464f813b7
Summary: If parent_revs gets an out-of-bound rev, it should fail.
Reviewed By: sfilipco
Differential Revision: D23036071
fbshipit-source-id: 7fae0fd5adf07ac3c933a29d7d06289d8d740c60
Summary:
If the text starts with `\0`, the `\0` should be considered as part of the
uncompressed text instead of a separated header.
Reviewed By: sfilipco
Differential Revision: D22970575
fbshipit-source-id: 49e8a1a1ea42a3c4cf153b70f59fd0558dcfcede
Summary:
The parent handling is unsound when there are revs that are skipped. Fix it by
reasoning about commit hashes for parents.
Reviewed By: sfilipco
Differential Revision: D23036078
fbshipit-source-id: 8f710171471025cd48b3bd8f6ea57c68330eb8b8
Summary: Use filesets so that we don't have to worry about exceeding command line
Reviewed By: quark-zju
Differential Revision: D23008329
fbshipit-source-id: bbfef9ecf23b810f4fa6ca8a6de1bbeb9330ae09
Summary: This diff has a unit test to make sure on the fly sparse profiles work. We also allow for more than one sparse profile, and for multiple exclusion sparse profiles.
Reviewed By: quark-zju
Differential Revision: D23004062
fbshipit-source-id: 15f00401084c89a77d8e7bdd8c7eb688e20d131b
Summary:
Windows defaults to checking a revocation server for ssl certs. Inside
our datacenter it can't reach the server and fails. We don't have this on for
any other platforms, so let's disable it.
Reviewed By: sfilipco
Differential Revision: D23121739
fbshipit-source-id: 4d44d2a065bf340a8f74332553deb09a9c61be9b
Summary:
The primary change is in `eden/scm/lib/edenapi/types`:
* Split `DataEntry` into `FileEntry` and `TreeEntry`.
* Split `DataError` into `FileError` and `TreeError`. Remove `Redacted` error variant from `TreeError` and `MaybeHybridManifest` error variant from `FileError`.
* Split `DataRequest`, `DataResponse` into appropriate File and Tree types.
* Refactor `data.rs` into `file.rs` and `tree.rs`.
* Lift `InvalidHgId` error, used by both File and Tree, into `lib.rs`.
* Bugfix: change `MaybeHybridManifest` to be returned only for hash mismatches with empty paths, to match documented behavior.
Most of the remaining changes are straightforward fallout of this split. Notable changes include:
* `eden/scm/lib/edenapi/tools/read_res`: I've split the "data" commands into "file" and "tree", but I've left the identical arguments sharing the same argument structs. These can be refactored later if / when they diverge.
* `eden/scm/lib/types/src/hgid.rs`: Moved `compute_hgid` from `eden/scm/lib/edenapi/types/src/data.rs` to as a new `from_content` constructor on the `HgId` struct.
* `eden/scm/lib/revisionstore/src/datastore.rs`: Split `add_entry` method on `HgIdMutableDeltaStore` trait into `add_file` and `add_tree` methods.
* `eden/scm/lib/revisionstore/src/edenapi`
* `mod.rs`: Split `prefetch` method on `EdenApiStoreKind` into `prefetch_files` and `prefetch_trees`, which are given a default implementation that fails with `unimplemented!`.
* `data.rs`: Replace blanket trait implementations for `EdenApiDataStore<T>` with specific implementations for `EdenApiDataStore<File>` and `EdenApiDataStore<Tree>` which call the appropriate fetch and add functions.
* `data.rs` `test_get_*`: Replace dummy hashes with real hashes. These tests were only passing due to the hash mismatches (incorrectly) being considered `MaybeHybridManifest` errors, and allowed to pass.
Reviewed By: kulshrax
Differential Revision: D22958373
fbshipit-source-id: 788baaad4d9be20686d527f819a7342678740bc3
Summary:
Biggrep recently moved to a sharded system where fbsource is spread
over three shards and can have three different corpus revs. Let's update hg's
biggrep integration to handle this new logic.
We basically just take the earliest corpus commit and diff that against the
users commit. There's room for some incorrectness here, like if the users
commit is based off a commit before the earliest corpus commit then it will
receive changes from the alter corpus commits. Long term biggrep will work on
unifying the various shards to use a consistent corpus rev, but for now this is
probably acceptable.
Reviewed By: quark-zju
Differential Revision: D23057055
fbshipit-source-id: 745cafe772cea44bbe2d8eb1eee70dd317f255fe
Summary:
On EdenFS on Windows (case insensitive), _dirs isn't set, but touching it will
lead to the rest of the dirstate code trying to use it. For instance, removing
a file in the dirstate would try to remove it from _dirs, which would fail as
_dirs is expected to be empty on EdenFS.
Reviewed By: fanzeyi
Differential Revision: D23067897
fbshipit-source-id: 0e0d4aa6457c31d572a2aebc87f0a2fa7215001f
Summary:
The `debugfsync` command calls fsync on newly modified files in svfs.
Right now it only includes locations that we know have constant number
of files.
The fsync logic is put in a separate crate to avoid slow compiles.
Reviewed By: DurhamG
Differential Revision: D22992103
fbshipit-source-id: b5503e498d5216d4ba19701ecd5582387e4f45f5
Summary: This allows callsites to get access to the storage.
Reviewed By: DurhamG
Differential Revision: D22992104
fbshipit-source-id: c72fa313be1468170c9728d3856f822bb6385dc8
Summary:
This makes the main command table cleaner.
I dropped the `indexedlogrepair` command as it cannot rebuild indexes. `hg
doctor` is a better replacement. Some debug commands are renamed so they
no longer have `-` in the command name.
Reviewed By: DurhamG
Differential Revision: D22992107
fbshipit-source-id: f65d74e36fb971e592ad0cc8be9a94e245c39662
Summary:
If every command lives in their module, then we can define the "module" interface:
- run(...): run the command
- doc(): the help text
- name(): command name, with aliases
Then the macro would make command registration look simpler.
This diff changes `status` to use the pattern as an example.
Reviewed By: DurhamG
Differential Revision: D22992109
fbshipit-source-id: eaf589863092ec2eb1f8c24c1c7e425492fe1e3a
Summary:
As the number of commands grows, it starts making sense to move them to
individual files. Let's create a directory for them.
Reviewed By: DurhamG
Differential Revision: D22992108
fbshipit-source-id: a0556be602b832579a8e027342d5b86d9d84d257
Summary:
EdenFS on Windows is a bit weird as ProjectedFS is implemented as a filter
driver that adds reparse point to all the files/directories to get notified of
filesystem operations on them. It then hides these reparse points from the
outside which means that the dwAttributes of a file in EdenFS will not claim
that a reparse point is attached to it. On top of this, newly created
files/directories won't have any reparse points attached to them, until they
start being tracked by EdenFS.
While the first issue can be solved by always querying the reparse tags, I'm
not entirely sure how to solve the second one. That second issue causes
Mercurial to always try to create hardlink in the .hg directory, while it shouldn't.
Reviewed By: DurhamG
Differential Revision: D22937788
fbshipit-source-id: 5d90cd37d40858ed60103ff2d17c2cef16472b38
Summary:
When running `edenfsctl prefetch **/BUCK` with an empty hgcache, EdenFS ends up
asking mercurial for every manifest one by one. Unfortunately, every manifest
fetched also causes the packfile to be flushed to disk, which then leads EdenFS
to rescan the filesystem for the new packfile. Once too many packfiles are
present on disk, Mercurial triggers a repack. Effectively, that means we have a
quadratic complexity both on Mercurial, and on EdenFS's side.
While this has been a long standing issue, we've so far avoided falling into
this complexity for a number of reason. The main one being that the hgcache is
very rarely empty, and thus the quadratic complexity is usually on low number
of files. Users also rarely run a prefetch of all the files for the entire
repo. However, on repositories with long standing branches, the hgcache is
effectively cold and thus any prefetch would trigger the pathological behavior.
To solve this, we take the same approach taken for files: sending the raw
manifest to EdenFS, which will then take care of deserializing it properly.
Reviewed By: DurhamG
Differential Revision: D23035335
fbshipit-source-id: 855e6fb4fabf81c427fad6c9f17d05f95c47e9ae
Summary:
On macOS, it appears that ssh has a ~1% chance of never being able to connect
too the server and just hang. This caused mactest to be completely unhealthy
for a couple of days and a similar hotfix was applied to mitigate the issue.
Since it proved to be working, let's now backport this hotfix in the actual
code.
Reviewed By: DurhamG
Differential Revision: D22953230
fbshipit-source-id: ead7662ea6d0a33efaa5c4044c9391b2835ee421
Summary: Client portion for the commit/revlog_data endpoint that was added to the server.
Reviewed By: kulshrax
Differential Revision: D23065989
fbshipit-source-id: 3115ad2b426daca22472e2106fcd293f3ccd70f3
Summary:
Pyre now has improved support for decorators and descriptors, which makes it
possible for us to add type annotations to `dirstate.py` without needing lots
of `pyre-ignore` comments everywhere. (Previously Pyre could not handle the
`propertycache` decorator, causing it to be confused about the type of
various dirstate members, like `_map`).
Reviewed By: mrkmndz
Differential Revision: D22969757
fbshipit-source-id: 1b54f1edfb56c20c237a34f14a47404d10605240
Summary: Begin adding some initial type annotations for the Rust Python bindings.
Reviewed By: quark-zju
Differential Revision: D22993222
fbshipit-source-id: 2073db93b22f6bb04e30b767594d435c36ddb17f
Summary:
EdenFS may spawn several Mercurial process concurrently and they would all try
to take the wlock at startup time, more often than not, one of these process
would die early due to the tmplock not being present on disk. This is due to
the other Mercurial process removing it, let's have a 10s grace period where
temporary locks aren't removed to avoid this race.
Reviewed By: DurhamG
Differential Revision: D22954997
fbshipit-source-id: ce191265c03a7042d9c6e45db0dc44a688fa204c
Summary:
When doing large clones or checkouts the amount of data we add to an
indexedlog can be many GB. On a laptop we don't have much memory, so let's set a
max memory threshold for the file data/history indexedlogs.
Reviewed By: xavierd
Differential Revision: D23046489
fbshipit-source-id: 43b7686b11fe05e4c074bcb02c475ebf8cf14ab1
Summary: Dump the text into file as it is
Reviewed By: markbt
Differential Revision: D23039839
fbshipit-source-id: 966d6c5e90f020efbb8123704f5c2749596fbab5
Summary:
There are two different magic background syncing that can be enabled. The first
is triggered by commit or any other local changes. The second is triggered by
SCM Daemon by any remote change in this workspace.
I would like to explain it a bit better in `hg cloud status` command.
This will also offer some reassurance to clients.
For example, assume they run `hg cloud disable` command that should disable all background Commit Cloud traffic for some time, so then they can run `hg cloud status` and verify that neither local changes, nor remote changes trigger any commit cloud traffic on this machine.
I also provide full log path to Scm Daemon logs if it is enabled.
Reviewed By: markbt
Differential Revision: D23038954
fbshipit-source-id: c3a5b8f58df729ee3f1c7f15da44ad6e6e0b98f6
Summary:
Matches the `getcommitdata` SSH endpoint.
This is going to be used to remove the requirement that client repostories
need to have all commits locally.
Reviewed By: krallin
Differential Revision: D22979458
fbshipit-source-id: 75d7265daf4e51d3b32d76aeac12207f553f8f61
Summary:
All of these are already valid utf-8 characters, no need to dance to
decode/encode them again.
Reviewed By: DurhamG
Differential Revision: D22978828
fbshipit-source-id: c5f6e25e71cdcaa1c0558d4a1181b667ffe379fb
Summary:
When computing which heads to remove because of bookmark removal,
ignore any heads that are not in the smartlog dag.
Heads might be missing from the smartlog dag if they're not
available on the server.
Reviewed By: quark-zju
Differential Revision: D22980810
fbshipit-source-id: f002eece8567aaf57780f592aaf29a790b8314ce
Summary:
A commit doesn't show up after `hg pull -r` command if it's known locally.
This is a bug that the test demonstrates.
Reviewed By: quark-zju
Differential Revision: D22977182
fbshipit-source-id: 428094568140892fc8a13004f3395371d8b55ebf
