Summary: Allow users to configure which HTTP client backend to use for the Eden API via the `edenapi.backend` config option. Valid options are `curl` and `hyper`, with `curl` being the default.
Reviewed By: quark-zju
Differential Revision: D14657871
fbshipit-source-id: 7a9972d2380fbbd5ed62d1accae764dc03ca4c29
Summary:
Add a new Eden API client based on libcurl (via the rust-curl crate). This should help us work around issues with Hyper.
This implementation is based on curl's "easy" API, and is intentionally naive. I intend to update it to use curl's "multi" API to send several concurrent HTTP requests per operation in a later diff.
Differential Revision: D14656756
fbshipit-source-id: 1f71074506844104f0f3237023b38317a7f41979
Summary: This is a stop gap measure while we move internal path encodings to UTF-8.
Reviewed By: quark-zju
Differential Revision: D14791761
fbshipit-source-id: e4d5a845aa4f28ed0744cf80021fd1446f3d6129
Summary: `commitctx` takes error as a named parameter shadowing the module import.
Reviewed By: quark-zju
Differential Revision: D14805901
fbshipit-source-id: 359a6d19d6a08b3c65ebf1af8a5e5baaec6711a2
Summary: Updated help text for hg commit
Reviewed By: kulshrax
Differential Revision: D14811336
fbshipit-source-id: dbc26b3409094b1f70aca01b0f9ff89693ce7861
Summary:
In most cases, we won't have more packfiles than what is held in the LRU cache,
and therefore, building a set is unecessary. Testing for the length is a O(1)
operation and is therefore a quick test to verify if more packs needs to be
iterated on.
Reviewed By: quark-zju
Differential Revision: D14802940
fbshipit-source-id: 8b39befb27368d474ab71f3eeac2340d8183e70e
Summary:
When many packfiles are present, and searching for missing data, we will
iterate on the packs twice. We can avoid the second iteration by testing if new
packfiles were detected.
Reviewed By: quark-zju
Differential Revision: D14802942
fbshipit-source-id: 624d9a86e65395af3141ea10d42bcfb8ee18db83
Summary:
Building a set is a O(n) operation, and in the cases where refresh is called
often, it starts showing up in profiles. Instead of rebuilding it everytime, we
can simply update it when adding/removing packs.
Reviewed By: quark-zju
Differential Revision: D14802945
fbshipit-source-id: f8967d09d2d7d0cc0d7400b047a587d536315002
Summary:
The repo.ui.configbool was showing in profiles when running `hg log -p`. Since
its value won't change for the duration of the operation, let's cache it.
Reviewed By: quark-zju
Differential Revision: D14802941
fbshipit-source-id: c1a2a764cb4d4a1c0d45d5118bb1e892c10798a6
Summary:
Failure makes it easier to chain errors, and backtraces. Use it.
There is probably still room for improvement, by chainning errors and avoiding
exposing low-level errors for APIs, and/or provide more context in error
messages. But it should be already much better than before.
Differential Revision: D14759305
fbshipit-source-id: b1d3a8ec959dde575f06533ea9e4cd0757057051
Summary:
Ordering of execution of the racing commands in the dirstate info test can vary
in a way that affects output ordering. Capture the log output and print it
after the commands have completed to ensure the ordering is consistent.
Reviewed By: quark-zju
Differential Revision: D14798950
fbshipit-source-id: e7c7dc63302be4764970b012a50f55bc631ebfc5
Summary:
Change the crash header to just a single line without blaming extensions or
Python. This makes the crash log shorter, and easier to read when there are
crashes in run-tests.py diffs.
Remove `ui.supportcontact` since it's no longer used.
Reviewed By: sfilipco
Differential Revision: D14393982
fbshipit-source-id: 3ede8d3d3d8fd5d125944a9a750350d3ce356b14
Summary: The german test was removed, which caused this test to fail.
Reviewed By: sfilipco
Differential Revision: D14778895
fbshipit-source-id: 35579fb6ec80fe3a435da8e6b93ccd14f1ac276d
Summary:
Add the checkout identifier to the loginfo for commit contexts that take data
from the working copy.
Reviewed By: quark-zju
Differential Revision: D14751601
fbshipit-source-id: 6ddcbf17b970ee24237edf501ae3d7fc8d320193
Summary:
Add `loginfo`, a new field to committable contexts, the contents of which will
be logged when the commit is committed to the repository.
Initially the logging only includes the commit node and author, but more things
will be added later.
Reviewed By: quark-zju
Differential Revision: D14751602
fbshipit-source-id: 2432dfb92dd2489cb8ad3464316a42ebaa79657c
Summary:
When the user checks out a new commit (either by updating to it, by
creating a new commit, or by amending or rebasing the current commit), create a
unique identifier for that checkout.
Log this identifier at the start and end of command processing, and allow other
tools to also query and log the identifier.
This allows both Mercurial commands and other commands that log the identifier
to be correlated with a particular checkout.
Reviewed By: quark-zju
Differential Revision: D14648523
fbshipit-source-id: 2fad79c3010f5fad1a0e180e3d3d6d9c0a7f8e85
Summary: Extract the `makerandomidentifier` function from the `clienttelemetry` extension so that we can use it elsewhere.
Reviewed By: quark-zju
Differential Revision: D14725866
fbshipit-source-id: 0b3ed2e8c9698e892ad0f524f6dde7bf7337f85c
Summary:
One of the drawback of only using packfiles is the cost of the pathological
cases when one pack file is being downloaded per file. This can happen during a
`hg log -p`, or on Eden when running `cat` on lots of not materialized files.
In these cases, mercurial will become slower and slower as the command is
running.
This is due to mercurial having to search each individual packfile to find
whether some data exist locally, or it needs to be fetched from the network.
This search will iterate over each packfile linearly, and on a miss, a
directory scan will be performed to see if a new packfile is present.
Previous patches attempted to solve this by running a repack when the command
finishes, but in the case of `hg log -p`, or when Eden is requesting the
content of some file, the command could be running for a long time.
As a solution, let's repack while the command is running if we detect that too
many pack files are present.
Reviewed By: quark-zju
Differential Revision: D14740566
fbshipit-source-id: 95ba381bd99e5404e352f799e9053a6375abac0a
Summary:
Practically there are many issues with a large max_log_count:
- The directory scan would be slower.
- The index would be slower.
Let's reduce it to u8 range to address the issues. This also makes the
directory name short.
Differential Revision: D14717896
fbshipit-source-id: d39f008abe576991e14d444c37a049a6132df507
Summary:
Some tests added by upcoming diffs were timing out while they don't seem that
expensive. I tracked it down to the use of `fsync` in atomicwrites.
In our case, we don't need `fsync`. `fsync` is useful for making sure the order
of file writes is desired even in case of system crash. For example, making
sure the "primary" log file is written before writing the "meta" file.
That's too expensive (esp. on filesystems like ext4) for our usecase.
Indexedlog is designed to make sure data corruption can be detected, and there
can be a "reasonable" way to recover (ex. by deleting all indexes, scanning
through entries and re-inserting them in a new log), not to fight against OS
crashes.
`cargo bench` change on a btrfs filesystem:
Before:
index flush 42.570 ms
log flush 7.712 ms
After:
index flush 36.485 ms
log flush 1.609 ms
Differential Revision: D14759304
fbshipit-source-id: 66b95d10040cf1480367b767811dfabee5e27ffe
Summary: Used `cargo fix --edition`. Removed some `mut`s according to rustc warnings.
Differential Revision: D14718308
fbshipit-source-id: 94e3c3f8e47143ede767fe883fdb5e9602b12854
Summary:
Mercurial recently started to generate empty pack files (`0x01`). This will cause this check to fail:
diffusion/FBS/browse/master/fbcode/scm/hg/lib/cdatapack/cdatapack.c;2c4197d003ed906dd8eaf70fbb04aa53440ce681$314-319
This will subsequently result as a double-free error between these two:
**In `error_cleanup`**
diffusion/FBS/browse/master/fbcode/scm/hg/lib/cdatapack/cdatapack.c;2c4197d003ed906dd8eaf70fbb04aa53440ce681$387-389
**In `close_datapack`**
diffusion/FBS/browse/master/fbcode/scm/hg/lib/cdatapack/cdatapack.c;2c4197d003ed906dd8eaf70fbb04aa53440ce681$401
This diff will fix this bug.
Differential Revision: D14759374
fbshipit-source-id: 06f192513a935740c2142b5a2baac87a28903496
Summary:
Zeyi realized that empty packfiles were problematic for the cdatapack code.
While its code should be fixed, having empty packfiles lying around is
unecessary anyway, so let's not write them.
Reviewed By: fanzeyi
Differential Revision: D14760942
fbshipit-source-id: a128eedaf79a6388a3c7142399715bb4eb96a2ae
Summary: We're intentionally exporting Python classes with names that match Mercurials convention of `allonewordlowercase`, so we should turn off this lint.
Differential Revision: D14760996
fbshipit-source-id: cf79e8ebb6a01bd7bcfb062754cb034d94ac493b
Summary:
While datapack repack is fairly inexpensive in memory and mostly limited to the
number of entries in its index, a historypack repack needs to keep both the
data and the index in memory. It appears that the overhead of doing so is a big
factor in repack taking a lot of memory as a resulting 100MB histpack would use
about 1.2GB of RAM. Extrapolating the numbers, a resulting 4GB histpack would
need 48GB, which is enough to put a devserver in a swapping state, and worse
for laptops. Limiting the historypack size to 400MB should cap the RAM usage to
a bit under 5GB.
Reviewed By: kulshrax
Differential Revision: D14757839
fbshipit-source-id: b08bf01bddad01f1cae9cc67d4bd3d637c0bf0db
Summary:
The test has been failing since D14627242. Since we're not updating our
i18n, let's just remove it.
Reviewed By: DurhamG
Differential Revision: D14761412
fbshipit-source-id: 8fa34121bffe159169a91a629996ae51036a7620
Summary: Now that the Hyper client is contained in a single module, its fields do not need to be crate-public.
Reviewed By: sfilipco
Differential Revision: D14733274
fbshipit-source-id: aa5c2f4fd9fdf6e686da1fed6300e8cf8f7e5dbc
Summary:
The phabricator team is trying to eliminate `DifferentialRevision` in favor of `PhabricatorDiff`
See context here
https://our.intern.facebook.com/intern/wiki/Dev_Tools/Phabricator/GraphQLObjectRename/
I'm using aliases to reduce the number of changes needed (the object from the graphql query will have the same keys)
Reviewed By: quark-zju
Differential Revision: D14703867
fbshipit-source-id: 32204315dc154db7cbff68ee286579c0200f57b2
Summary:
Loosefiles are quite slow to repack, and this is significantly affecting
laptops as the repack can potentially run for hours when a large amount of
loosefiles are present.
One of the reason for that is that an incremental repack really isn't
incremental for loosefiles as all of them are repacked at the same time.
Instead let's repack only one of the top-level directory at a time.
Reviewed By: quark-zju
Differential Revision: D14690348
fbshipit-source-id: ffba49840302ae0d99e32db410647e83e213fe64
Summary:
Currently, if you lookup a globalrev which does not exist in the
repository, it leads to a linear scan of the entire changelog even while using
the globalrev based fast commit lookup. This commit speeds up the same by
relying on the indexing information.
Differential Revision: D14726937
fbshipit-source-id: 076ad8608aa656075dcb2c1287502ffe93bed851
Summary:
This is just pending feedback from D14694620 which makes the class
interface more cleaner.
Reviewed By: quark-zju
Differential Revision: D14718748
fbshipit-source-id: 032fc25d7e5922aac5f4195eb007bd009822cb86
Summary:
Rather than having a `Builder` struct that knows how to build just one kind of Eden API client, let's have a common `Config` type that can be potentially passed to the constructor of several different client implementations.
This will allow the same config code to be re-used across different client types, as seen later in this stack.
Differential Revision: D14656757
fbshipit-source-id: 883ffd2dc0302ebe08960f079c113e2d0da2d2ca
Summary: Move the implementation of the `EdenApi` trait for the current Hyper-backed Eden API client to the same module as the client, so that we can have several side-by-side implementations, each contained in their own respective module.
Differential Revision: D14656758
fbshipit-source-id: a0d5fa36ec346c40466df559ccc900b14a7c542f
Summary:
When the sql_repo_lock.py hook is run, the sql connection won't have been
established yet. Let's open a connection and clean it up afterwards.
Reviewed By: quark-zju
Differential Revision: D14708851
fbshipit-source-id: f20b6dacdcb12cee839163325164d2e75816100c
Summary:
The type for globalrev during lookup should be an integer instead of a
string after D14694623.
Reviewed By: quark-zju
Differential Revision: D14718749
fbshipit-source-id: eddc49ea0faa61df6ad435db2192fba4f61b0075
Summary:
The Rust stdlib uses this pattern. This is done by:
sed -i 's/\[\([A-Z][a-zA-Z:]*\)\]/[`\1`]/g' *.rs
Unfortunately it seems only rustdoc nightly can linkify things correctly.
More context: https://github.com/rust-lang/rust/issues/43466
Reviewed By: kulshrax
Differential Revision: D14689887
fbshipit-source-id: ba2b5968bdaad06f39dc43962430906ee80692fd
Summary:
rotate::OpenOptions is a superset of log::OpenOptions. Change the code to reuse
logic in log::OpenOptions as much as possible.
Reviewed By: kulshrax
Differential Revision: D14689888
fbshipit-source-id: a6958723c49f9d41b03100f01283a8c3fb37a1ab
Summary:
The motivation of this is, LogRotate might copy dirty (non-flushed) entries
from one Log to another, and it cannot preserve the checksum type for those
entries. There are 2 solutions:
- Make `iter_dirty` return checksum type.
- Make checksum type known by Log directly.
The second choice provides a simpler public API. `append_advanced` can be
removed, then `iter_dirty` is still consistent with `iter`. Therefore this
change.
Differential Revision: D14688174
fbshipit-source-id: 09e07d64c886a5ce9bc48dce8e29d036af1c0381
Summary: A later diff adds another field to OpenOptions that Log needs access to.
Differential Revision: D14688171
fbshipit-source-id: 33170a2b74639ba0fd8a9c86207d840fb6427580
Summary: This is the final piece to make space usage bounded.
Differential Revision: D14688179
fbshipit-source-id: a6e0058b9022789fcf036c4427d29eab19144b53
Summary:
If "latest" pointer has changed, we should write to the new "latest" Log,
instead of the stale one.
Differential Revision: D14688180
fbshipit-source-id: eab8df8ddb8f311e472361ecc2b1bc4155f2aba4
Summary:
This API iterates entries that are in-memory only. It is useful to extract
entries and store them elsewhere.
Differential Revision: D14688178
fbshipit-source-id: 6ace51d859ba6886aeb94689f6c45162b9c6958e
Summary: Implement the basic flush logic. Missing bits are listed as TODO items.
Differential Revision: D14688177
fbshipit-source-id: 3613009ec2c216398af6eaff44487a20ceeb97ef
Summary:
The file size will be used to decide whether the Log needs "rotate" in upcoming
changes.
Reviewed By: kulshrax
Differential Revision: D14688169
fbshipit-source-id: b273abcc870b96650d2c76e6e742a3141ce48f13
Summary:
These methods just delegate to `Log` structures. Unfortunately, the key has to
be copied so it can be used by the iterator to query remaining logs.
Differential Revision: D14688172
fbshipit-source-id: fd581f7256031a0622ec0533c84daaab89f9bb82
