Summary:
Currently we skip metadata when constructing data wirepacks.
We need copy metadata in the data packflies, because the client expects the base text to contain copy metadata since the delta base on the server also contains copy metadata.
It is also needed for future file nodes validation.
Differential Revision: D14851678
fbshipit-source-id: 1a3f79dc2565cdb864bee2400d331ae3a7c3751b
Summary: Added new test to show that applying changes on the renamed file fails because of missing copy-from data in the datapacks.
Reviewed By: StanislavGlebik
Differential Revision: D14851679
fbshipit-source-id: 41fa2008bdfe0a74c97a0a24af58bf6abf4ebd80
Summary:
The Dxxxx revset does two things in parallel:
* walks the changelog in the search of commit associated with Dxxx.
* makes graphql query to resolve Dxxxx to the hash associated with the last
submitted version.
This behaviour is clowny and confusing and should be changed long-term. This is
just a short term fix to prevent it from crashing in cases there's no local
commit associated with Dxxx (or it's very old) and the graphql query comes back
with empty result (because there are no versions for the diff).
Reviewed By: markbt
Differential Revision: D14851313
fbshipit-source-id: 14f7531aa4691007266291a38a016242de563de4
Summary:
The progress indicator for run-tests.py disables terminal line wrapping at the
start of run-tests, and re-enables it at the end. This causes two problems:
* Diff output is not wrapped, so only the start of changed lines are visible.
* Sometimes run-tests.py exits without restoring line wrapping. In this case
the shell session is adversely affected.
Disabled line wrapping is only necessary when outputting the progress, so
restrict it to there.
Reviewed By: quark-zju
Differential Revision: D14834719
fbshipit-source-id: 2c0380e7e1a1d5be195b025c252c80208e83a545
Summary:
Building test objects can be tedious using various of our bottom bytes.
This diff addresses that issue by adding helper functions in a new module
in the types crate.
Handling this case could be improved in rust.
Differential Revision: D14660307
fbshipit-source-id: a866c1f3ede60ba1b87eb17d35817b8a8d7674a4
Summary:
Files with invalid `UTF-8` names are not allowed on OSX anyway and
therefore, this test can be skipped on OSX.
Reviewed By: sfilipco
Differential Revision: D14835026
fbshipit-source-id: 5ad80a4e98ab5088951b1e8209981a3e7fdb03a1
Summary:
When determining the contexts to compare trees against, consider all parents
of the missing nodes that aren't themselves missing.
In the case of:
```
C
/ \
D B
|
A
```
Where A and D are common and B and C are missing, the parents of the roots
are only A, which means D's tree will not be compared against and all of its
trees will be included.
Reviewed By: StanislavGlebik
Differential Revision: D14798681
fbshipit-source-id: c80add907338a3011daf54a2304ea19415b7f9fb
Summary:
When infinitepush is backing up merge commits that have a public commit as one
of their parents, and draft commits as the other, treemanifest doesn't look at
the public parents' trees when building the bundle. This means the bundle may
contain the public parent's trees. If this is a merge of two distinct
histories, it will contain a full tree for the public parent.
Add a test that shows the generated bundle contains too many trees.
Reviewed By: StanislavGlebik
Differential Revision: D14798683
fbshipit-source-id: 1bc1324cef2c2ade74ff4aa59a391cc0fcccf12f
Summary:
The debugbundle implementation for mutation parts was not updated with
the recent changes made to mutation entries. Update it.
Reviewed By: quark-zju
Differential Revision: D14798682
fbshipit-source-id: eb1defa33e8a0eed807360af2c130a225af15030
Summary: Allow users to configure which HTTP client backend to use for the Eden API via the `edenapi.backend` config option. Valid options are `curl` and `hyper`, with `curl` being the default.
Reviewed By: quark-zju
Differential Revision: D14657871
fbshipit-source-id: 7a9972d2380fbbd5ed62d1accae764dc03ca4c29
Summary:
Add a new Eden API client based on libcurl (via the rust-curl crate). This should help us work around issues with Hyper.
This implementation is based on curl's "easy" API, and is intentionally naive. I intend to update it to use curl's "multi" API to send several concurrent HTTP requests per operation in a later diff.
Differential Revision: D14656756
fbshipit-source-id: 1f71074506844104f0f3237023b38317a7f41979
Summary: This is a stop gap measure while we move internal path encodings to UTF-8.
Reviewed By: quark-zju
Differential Revision: D14791761
fbshipit-source-id: e4d5a845aa4f28ed0744cf80021fd1446f3d6129
Summary: `commitctx` takes error as a named parameter shadowing the module import.
Reviewed By: quark-zju
Differential Revision: D14805901
fbshipit-source-id: 359a6d19d6a08b3c65ebf1af8a5e5baaec6711a2
Summary: Updated help text for hg commit
Reviewed By: kulshrax
Differential Revision: D14811336
fbshipit-source-id: dbc26b3409094b1f70aca01b0f9ff89693ce7861
Summary:
In most cases, we won't have more packfiles than what is held in the LRU cache,
and therefore, building a set is unecessary. Testing for the length is a O(1)
operation and is therefore a quick test to verify if more packs needs to be
iterated on.
Reviewed By: quark-zju
Differential Revision: D14802940
fbshipit-source-id: 8b39befb27368d474ab71f3eeac2340d8183e70e
Summary:
When many packfiles are present, and searching for missing data, we will
iterate on the packs twice. We can avoid the second iteration by testing if new
packfiles were detected.
Reviewed By: quark-zju
Differential Revision: D14802942
fbshipit-source-id: 624d9a86e65395af3141ea10d42bcfb8ee18db83
Summary:
Building a set is a O(n) operation, and in the cases where refresh is called
often, it starts showing up in profiles. Instead of rebuilding it everytime, we
can simply update it when adding/removing packs.
Reviewed By: quark-zju
Differential Revision: D14802945
fbshipit-source-id: f8967d09d2d7d0cc0d7400b047a587d536315002
Summary:
The repo.ui.configbool was showing in profiles when running `hg log -p`. Since
its value won't change for the duration of the operation, let's cache it.
Reviewed By: quark-zju
Differential Revision: D14802941
fbshipit-source-id: c1a2a764cb4d4a1c0d45d5118bb1e892c10798a6
Summary:
Failure makes it easier to chain errors, and backtraces. Use it.
There is probably still room for improvement, by chainning errors and avoiding
exposing low-level errors for APIs, and/or provide more context in error
messages. But it should be already much better than before.
Differential Revision: D14759305
fbshipit-source-id: b1d3a8ec959dde575f06533ea9e4cd0757057051
Summary:
Ordering of execution of the racing commands in the dirstate info test can vary
in a way that affects output ordering. Capture the log output and print it
after the commands have completed to ensure the ordering is consistent.
Reviewed By: quark-zju
Differential Revision: D14798950
fbshipit-source-id: e7c7dc63302be4764970b012a50f55bc631ebfc5
Summary:
Change the crash header to just a single line without blaming extensions or
Python. This makes the crash log shorter, and easier to read when there are
crashes in run-tests.py diffs.
Remove `ui.supportcontact` since it's no longer used.
Reviewed By: sfilipco
Differential Revision: D14393982
fbshipit-source-id: 3ede8d3d3d8fd5d125944a9a750350d3ce356b14
Summary: The german test was removed, which caused this test to fail.
Reviewed By: sfilipco
Differential Revision: D14778895
fbshipit-source-id: 35579fb6ec80fe3a435da8e6b93ccd14f1ac276d
Summary:
Add the checkout identifier to the loginfo for commit contexts that take data
from the working copy.
Reviewed By: quark-zju
Differential Revision: D14751601
fbshipit-source-id: 6ddcbf17b970ee24237edf501ae3d7fc8d320193
Summary:
Add `loginfo`, a new field to committable contexts, the contents of which will
be logged when the commit is committed to the repository.
Initially the logging only includes the commit node and author, but more things
will be added later.
Reviewed By: quark-zju
Differential Revision: D14751602
fbshipit-source-id: 2432dfb92dd2489cb8ad3464316a42ebaa79657c
Summary:
When the user checks out a new commit (either by updating to it, by
creating a new commit, or by amending or rebasing the current commit), create a
unique identifier for that checkout.
Log this identifier at the start and end of command processing, and allow other
tools to also query and log the identifier.
This allows both Mercurial commands and other commands that log the identifier
to be correlated with a particular checkout.
Reviewed By: quark-zju
Differential Revision: D14648523
fbshipit-source-id: 2fad79c3010f5fad1a0e180e3d3d6d9c0a7f8e85
Summary: Extract the `makerandomidentifier` function from the `clienttelemetry` extension so that we can use it elsewhere.
Reviewed By: quark-zju
Differential Revision: D14725866
fbshipit-source-id: 0b3ed2e8c9698e892ad0f524f6dde7bf7337f85c
Summary:
One of the drawback of only using packfiles is the cost of the pathological
cases when one pack file is being downloaded per file. This can happen during a
`hg log -p`, or on Eden when running `cat` on lots of not materialized files.
In these cases, mercurial will become slower and slower as the command is
running.
This is due to mercurial having to search each individual packfile to find
whether some data exist locally, or it needs to be fetched from the network.
This search will iterate over each packfile linearly, and on a miss, a
directory scan will be performed to see if a new packfile is present.
Previous patches attempted to solve this by running a repack when the command
finishes, but in the case of `hg log -p`, or when Eden is requesting the
content of some file, the command could be running for a long time.
As a solution, let's repack while the command is running if we detect that too
many pack files are present.
Reviewed By: quark-zju
Differential Revision: D14740566
fbshipit-source-id: 95ba381bd99e5404e352f799e9053a6375abac0a
Summary:
Practically there are many issues with a large max_log_count:
- The directory scan would be slower.
- The index would be slower.
Let's reduce it to u8 range to address the issues. This also makes the
directory name short.
Differential Revision: D14717896
fbshipit-source-id: d39f008abe576991e14d444c37a049a6132df507
Summary:
Some tests added by upcoming diffs were timing out while they don't seem that
expensive. I tracked it down to the use of `fsync` in atomicwrites.
In our case, we don't need `fsync`. `fsync` is useful for making sure the order
of file writes is desired even in case of system crash. For example, making
sure the "primary" log file is written before writing the "meta" file.
That's too expensive (esp. on filesystems like ext4) for our usecase.
Indexedlog is designed to make sure data corruption can be detected, and there
can be a "reasonable" way to recover (ex. by deleting all indexes, scanning
through entries and re-inserting them in a new log), not to fight against OS
crashes.
`cargo bench` change on a btrfs filesystem:
Before:
index flush 42.570 ms
log flush 7.712 ms
After:
index flush 36.485 ms
log flush 1.609 ms
Differential Revision: D14759304
fbshipit-source-id: 66b95d10040cf1480367b767811dfabee5e27ffe
Summary: Used `cargo fix --edition`. Removed some `mut`s according to rustc warnings.
Differential Revision: D14718308
fbshipit-source-id: 94e3c3f8e47143ede767fe883fdb5e9602b12854
Summary:
Mercurial recently started to generate empty pack files (`0x01`). This will cause this check to fail:
diffusion/FBS/browse/master/fbcode/scm/hg/lib/cdatapack/cdatapack.c;2c4197d003ed906dd8eaf70fbb04aa53440ce681$314-319
This will subsequently result as a double-free error between these two:
**In `error_cleanup`**
diffusion/FBS/browse/master/fbcode/scm/hg/lib/cdatapack/cdatapack.c;2c4197d003ed906dd8eaf70fbb04aa53440ce681$387-389
**In `close_datapack`**
diffusion/FBS/browse/master/fbcode/scm/hg/lib/cdatapack/cdatapack.c;2c4197d003ed906dd8eaf70fbb04aa53440ce681$401
This diff will fix this bug.
Differential Revision: D14759374
fbshipit-source-id: 06f192513a935740c2142b5a2baac87a28903496
Summary:
Zeyi realized that empty packfiles were problematic for the cdatapack code.
While its code should be fixed, having empty packfiles lying around is
unecessary anyway, so let's not write them.
Reviewed By: fanzeyi
Differential Revision: D14760942
fbshipit-source-id: a128eedaf79a6388a3c7142399715bb4eb96a2ae
Summary: We're intentionally exporting Python classes with names that match Mercurials convention of `allonewordlowercase`, so we should turn off this lint.
Differential Revision: D14760996
fbshipit-source-id: cf79e8ebb6a01bd7bcfb062754cb034d94ac493b
Summary:
While datapack repack is fairly inexpensive in memory and mostly limited to the
number of entries in its index, a historypack repack needs to keep both the
data and the index in memory. It appears that the overhead of doing so is a big
factor in repack taking a lot of memory as a resulting 100MB histpack would use
about 1.2GB of RAM. Extrapolating the numbers, a resulting 4GB histpack would
need 48GB, which is enough to put a devserver in a swapping state, and worse
for laptops. Limiting the historypack size to 400MB should cap the RAM usage to
a bit under 5GB.
Reviewed By: kulshrax
Differential Revision: D14757839
fbshipit-source-id: b08bf01bddad01f1cae9cc67d4bd3d637c0bf0db
Summary:
The test has been failing since D14627242. Since we're not updating our
i18n, let's just remove it.
Reviewed By: DurhamG
Differential Revision: D14761412
fbshipit-source-id: 8fa34121bffe159169a91a629996ae51036a7620
Summary: Now that the Hyper client is contained in a single module, its fields do not need to be crate-public.
Reviewed By: sfilipco
Differential Revision: D14733274
fbshipit-source-id: aa5c2f4fd9fdf6e686da1fed6300e8cf8f7e5dbc
Summary:
The phabricator team is trying to eliminate `DifferentialRevision` in favor of `PhabricatorDiff`
See context here
https://our.intern.facebook.com/intern/wiki/Dev_Tools/Phabricator/GraphQLObjectRename/
I'm using aliases to reduce the number of changes needed (the object from the graphql query will have the same keys)
Reviewed By: quark-zju
Differential Revision: D14703867
fbshipit-source-id: 32204315dc154db7cbff68ee286579c0200f57b2
Summary:
Loosefiles are quite slow to repack, and this is significantly affecting
laptops as the repack can potentially run for hours when a large amount of
loosefiles are present.
One of the reason for that is that an incremental repack really isn't
incremental for loosefiles as all of them are repacked at the same time.
Instead let's repack only one of the top-level directory at a time.
Reviewed By: quark-zju
Differential Revision: D14690348
fbshipit-source-id: ffba49840302ae0d99e32db410647e83e213fe64
Summary:
Currently, if you lookup a globalrev which does not exist in the
repository, it leads to a linear scan of the entire changelog even while using
the globalrev based fast commit lookup. This commit speeds up the same by
relying on the indexing information.
Differential Revision: D14726937
fbshipit-source-id: 076ad8608aa656075dcb2c1287502ffe93bed851
Summary:
This is just pending feedback from D14694620 which makes the class
interface more cleaner.
Reviewed By: quark-zju
Differential Revision: D14718748
fbshipit-source-id: 032fc25d7e5922aac5f4195eb007bd009822cb86
Summary:
Rather than having a `Builder` struct that knows how to build just one kind of Eden API client, let's have a common `Config` type that can be potentially passed to the constructor of several different client implementations.
This will allow the same config code to be re-used across different client types, as seen later in this stack.
Differential Revision: D14656757
fbshipit-source-id: 883ffd2dc0302ebe08960f079c113e2d0da2d2ca
Summary: Move the implementation of the `EdenApi` trait for the current Hyper-backed Eden API client to the same module as the client, so that we can have several side-by-side implementations, each contained in their own respective module.
Differential Revision: D14656758
fbshipit-source-id: a0d5fa36ec346c40466df559ccc900b14a7c542f
