Summary:
Python's `next()` can raise the `StopIteration` exception, unlike Rust's `next`
which just returns `None` instead.
Fix it by providing a default value `None`.
Reviewed By: singhsrb
Differential Revision: D15008773
fbshipit-source-id: df885c63b8130ceac38f86c89f2547dde2d519ba
Summary:
The warning isn't that useful, and can actually cause more harm than good, as running `hg prefetch -r .`
can download gigabytes of unnecessary data to the hgcache.
Reviewed By: quark-zju
Differential Revision: D14999458
fbshipit-source-id: b0ff2c2ad0e441622066fac10a5efafe8de588db
Summary:
The `if dest` test fails with infinitepush changing `dest` to non-None.
Fix it by also checking if `dest` matches `default-push` (or `default`).
Reported by: fryfrog
Reviewed By: markbt
Differential Revision: D14965995
fbshipit-source-id: 91e68368eda4457d06059387307a9572bc6d2906
Summary:
The way this command was implemented before this change would collapse
existing case collisions into a single value of the `lowerdirlist` dict.
The value that was chosen would be dependent on the traversal ordering.
If this value would be equivalent to the newly-introduced file, we would
miss the collision.
Reviewed By: markbt
Differential Revision: D14971701
fbshipit-source-id: d352e96e512154d92fe6bc49dd76aec63b954fef
Summary:
Change the fastlog approach to patch the "follow" revset function instead of
"getlogrevs". This makes it more general purposed so it can speed up commands
like `hg log -r 'reverse(follow("dir"))'`.
Note: This will regress `log dir` performance, which will be fixed in the next
diff.
Reviewed By: sfilipco
Differential Revision: D14764074
fbshipit-source-id: c2a4c8e91359d971e6ea668e5ff1f0ab6eb0534c
Summary:
The "hg log" implementation chooses to not use the "follow" revset when logging
a directory, but use "_matchfiles" instead. In an upcoming change, we'd like
"follow" to handle directories so fastlog only needs to patch the "follow"
revset.
The "follow" revset can take a directory pattern just fine. The problem is
"follow" will follow *every* file inside the specified directory, which is
quite expensive.
For now, I just moved the "_matchfiles" fallback path to "follow" so when it
detects there are too many files to follow, it will switch to "_matchfiles"
directly.
In theory, tree manfiest repos would have "tree manifest" infomation that speed
up "follow" on a directory. But that's a bigger change, and it's probably very
slow in our setup because our trees are lazy.
This changes some behaviors subtly, as reflected in tests:
- `-f path` can use DAG range instead of rev range, which is a good thing as
rev range does not make much sense to end-users. This removes a "BUG" in
test-commit-amend.t
- `-f dir` can follow renames if the directory contains just a few files.
This looks like a good thing, and is reflected in `test-log.t`.
Reviewed By: sfilipco
Differential Revision: D14863134
fbshipit-source-id: 99ddff46d43f63ce03dc7bf005e3ac1cb9b39d03
Summary:
With upcoming changes, I noticed `limit(reverse(:.) & follow("path:fbcode/scm/hg"), 10)`
is much slower than `limit(reverse(::.) & follow("path:fbcode/scm/hg"), 10)`. I tracked
it down to the fact that spanset introduces a lot of unnecessary checks. Optimize it
by avoid using spanset in this case.
The revset pattern is used by `hg log`, the `reverse(:.)` part is to "define" the
order. Perhaps we sould replace it with `reverse(::.)`. But that's a BC change that
might have some unwanted side effects.
Reviewed By: sfilipco
Differential Revision: D14863135
fbshipit-source-id: 6ba8a02b58e1109bdf8370f03965a3b302cba6c0
Summary: Removing this function in favor of using Key.path
Reviewed By: quark-zju
Differential Revision: D14945331
fbshipit-source-id: 6b6bb70375629edf37b2b04a86545f18e15b33b4
Summary:
When the option is set, the gets will be stored in the indexed log instead of
in a packfile. With the Python code now able to read from it, this will allow
us to get more real world testing.
Reviewed By: quark-zju
Differential Revision: D14895509
fbshipit-source-id: d73d49a028f7af199b7a0873551d7b18b047e50c
Summary: The super-smartlog shows "local changes" even when you have some unattached draft changes associated with the diff. This diff changes that.
Reviewed By: quark-zju
Differential Revision: D14936057
fbshipit-source-id: 32204ef9c5fa862923a165624913ecb344ed4a73
Summary:
Context: for Mononoke sync job we are considering applying a few pushrebase
bundles under the same transaction to make sync job faster. More details -
https://fburl.com/y4we86hc
Doing two pushrebases under the same transaction seems to fail for just one
reason - `hookargs` is None on the second bundle. The reason is this line -
https://fburl.com/y4we86hc, after gettransaction() is called
bundleoperation.hookargs is set to None, but transaction.hookargs is set to
current hookargs value.
In this diff pushrebase is changed to check both bundleoperation.hookargs and
transaction.hookargs
Reviewed By: quark-zju
Differential Revision: D14930970
fbshipit-source-id: e4a4dd1c85b1fdca2699dd431040d65f2642ec8a
Summary:
`ui.log` expects to be called with valid format arguments. If the arguments
are not a valid format string, or the number of arguments doesn't match the
number of format placeholders, formatting will fail.
In this case, catch the exception and fail gracefully. Don't even bother
formatting if there is exactly one argument.
The `blackbox` extension already does this, so extend to the `sampling`
extension.
Also fix the place where `perftrace` calls `ui.log` with a string that might
contain formatting placeholders.
Reviewed By: quark-zju
Differential Revision: D14938952
fbshipit-source-id: 1d9802308dba925109c018124d51273c348526b4
Summary: `log` itself does the interpolation, so don't do it in advance.
Reviewed By: quark-zju
Differential Revision: D14941862
fbshipit-source-id: 6885202cfbff4ce0310d68bb1676369c3ef64c53
Summary:
Looking at the Hgerrors scuba table, I noticed that a lot of the sandcastle
machines had repack failures due to "No such file or directory". I'm suspecting
that's due to not having a local store to repack, and therefore listing of
files to repack would fail. Let's verify that the directory is present before
repacking to avoid this issue.
Reviewed By: quark-zju
Differential Revision: D14906503
fbshipit-source-id: 98fbe57310511df4fc9856bf71f836adefb3d855
Summary:
This functionality is generic enough to be inside the common module so
that it can be shared easily. Upcoming commits will make use of this
functionality by importing from the common module.
Reviewed By: markbt
Differential Revision: D14919848
fbshipit-source-id: b3b7eef9eb0929334cba7ce869e7c9377c265da2
Summary:
This functionality is generic enough to be inside the common module so
that it can be shared easily. Upcoming commits will make use of this
functionality by importing from the common module.
Reviewed By: markbt
Differential Revision: D14919849
fbshipit-source-id: 74c6ad5158bd196bb062e2ec7b04ca121b5d2c3a
Summary:
This diff adds args escaping using single quote symbol before sending them to hooks.
Before all arguments where joined by space symbol " " which was producing incorrect result when argument itself contains space symbol.
Reviewed By: markbt
Differential Revision: D14799188
fbshipit-source-id: df5a4324d138515a4b881df96f2991de03df7a5b
Summary:
Add `shelvename` template keyword, which expands to the name of the shelve for
commits that contain shelves.
Reviewed By: farnz
Differential Revision: D14932985
fbshipit-source-id: cddebd2dbc6454f7c61ed296f37822179da8a2de
Summary:
We should update the builder for Key to take a repo path. We could build
the key directly using the default struct constructor but representing
the two constructors as functions is more clear.
Reviewed By: quark-zju
Differential Revision: D14877543
fbshipit-source-id: 328906521cdbad535e28df22fea82f21e8b5410a
Summary:
This function is difficult to justify in the context of the Rust borrow checker.
The primary concern for this pattern is preventing mutation when the object is
passed around.
We can always add the function back if it has to more than just return the
underlying value.
Reviewed By: quark-zju
Differential Revision: D14877545
fbshipit-source-id: acdd796e1bee5445c1bce5ce0ceb41a7334e4966
Summary:
It makes this method 25-30% faster (shaves off 250-300 ms).
Also it counts number of fetched rows correctly - fetchall method was
overriden, but looks like __iter__ method wasn't
Reviewed By: ikostia
Differential Revision: D14915472
fbshipit-source-id: 313695c1a83d05dac2fc801792226b6b64539cb5
Summary: This extension has been refactored many times, resulting in some code and config options that are (as far as I can tell) completely unused. Let's remove them.
Reviewed By: singhsrb
Differential Revision: D14920751
fbshipit-source-id: 8d716a1d6144ba51cdb2750f6cfc6d6262b3b8a3
Summary:
While the Rust code can read/write content out of an indexedlog, the Python
code cannot. For now, all the writes will be done in Rust, and the Python code
will only be able to read from it.
Reviewed By: quark-zju
Differential Revision: D14894330
fbshipit-source-id: 5c1698d31412bc93e93dabb93be106a2ef17d184
Summary:
While not a correctness issue, taking and releasing the repack lock multiple
times per repack makes `hg debugwaitonrepack` unreliable, which in turns makes
test flakey.
Reviewed By: quark-zju
Differential Revision: D14877896
fbshipit-source-id: 682b649f388d19fd51bcf8dd205ac96944039e86
Summary:
We currently send perf traces to the blackbox, but we also want to log
them to our metrics. This configures hg to log the perf trace as a sample. Later
diffs will configure our wrapper to route these samples to manifold.
Reviewed By: mitrandir77
Differential Revision: D14881308
fbshipit-source-id: 07690f9af9fe14279a9ebf5089548c5880cba45d
Summary:
Autorels attempts to detect the scenario where `P -> Q` and `X -> Y` are
being added, and there already exists a `P -> ... -> X` relationship.
In this case it will create a `Q -> Y` "copy" marker to express the fact
that `Q` should be copied.
However, this also triggers in the case where `Q == Y`, creating a revive
marker for `Q`.
Normally this is benign, as Q is probably visible anyway, however when there
are two commits associated with a diff that has been landed, pullcreatemarkers
can create two markers: `P -> L` and `X -> L`. Since P and X are for the same
diff, there probably exists a `P -> ... -> X` relationship, and so autorels
attempts to make an `L -> L` marker. This fails because L is public.
Differential Revision: D14891063
fbshipit-source-id: 3f076a003508dd7b7d17e3eb7cdaeb8ac09e6b15
Summary:
With explicit visibility tracking, some commits may be hidden but not obsolete.
Pullcreatemarkers should not include these hidden commits when it is considering
what to link to landed commits.
Differential Revision: D14891062
fbshipit-source-id: 542363c979aca85d6de2e2161c00d91f67f0add3
Summary:
The `subonly` option in the command decorator indicates that a command must be
called with a subcommand. Document this in the decorator documentation.
Differential Revision: D14876782
fbshipit-source-id: d121f94437e47cad2a7f3abe0faac300792c1c76
Summary:
Add support for explicit visibility tracking in commit cloud sync.
This means commit cloud reads the visibleheads and syncs these with the commit
cloud heads directly, removing the source of problems where obsmarkers disagree
on different hosts.
Commit cloud requires that the ordering of heads is maintained to get stable
ordering of new commits. Update the visibleheads tracking to maintain
ordering, rather than using sets.
Finally, the calculation of the replacement node was slightly off. This was
revealed in the new test case that is being added, so it is also fixed.
Differential Revision: D14876266
fbshipit-source-id: fe5b6bffd196d3bd74e7582e29484969495eac8e
Summary:
The computation of whether a commit is obsolete or not can be improved.
We can cache which commits are known to not be obsolete.
We can also have a cache for each filter type so that we only need to compute
obsolete nodes that match the filter.
Finally, when we need to compute all obsolete commits, we can start by looking
for commits which are made obsolete by only their closest successors, and then
filling back obsolescence to the predecessors of these obsolete commits.
Reviewed By: DurhamG
Differential Revision: D14858655
fbshipit-source-id: 1d03e214ad878ecb6ae548f80373702e2a184146
Summary: The `absorb` command should also record mutation information when it modifies commits.
Reviewed By: DurhamG
Differential Revision: D14871232
fbshipit-source-id: 46bc95b7f5781f0b5f5e057a34c755fcfe653f7e
Summary:
With explicit visibility tracking, the hiddenoverride feature of the amend
extension is no long necessary. When upgrading to explicit visibility,
include the pinned visible commits, however after that point, hiddenoverride
should have no effect.
Users can permanently reveal obsoleted commits using `hg unhide`.
Reviewed By: DurhamG
Differential Revision: D14851269
fbshipit-source-id: b516fa97ed1545d39b51baaf9c574cb73a571645
Summary:
Rather than implicitly upgrading or downgrading a repo based on the
`visibility.tracking` config option, add a new `hg debugvisibility` command to
do this explicitly.
Reviewed By: DurhamG
Differential Revision: D14871231
fbshipit-source-id: 73f4648408b3eca9ac12bd77e54d2d37ee342069
Summary:
When run with `--hidden`, the calculation for `backupheads()` may include extra
commits (ones that are hidden but not obsolete). This can cause some of the old
nodes to apper with "Not backed up" annotations. They should be excluded, too.
Reviewed By: DurhamG
Differential Revision: D14851035
fbshipit-source-id: f176b50a7ac62dc493a5c4df1e0c72532c43b680
Summary:
The pullcreatemarkers extension creates fake markers for landed commits by comparing
the differential revision of draft commits with those of landed commits.
In the future, mutation records will handle lands correctly, however until that stage,
replicate the pullcreatemarkers behaviour.
Reviewed By: DurhamG
Differential Revision: D14834746
fbshipit-source-id: e196ee28645270ccab1775ea896f0e453740c03e
Summary: This will help us debugging of what's going on on the server side.
Reviewed By: StanislavGlebik
Differential Revision: D14876167
fbshipit-source-id: 17aaa6dca9d5b5d7bbf69be46b58e454fe9f3fc3
Summary:
Record the commit predecessors and the mutation operation in the commit loginfo
so that it can be logged to telemetry.
Reviewed By: quark-zju
Differential Revision: D14798032
fbshipit-source-id: 9c4ac1a4df3c91087c776d1f8e5fca94713b0390
Summary:
Most of the extern crate are now removed. The cpython one is kept around as it
heavily depends on internal macros that would need to be manually imported.
Reviewed By: quark-zju
Differential Revision: D14864995
fbshipit-source-id: 05405ac7310e4dca65daf230cc8f574da32ed4c9
Summary: Make the batch size of data and history requests independently configurable, since data responses are typically much larger than history responses (since the former contains actual file data whereas the latter is only metadata).
Differential Revision: D14859686
fbshipit-source-id: c87c31f3e6611a55ae712e7f0ed9bb392d31a579
Summary: Use the curl multi interface to fetch multiple batches of files or history entries concurrently.
Differential Revision: D14718547
fbshipit-source-id: c5a740c7e9106b719e825540f8182be31a72bae7
Summary:
The command is supposed to check if the given commits are backed up and if not to back them up.
This is needed for `jf submit` to make sure that all the commits submitted are backed up by hg.
Reviewed By: markbt
Differential Revision: D14685222
fbshipit-source-id: 224a438ec1c5c17af6988c511be4af4a3a988a79
Summary:
Currently we skip metadata when constructing data wirepacks.
We need copy metadata in the data packflies, because the client expects the base text to contain copy metadata since the delta base on the server also contains copy metadata.
It is also needed for future file nodes validation.
Differential Revision: D14851678
fbshipit-source-id: 1a3f79dc2565cdb864bee2400d331ae3a7c3751b
Summary:
The Dxxxx revset does two things in parallel:
* walks the changelog in the search of commit associated with Dxxx.
* makes graphql query to resolve Dxxxx to the hash associated with the last
submitted version.
This behaviour is clowny and confusing and should be changed long-term. This is
just a short term fix to prevent it from crashing in cases there's no local
commit associated with Dxxx (or it's very old) and the graphql query comes back
with empty result (because there are no versions for the diff).
Reviewed By: markbt
Differential Revision: D14851313
fbshipit-source-id: 14f7531aa4691007266291a38a016242de563de4
Summary:
When determining the contexts to compare trees against, consider all parents
of the missing nodes that aren't themselves missing.
In the case of:
```
C
/ \
D B
|
A
```
Where A and D are common and B and C are missing, the parents of the roots
are only A, which means D's tree will not be compared against and all of its
trees will be included.
Reviewed By: StanislavGlebik
Differential Revision: D14798681
fbshipit-source-id: c80add907338a3011daf54a2304ea19415b7f9fb
Summary:
The debugbundle implementation for mutation parts was not updated with
the recent changes made to mutation entries. Update it.
Reviewed By: quark-zju
Differential Revision: D14798682
fbshipit-source-id: eb1defa33e8a0eed807360af2c130a225af15030
Summary: Allow users to configure which HTTP client backend to use for the Eden API via the `edenapi.backend` config option. Valid options are `curl` and `hyper`, with `curl` being the default.
Reviewed By: quark-zju
Differential Revision: D14657871
fbshipit-source-id: 7a9972d2380fbbd5ed62d1accae764dc03ca4c29
Summary:
Add a new Eden API client based on libcurl (via the rust-curl crate). This should help us work around issues with Hyper.
This implementation is based on curl's "easy" API, and is intentionally naive. I intend to update it to use curl's "multi" API to send several concurrent HTTP requests per operation in a later diff.
Differential Revision: D14656756
fbshipit-source-id: 1f71074506844104f0f3237023b38317a7f41979
Summary: This is a stop gap measure while we move internal path encodings to UTF-8.
Reviewed By: quark-zju
Differential Revision: D14791761
fbshipit-source-id: e4d5a845aa4f28ed0744cf80021fd1446f3d6129
Summary: `commitctx` takes error as a named parameter shadowing the module import.
Reviewed By: quark-zju
Differential Revision: D14805901
fbshipit-source-id: 359a6d19d6a08b3c65ebf1af8a5e5baaec6711a2
Summary: Updated help text for hg commit
Reviewed By: kulshrax
Differential Revision: D14811336
fbshipit-source-id: dbc26b3409094b1f70aca01b0f9ff89693ce7861
Summary:
In most cases, we won't have more packfiles than what is held in the LRU cache,
and therefore, building a set is unecessary. Testing for the length is a O(1)
operation and is therefore a quick test to verify if more packs needs to be
iterated on.
Reviewed By: quark-zju
Differential Revision: D14802940
fbshipit-source-id: 8b39befb27368d474ab71f3eeac2340d8183e70e
Summary:
When many packfiles are present, and searching for missing data, we will
iterate on the packs twice. We can avoid the second iteration by testing if new
packfiles were detected.
Reviewed By: quark-zju
Differential Revision: D14802942
fbshipit-source-id: 624d9a86e65395af3141ea10d42bcfb8ee18db83
Summary:
Building a set is a O(n) operation, and in the cases where refresh is called
often, it starts showing up in profiles. Instead of rebuilding it everytime, we
can simply update it when adding/removing packs.
Reviewed By: quark-zju
Differential Revision: D14802945
fbshipit-source-id: f8967d09d2d7d0cc0d7400b047a587d536315002
Summary:
The repo.ui.configbool was showing in profiles when running `hg log -p`. Since
its value won't change for the duration of the operation, let's cache it.
Reviewed By: quark-zju
Differential Revision: D14802941
fbshipit-source-id: c1a2a764cb4d4a1c0d45d5118bb1e892c10798a6
Summary:
Change the crash header to just a single line without blaming extensions or
Python. This makes the crash log shorter, and easier to read when there are
crashes in run-tests.py diffs.
Remove `ui.supportcontact` since it's no longer used.
Reviewed By: sfilipco
Differential Revision: D14393982
fbshipit-source-id: 3ede8d3d3d8fd5d125944a9a750350d3ce356b14
Summary:
Add the checkout identifier to the loginfo for commit contexts that take data
from the working copy.
Reviewed By: quark-zju
Differential Revision: D14751601
fbshipit-source-id: 6ddcbf17b970ee24237edf501ae3d7fc8d320193
Summary:
Add `loginfo`, a new field to committable contexts, the contents of which will
be logged when the commit is committed to the repository.
Initially the logging only includes the commit node and author, but more things
will be added later.
Reviewed By: quark-zju
Differential Revision: D14751602
fbshipit-source-id: 2432dfb92dd2489cb8ad3464316a42ebaa79657c
Summary:
When the user checks out a new commit (either by updating to it, by
creating a new commit, or by amending or rebasing the current commit), create a
unique identifier for that checkout.
Log this identifier at the start and end of command processing, and allow other
tools to also query and log the identifier.
This allows both Mercurial commands and other commands that log the identifier
to be correlated with a particular checkout.
Reviewed By: quark-zju
Differential Revision: D14648523
fbshipit-source-id: 2fad79c3010f5fad1a0e180e3d3d6d9c0a7f8e85
Summary: Extract the `makerandomidentifier` function from the `clienttelemetry` extension so that we can use it elsewhere.
Reviewed By: quark-zju
Differential Revision: D14725866
fbshipit-source-id: 0b3ed2e8c9698e892ad0f524f6dde7bf7337f85c
Summary:
One of the drawback of only using packfiles is the cost of the pathological
cases when one pack file is being downloaded per file. This can happen during a
`hg log -p`, or on Eden when running `cat` on lots of not materialized files.
In these cases, mercurial will become slower and slower as the command is
running.
This is due to mercurial having to search each individual packfile to find
whether some data exist locally, or it needs to be fetched from the network.
This search will iterate over each packfile linearly, and on a miss, a
directory scan will be performed to see if a new packfile is present.
Previous patches attempted to solve this by running a repack when the command
finishes, but in the case of `hg log -p`, or when Eden is requesting the
content of some file, the command could be running for a long time.
As a solution, let's repack while the command is running if we detect that too
many pack files are present.
Reviewed By: quark-zju
Differential Revision: D14740566
fbshipit-source-id: 95ba381bd99e5404e352f799e9053a6375abac0a
Summary:
Mercurial recently started to generate empty pack files (`0x01`). This will cause this check to fail:
diffusion/FBS/browse/master/fbcode/scm/hg/lib/cdatapack/cdatapack.c;2c4197d003ed906dd8eaf70fbb04aa53440ce681$314-319
This will subsequently result as a double-free error between these two:
**In `error_cleanup`**
diffusion/FBS/browse/master/fbcode/scm/hg/lib/cdatapack/cdatapack.c;2c4197d003ed906dd8eaf70fbb04aa53440ce681$387-389
**In `close_datapack`**
diffusion/FBS/browse/master/fbcode/scm/hg/lib/cdatapack/cdatapack.c;2c4197d003ed906dd8eaf70fbb04aa53440ce681$401
This diff will fix this bug.
Differential Revision: D14759374
fbshipit-source-id: 06f192513a935740c2142b5a2baac87a28903496
Summary: We're intentionally exporting Python classes with names that match Mercurials convention of `allonewordlowercase`, so we should turn off this lint.
Differential Revision: D14760996
fbshipit-source-id: cf79e8ebb6a01bd7bcfb062754cb034d94ac493b
Summary:
The phabricator team is trying to eliminate `DifferentialRevision` in favor of `PhabricatorDiff`
See context here
https://our.intern.facebook.com/intern/wiki/Dev_Tools/Phabricator/GraphQLObjectRename/
I'm using aliases to reduce the number of changes needed (the object from the graphql query will have the same keys)
Reviewed By: quark-zju
Differential Revision: D14703867
fbshipit-source-id: 32204315dc154db7cbff68ee286579c0200f57b2
Summary:
Loosefiles are quite slow to repack, and this is significantly affecting
laptops as the repack can potentially run for hours when a large amount of
loosefiles are present.
One of the reason for that is that an incremental repack really isn't
incremental for loosefiles as all of them are repacked at the same time.
Instead let's repack only one of the top-level directory at a time.
Reviewed By: quark-zju
Differential Revision: D14690348
fbshipit-source-id: ffba49840302ae0d99e32db410647e83e213fe64
Summary:
Currently, if you lookup a globalrev which does not exist in the
repository, it leads to a linear scan of the entire changelog even while using
the globalrev based fast commit lookup. This commit speeds up the same by
relying on the indexing information.
Differential Revision: D14726937
fbshipit-source-id: 076ad8608aa656075dcb2c1287502ffe93bed851
Summary:
This is just pending feedback from D14694620 which makes the class
interface more cleaner.
Reviewed By: quark-zju
Differential Revision: D14718748
fbshipit-source-id: 032fc25d7e5922aac5f4195eb007bd009822cb86
Summary:
Rather than having a `Builder` struct that knows how to build just one kind of Eden API client, let's have a common `Config` type that can be potentially passed to the constructor of several different client implementations.
This will allow the same config code to be re-used across different client types, as seen later in this stack.
Differential Revision: D14656757
fbshipit-source-id: 883ffd2dc0302ebe08960f079c113e2d0da2d2ca
Summary:
When the sql_repo_lock.py hook is run, the sql connection won't have been
established yet. Let's open a connection and clean it up afterwards.
Reviewed By: quark-zju
Differential Revision: D14708851
fbshipit-source-id: f20b6dacdcb12cee839163325164d2e75816100c
Summary:
The type for globalrev during lookup should be an integer instead of a
string after D14694623.
Reviewed By: quark-zju
Differential Revision: D14718749
fbshipit-source-id: eddc49ea0faa61df6ad435db2192fba4f61b0075
Summary:
Importing an extension should be side-effect free. Move the patching to uisetup
so they only get executed when the extension is enabled.
Reviewed By: kulshrax
Differential Revision: D14694622
fbshipit-source-id: c2802e0f4d852445dd7900f11d4ede6e58883d94
Summary:
Change the index key from decimal to binary to make the index more compact and
efficient. Move the binary to the beginning to avoid common prefixes and speed
up lookups.
Reviewed By: singhsrb
Differential Revision: D14694623
fbshipit-source-id: e908d287c1d3559639dbafa39a465f89b8f6f14e
Summary: This makes the progress visible.
Reviewed By: singhsrb
Differential Revision: D14694621
fbshipit-source-id: bb80546cb513c423471ba004bf00db7431046bcf
Summary:
Use revision numbers to make sure the index contains all commits.
Also include svnrev in the same index. This has multiple benefits:
- Handles svnrev mapping, too. Easier to drop hgsubversion.
- Handles the special commit without globalrev or svnrev gracefully.
It takes about 3 minutes to build the index for www.
I removed code about phases, or heads. The code runs on hg servers
and hg servers (hgsql) have hard dependency on rev numbers.
If we get a chance to migrate away from client-side local changelog,
we'll have Mononoke equivalent commit mapping by that time (or, push
commit mapping work to other teams).
Reviewed By: singhsrb
Differential Revision: D14694620
fbshipit-source-id: 9b74782f4c37b6620ab7168f7c7199d0cc5bc396
Summary:
This is a requirement to enable Phabricator migration to a globalrev
based repository. The release engineering team has also expressed interest in a
faster way to lookup the commits using globalrev.
I am just using indexed log for now to enable faster lookup. We might not need
to backfill the indexed log since we mostly care for the newer commits. But if
we do need to, we can do it later.
Reviewed By: quark-zju
Differential Revision: D14687707
fbshipit-source-id: fe02e9a72b9edb57cfd49a294642014c7204c058
Summary:
This is required for the fast commit lookup based on globalrev in
D14687707. This code is heavily inspired from DurhamG's implementation for the
git-hg mapping in the `hggit` extension.
quark-zju has good suggestions for improving this to include the svnrev as well
which can eliminate the requirement for a svnrev-hg mapping in the future. We
can certainly do those in a later iteration.
Reviewed By: quark-zju
Differential Revision: D14674111
fbshipit-source-id: d09d9e65ac0abcafd478c873086898f787398867
Summary: This is just for convenience and making the code cleaner.
Reviewed By: quark-zju
Differential Revision: D14689470
fbshipit-source-id: 911f49e24a3bb504098537169e9096b080a86535
Summary:
Allow running `hg push --to scratch/bookmarkname -r X` where X is a public
commit.
Note that updating a scratch bookmark that is on a public commit will
require a force push.
Reviewed By: quark-zju
Differential Revision: D14527730
fbshipit-source-id: dbc234b43d74a36b7403bea19e1e917ad69cdbbf
Summary:
Before this change, the following command will redraw the progress bar
frequently:
./hg debugprogress --sleep 1 100000 --config progress.refresh=1 --nested
After this change, the above command only redraws once per second.
This should reduce overhead in `repack.py:repacker.repackdata`.
Differential Revision: D14693924
fbshipit-source-id: baa281ba2e262d2ef8e83c2460f7edf5400db1af
Summary: This makes it easier to debug real world progress related issues.
Differential Revision: D14691592
fbshipit-source-id: 2a653f657b6ba9f9369f1551845949ac7e4d0b9d
Summary: Previously the debug commands for the Eden API did not check whether HTTP data fetching was enabled before trying to access the client singleton, which would result in a crash if HTTP was disabled. This diff adds explicit checks so that the commands will abort instead of crashing.
Reviewed By: quark-zju
Differential Revision: D14692893
fbshipit-source-id: d36e241d8460dadb555a15c92aca9334ba00f34c
Summary:
When doing a pushrebase, the server needs to mount a bundle over the
exist repository to run hooks. With lfs enabled this uses changegroupv3, and
with a treeonly client no manifests are sent as part of the changegroup (they
are instead sent as a separate part).
The combination of these three things hits an edge cases where if there are no
manifests in a changegroupv3 bundle, the bundlerepo class may not skip the
entire manifest part, and therefore the bundle appears to have no files in it,
even though the data is there.
This only happens if the server is in treeonly mode, or during pushrebase hooks
where we fake treeonly mode briefly. Since we don't have any server repos that
are treeonly and lfs yet, this didn't break us, but it would have. We caught it
because some of our hooks are affected when they try to read file contents.
The fix is to account for the cgv3 case in the bundle manifest consuming code.
Eventually we can get rid of all of this logic once we get rid of revlogs and
changegroups.
Differential Revision: D14676674
fbshipit-source-id: 86ac1ff4a92e37c4da3767a66760b55b658c836f
Summary:
With `remotefilelog.fetchpacks`, some pathological cases would end up creating
tons of packfiles and we would thus spend a significant amount of time in
refresh, scanning for new packfiles. Jeremy noted that for every refresh,
stat(2) would be ran on every file, even though only a handful of packfile
would be added.
Differential Revision: D14674093
fbshipit-source-id: df198d208e3f4e1d667e7bdd069a793984683282
Summary:
Now that repacks runs more often, it's easier to trigger a race between repack
deleting packfiles, and another mercurial process listing the packfiles and
trying to open them. If the packfiles are deleted after the directory listing,
all the packfiles will fail to be opened and were mis-reported as corrupted.
Reviewed By: quark-zju
Differential Revision: D14648308
fbshipit-source-id: c3b852f669e28db6f622bde217f339533e094223
Summary:
On Windows, SIGPIPE isn't present in the signal module. This caused the exit
code to raise an exception when trying to install a handler for it.
Reviewed By: DurhamG, strager
Differential Revision: D14655142
fbshipit-source-id: 68ddfc5250a4b706c5bc6a7ad1add37e1ccbe26e
Summary:
Add basic support for `undo` by updating visibility according to what the undo
extension would normally do with obsmarkers.
A future enhancement would be to use the `draftheads` directly as the new
visibleheads, however:
* `undo` currently uses `heads(draft())` which doesn't take into account
secret commits.
* In order to support the `--branch` flag, we would need to make significant
changes. The flag is marked as ADVANCED and isn't used very much, so we may
consider removing it.
Reviewed By: quark-zju
Differential Revision: D14603196
fbshipit-source-id: 5b8c91b4d8fa2b28cf9ad5726d88a3735aca50ad
Summary:
When changes are undone so that an earlier view of the repo is present, fate
should only be calculated for commits that are obsolete in the current view.
This makes smartlog look correct after `undo` commands have been used.
Reviewed By: quark-zju
Differential Revision: D14603198
fbshipit-source-id: 3adaad94b9fe80e8a622d469513c784b772ee235
Summary:
Record mutation entries inside transactions. This ensures that they only get
written when the transaction completes.
Indexedlog doesn't have support for pending writes, so mutation entries are not
visible to hook scripts.
Reviewed By: quark-zju
Differential Revision: D14566781
fbshipit-source-id: eb4c7bbd3878df82e8e7096a69509525f9fb93c0
Summary:
Remove the last parts of the mutationcache, relying directly on the indexedlog
indexes.
The set of obsolete commits is now computed on the fly - we will cache this
later.
Reviewed By: quark-zju
Differential Revision: D14566780
fbshipit-source-id: 770d0d9d507bed982e88518dcf63a4c2d6b46a69
Summary:
When visibility changes, the cache of filtered revs is no longer valid and must
be cleared.
Reviewed By: quark-zju
Differential Revision: D14566777
fbshipit-source-id: 681a73c126f3258baa7f61157aba557ea8aceeca
Summary:
To prepare for transaction support, we need to move the store out of the cache
object and into its own property on the localrepo.
Reviewed By: quark-zju
Differential Revision: D14566776
fbshipit-source-id: 8fff03a86953fb60ed06dbbdcdd0bffc379bd047
Summary:
Computing mutation entries for all local commits is expensive when there are
lots of local draft commits. Ideally we would have an indexed changelog that
would make these lookups fast, but until we have that, put entries in the
mutation store for these commits to take advantage of the fast lookup there.
Reviewed By: quark-zju
Differential Revision: D14566782
fbshipit-source-id: cc3a05715337a510a65d8ff436c59d16d0f0447e
Summary:
The computations for instabilities revsets (`orphan`, `contentdivergent` and
`phasedivergent`) are complex, and aren't really necessary for the way we use
mutation information.
The `orphan` revset can be implemented as an alias for `obsolete():: -
obsolete()`. The other conditions can be detected on a case-by-case basis
later if we need them.
Reviewed By: quark-zju
Differential Revision: D14566784
fbshipit-source-id: 60b8443ad4c0c82d8250d8e9a10e73fae770daa8
Summary:
To support future alternate hashing schemes, predecessor identities should be
prefixed with the hashing scheme in use.
Currently there is only one hashing scheme: `hg`, which is the Mercurial
hashing scheme.
Reviewed By: quark-zju
Differential Revision: D14566778
fbshipit-source-id: baaaf2f078886a1cc7ac20d12923a63b4da09db6
Summary:
The upstream has removed it in https://phab.mercurial-scm.org/D2393. Do the
same.
The deleted C++ code seems to leak `Py_False` if `usemanifestv2` is not set.
Reviewed By: singhsrb
Differential Revision: D14611525
fbshipit-source-id: d828526c31aaa861d100a846bba79d1f5898e245
Summary:
A user hit cryptic error message:
File "revset.py", line 423, in ancestor
ancestors = repo.changelog.index.ancestors
AttributeError: 'list' object has no attribute 'ancestors'
That was because they have run `rm -rf` and the `.hg` directory was mostly
removed. `hg` falls back to revlog v0, and the code in question obviously
dislikes revlog v0.
Change the error message to just say repo corruption.
Since there are code depending on revlog v1, and we don't use v0 or v2. Remove
both of them.
Reviewed By: singhsrb
Differential Revision: D14610769
fbshipit-source-id: 9015b74989d8fa4ad50bc36ec48bca26a1faa361
Summary:
Callers may call `remotenames.activepath` with the remote as a string URL. For
remotenames we must normalize these paths as we do for remotes that are passed
in as objects, for the same reason as in D14051479.
Differential Revision: D14621090
fbshipit-source-id: 8a02e9554039ad3daf3093001889cf1746ce395a
Summary:
The extdiff extension uses archive on the workingctx to make snapshots
of certain files. Prefetching fails on this because it sees placeholder file
nodes. Let's just not prefetch in this case at all.
Differential Revision: D14609691
fbshipit-source-id: 2d06bf015a8b10afa16b3b0bcb266afb102d63b9
Summary:
Now that we have perftracing infra, let's trace a bunch of likely
problem spots.
Reviewed By: sfilipco
Differential Revision: D14426367
fbshipit-source-id: 354a241aa9ac5d75d34062a9838d581b4f46746f
Summary:
Most of the repacks are background repacks, and most of the complaints are
coming from laptops users. Thanks to the rust repack, most of time during
repack is now spent in repacking loosefiles. While repacking them is expensive,
testing whether data is in a loosefile and obtaining it is actually pretty
fast. Packfiles have the opposite issue, repacking them is fast, but finding
data in them may be expensive if a lot of them are present.
Based on this, it makes sense to repack packfiles more frequently than
loosefiles. At first, the newly added option will be used to turn-off loosefile
repack for laptop users. A less frequent loosefile repack will be implemented
in a later patch.
Reviewed By: DurhamG
Differential Revision: D14586986
fbshipit-source-id: 5bc5c839cf8d2d78bcc4ffa016bbe3cf1b2ef3f7
Summary:
See D14606986 for context. `repo.close` should be protected by not crashing on
stdout or stderr write errors (SIGPIPE or EPIPE). Move `deferred` to `atexit`
handler to get the protection.
Test changes are caused by ordering changes.
Differential Revision: D14607407
fbshipit-source-id: 5a42aefcec395f48b8ecb67426429ef2e41f5666
Summary:
This is the first diff of an alternative fix of D14528603. See also D14603974
for context.
We'd like atexit handlers to not crash with EPIPE or SIGPIPE when writing to
stderr or stdout. Therefore disable SIGPIPE signal handler and patch ui methods
to do so.
Differential Revision: D14607159
fbshipit-source-id: 274c5174813d402a7e0b8b5be7c8fcb0524fcdb3
Summary:
akushner and I both hit an issue where `hg log` hangs for a long time after
exiting the pager by pressing `q`. That was caused by D14528603. Revert it to
fix it.
The root cause is apparently a bug in Python 2 (or C stdlib?):
# a.py
import signal, os
signal.signal(signal.SIGPIPE, signal.SIG_IGN)
out = os.fdopen(1, 'wb', 1)
while True:
out.write(b"a\n")
# shell
python2 a.py | head -n 1
# hangs
Python 3 does not have the issue.
Reviewed By: singhsrb
Differential Revision: D14603974
fbshipit-source-id: 3b4d716133853632e56dfeb59b1858f926a2c146
Summary:
Now that we're recording perftraces, let's log their results to the
blackbox if the command exceeds a configurable threshold.
Reviewed By: sfilipco
Differential Revision: D14426366
fbshipit-source-id: 24f261aaa2b089dbc959d709e51de1c0359d976d
Summary:
Adds tracing for the command dispatch. All other traces will be a
child of this root level trace.
Reviewed By: sfilipco
Differential Revision: D14426369
fbshipit-source-id: 2b996e027a2d6b163162bdceba571a3ee2186f6b
Summary:
Adds the initial perftracing infra. It's just a collection of functions
for starting and ending tracing and associating data with the traces.
Reviewed By: quark-zju
Differential Revision: D14426368
fbshipit-source-id: 72a2fad901ff52877f3958db2b924d95f45eeac7
Summary:
```
building 'indexes' library extension
warning: profiles for the non root package will be ignored, specify profiles at the workspace root:
package: /data/users/sfilip/fbsource/fbcode/scm/hg/edenscm/hgext/extlib/indexes/Cargo.toml
workspace: /data/users/sfilip/fbsource/fbcode/scm/hg/edenscm/hgext/extlib/Cargo.toml
warning: profiles for the non root package will be ignored, specify profiles at the workspace root:
package: /data/users/sfilip/fbsource/fbcode/scm/hg/edenscm/hgext/extlib/pyrevisionstore/Cargo.toml
workspace: /data/users/sfilip/fbsource/fbcode/scm/hg/edenscm/hgext/extlib/Cargo.toml
```
`profile` settings are ignored for non root packages. I introduced this issue
when I added a workspace for extlib: D14543989.
Reviewed By: kulshrax
Differential Revision: D14606019
fbshipit-source-id: 7ec4743d0913e443c378ae83f392817f6e6d3aab
Summary:
Closing a repo is required to commit the pending transactions, or to run a
background repack when packfiles were downloaded. Usually, the close method is
called in dispatch after a command is run and a valid repository was passed via
-R. The debugedenimporthelper however may open the repository manually, and
since it doesn't close it, we may end up with a large number of packfiles.
Let's manually close it to avoid this issue.
Reviewed By: simpkins
Differential Revision: D14531514
fbshipit-source-id: 9b2d1648775e4a2c9eea4c232bf0ae4fee43297d
Summary:
If the cloud sync or pushbackup commands start in background themselves, it will trigger a new background command to back up to the secondary afterwards.
Note that for cloud sync, for the secondary store we trigger pushbackup command, not cloud sync. If should be fine. We don't backup duplicates as we always check with the server first what is already backup. To call pushback looks simpler as it doesn't involve all the complexity of cloud sync, and basically we just need to backup to the secondary.
Reviewed By: markbt
Differential Revision: D14578770
fbshipit-source-id: f81a50fd76e64f2d77d8d7004b201fcfe8a090d2
Summary:
Change debugignore to test directories properly (which, only affects sparse
repos in practise) and stop printing out the source ignore rule location, as we
no longer have that information in the gitignore world.
Reviewed By: markbt
Differential Revision: D14547049
fbshipit-source-id: e7c307b1e1f4a8dcd30d00bec1f3e7a84a14b47b
Summary:
This particular situation happens in the wild when the amend doesn't rebase
because of conflicts and users work on their stack using `hg prev` and `hg next
--rebase`. In this case when there's non-obsolete child that's always the child
we want to choose.
We're verbose about what we're doing so it's not confusing to the users.
Reviewed By: quark-zju
Differential Revision: D14560584
fbshipit-source-id: a453c0301a5156eea0d19ceb40d9a64e80b7fca7
Summary:
Move the logic of adding the common ancestor to make the DAG connected to the
smartlog revset. This makes it handy for power users to just use `log` and
`smartlog` revset to get interesting graphs. `sl` is now a very thin wrapper
around the `smartlog` revset function.
Reviewed By: DurhamG
Differential Revision: D14461520
fbshipit-source-id: 78e3991059c9da7ef4410e252a2b69b1e54918cb
Summary:
Wrap user-provided revs with `smartlog` revset function. This makes more sense
together with the next change.
The test change is because "parents" of drafts are selected.
Reviewed By: DurhamG
Differential Revision: D14461519
fbshipit-source-id: 2a48931680f0dc50b80b87cea827152c21cf4791
Summary:
With the last change, the benefit of ancestorcache is limited. Therefore just
remove it to reduce complexity. This also makes `smartlog` closer to `log`.
Reviewed By: DurhamG
Differential Revision: D14461523
fbshipit-source-id: eb108a09e12b07e5012f70aef0b2940b07d746fb
Summary:
Use the `ancestor` revset to replace the adhoc ancestor calcuation. This makes
the code much shorter.
It's in theory slightly different from the old logic. But there are no test changes.
The new code can no longer take advantage of ancestorcache. Fortunately, with
optimizations, it is pretty close to a fully warmed up ancestorcache. Of course,
it's much faster than a cold ancestorcache.
Before (ancestorcache disabled):
quark@devvm33994 ~/fbcode/scm/hg % ./hg sl -T '.' --time --pager=off --all --config smartlog.useancestorcache=0 >/dev/null
time: real 75.050 secs (user 52.540+0.000 sys 22.520+0.000)
Before (ancestorcache warmed up):
quark@devvm33994 ~/fbcode/scm/hg % ./hg sl -T '.' --time --pager=off --all --config smartlog.useancestorcache=1 >/dev/null
time: real 2.670 secs (user 2.550+0.000 sys 0.100+0.000)
After:
quark@devvm33994 ~/fbcode/scm/hg % ./hg sl -T '.' --time --pager=off --all --config smartlog.useancestorcache=0 >/dev/null
time: real 2.970 secs (user 2.760+0.000 sys 0.160+0.000)
There are 5110 commits in the above smartlog graph.
Reviewed By: DurhamG
Differential Revision: D14461524
fbshipit-source-id: 68bee3c4397be833e381c582c20a849b768b144d
Summary:
Previously, the default master is `.^` when `--rev` is passed. Change it to
null so we're not adding unexpected "master" if `--rev` is used.
Reviewed By: DurhamG, sfilipco
Differential Revision: D14516266
fbshipit-source-id: ce93f5e905d674c21cc07bb5a2957d0fad302722
Summary:
We encountered a bug where if a file descriptor was not flushed prior
to forking the process, then both processes would end up flushing the same data
to the file, resulting in duplicate data and a corrupt file.
The real fix is to stop using fork, but for now let's address the most visible
problem by flushing pack files before forking.
Reviewed By: kulshrax
Differential Revision: D14569005
fbshipit-source-id: e002abe72c8014cbe49ccffab6159f8372affdb0
Summary: Deprecate due to complexity of the code.
Reviewed By: mitrandir77
Differential Revision: D14561405
fbshipit-source-id: 6184317f549c0ab84335b09c4b48efccdf31f7fc
Summary: Added new metrics to log loose files size and number during repack. We need it to understand how much better the pack files work in terms of disc and memory usage.
Reviewed By: markbt
Differential Revision: D14544811
fbshipit-source-id: 5a4d894bd5a3358c7e0f93ecc9db5e9f2c2f2372
Summary:
Basically we should check that the commits have been backed up.
If this is not true and the commits are local we can just back them up.
If they are not known by this repo, pull from the old one and then back them
up.
Reviewed By: markbt
Differential Revision: D14508239
fbshipit-source-id: 3fdd83335cb937b153510ec3c7510ecd1167d0ca
Summary: Log data about round-trip count, and object count for files, trees, and SSH calls.
Differential Revision: D14515776
fbshipit-source-id: cce416fd7dccdad3c73a9f1751a04ddac0d2c507
Summary:
Make it possible to use `ui.metrics.gauge` to collect metrics for a single
command, via the sampling extension.
Differential Revision: D14515775
fbshipit-source-id: e8a53549b00c1bc7b6509a5990a51d955d767d7e
Summary: This makes sure `ui.log` always runs through the sampling code path.
Differential Revision: D14515774
fbshipit-source-id: 585cd14eaecda12a9c2dd6ed003f0a457d67daf1
Summary:
Before this patch, metrics was designed to send stats to a global counter. I'd
like to use it for stats local to the current command (ex. count of file
fetching, count of round-trips, etc).
Change the API so "entity=None" forbids stats from sending to a global counter.
Differential Revision: D14515779
fbshipit-source-id: b5b3b040d674c71f467153c308b56aa6f506eb0c
Summary:
Disable parsing `.hgignore` and related fileset `hgignore()` by default. They can
still be enabled via configuration. The plan is to completely remove them
later.
A replacement for `hgignore()` fileset was added as `gitignore()`.
The `hgignore()` fileset seems to be only used by zertosh in the past 3 months.
Reviewed By: singhsrb
Differential Revision: D14543232
fbshipit-source-id: f2385062a0e816331f693239f62448979876078a
Summary: This simplifies testing setup for all crates in CI.
Reviewed By: quark-zju
Differential Revision: D14543989
fbshipit-source-id: 83693fada6e64b7c21fd89a880d6452d811ea90d
Summary:
Part way through our hg-git repo's history we started adding the git
hash to the extras so it was easier to generate the map files. Let's make
git-updatemeta stop when it passes this boundary.
This is useful for speeding up internal infrastructure that needs to generate
the mapfile from scratch.
Reviewed By: kulshrax
Differential Revision: D14542354
fbshipit-source-id: 7c17fb1b1439f9b4c0c0acf8b5a85790d02a0861
Summary:
git-updatemeta performance starts to matter when we have to run
it from scratch in new containers. Let's optimize this a bit. In a large repo
this improves the speed by 30%.
Reviewed By: quark-zju
Differential Revision: D14542339
fbshipit-source-id: 34f06369543b8d4d22838fd4e3878c6bec9a597c
Summary:
Improve the performance of the revsets that calculate which commits to back up
by limiting them to consider only the non-obsolete commits that are also draft.
Reviewed By: quark-zju
Differential Revision: D14544883
fbshipit-source-id: db9ed4a7abd81956762f56140321242dbccf2df0
Summary:
Not every command requires a valid repo, but when one is used, it is always
properly closed. Hence, let's simply wrap the repo.close method instead of
wrapping around the runcommand function.
Reviewed By: kulshrax
Differential Revision: D14531515
fbshipit-source-id: bcdbe6530c94041c1185b18570846ba609b5f605
Summary:
Attempt to detect oscillation of commit cloud workspaces by comparing the new
state after the current sync with the state before the previous application of
cloud changes. If the version number is incremented by 1, the workspace is
brought back to the same state, and less than a minute has passed since the
last time that commit cloud sync ran, abort the current sync step.
This happens after the commits have been backed up, but before the new cloud
workspace is synced to the commit cloud service. This prevents further
oscillation whilst ensuring the user's commits are still backed up.
Reviewed By: quark-zju
Differential Revision: D14540355
fbshipit-source-id: 20e4b0333f5a7e34b512967a03099625f62ff9d5
Summary:
Change how infinitepush determines what to back up.
Commits to back up are all draft ancestors of non-obsolete commits, *plus* all
draft ancestors of bookmarked commits, which may be obsolete.
Previously, in a stack of the form:
```
| x B (obsolete, bookmarked)
| |
| o A
|/
o
```
neither A or B would be backed up, despite normally being visible to the user.
Reviewed By: liubov-dmitrieva
Differential Revision: D14540356
fbshipit-source-id: 0d6ad330c53c818b08f736a9af64704cf0be7cd5
Summary:
Change how commit cloud determines what to sync and what has been successfully
synced when some commits fail to push.
Commits to sync are all draft ancestors of non-obsolete commits, *plus* all
draft ancestors of bookmarked commits.
Commits to sync when only some commits have successfully been pushed are
ancestors of the newly pushed heads, *plus* ancestors of the commits to be
synced that were successfully synced last time.
Reviewed By: liubov-dmitrieva
Differential Revision: D14540357
fbshipit-source-id: c082a2f2822f8bce4cd2bbac93a70e27e2ffaa59
Summary:
Now at the beginning of pushbackup we always check what the server already
have, this triggered changes in the output of the tests.
Before the limit of the check was 3 heads. I made it to 0 by default earlier and broke tests. Now we have fast way to check on the server side, so the limit 3 was not good. There is no reason to re-back up those.
Basically before we could backup the same thing again.
Reviewed By: markbt
Differential Revision: D14539582
fbshipit-source-id: 569b354580128c944a95fa64c0c964304a2cca8b
Summary:
SIGPIPE is a weird signal. It is sent to a process that tries to write to a
pipe (or socket) whose other end has been closed. On top of the signal being
sent, the write will also fails with EPIPE, and thus SIGPIPE is sometimes used
to avoid checking the return status of write (!!!) and to gracefully exit.
In Mercurial, the SIGPIPE is used to raise an exception to do exactly this, but
it also appears that the writes to stdout/stderr are already checking the
return value and doing the right thing, hence SIGPIPE is unecessary.
In the case of Mercurial, catching SIGPIPE has another interesting behavior.
The command `hg log -p` has a different behavior once the pager is killed
depending on whether HGPLAIN is set or not. When unset, `stderr` is also
redirected to the pipe, and thus any logs printed after the pipe is closed will
force Mercurial to quit. While this is the expected behavior in most cases,
it is not when the logs are happening during cleanup, as they will effectively
abort the teardown process.
Overall, it's better to just not install a SIGPIPE handler, so let's remove it.
Reviewed By: kulshrax
Differential Revision: D14528603
fbshipit-source-id: f51fb13016cc7b8d622e91c60d4c5286c7b404e5
Summary:
The check was added to "distrust" st_nlink, to workaround issues on CIFS [1].
The check only works for filesystem supporting hardlinks.
If a filesystem is marked as "st_nlink cannot be trusted", then vfs.py will take
a conservative approach to break hardlinks when writing, by copying the file
followed by a rename.
Therefore, filesystems without hardlink support (esp. edenfs) would go through
the slow path - every file written via vfs.py would be copied unnecessarily.
Fix it by skipping the check for known filesystems, including edenfs.
[1]: https://bz.mercurial-scm.org/1866
Reviewed By: DurhamG
Differential Revision: D14527370
fbshipit-source-id: 579111f5c72fa3d016f98c248b29c96afdc59086
Summary:
the test have been broken
```
buck test mode/dev scm/hg/fb/tests:fb_run_tests -- 'test_sl_output_t \(scm\.hg\.fb\.tests\.unittestify\.hgtests\)'
```
In the test the remote is not defined, so the test failed to check with the
server what has been backed up
Reviewed By: ikostia
Differential Revision: D14521930
fbshipit-source-id: 3363bb3055941decdfc65165860e1ef25a7a7891
Summary:
This is particulary useful for `hg cloud sync` when it calls pushbackup in the
background to the secondary storage at the end of cloud sync.
Pushbackup is not smart enough, so it will back up again what we just pulled from cloud sync.
However, all those commits are probably already backed up on the secondary
storage on another machine while cloud sync.
Reviewed By: singhsrb
Differential Revision: D14386616
fbshipit-source-id: e62ed0afb89c28fe6880346077c279e6705da602
Summary:
Using lookup command is slow because if something is not backed up, it throws
exception and we have to re-establish ssh connection
Differential Revision: D14386150
fbshipit-source-id: 8d5caea93516571ff36c80adb6406b0347d90384
Summary:
Reading a line over a pipe involves reading every character of the line
individually. This is extremely inefficient and slow, which cause prefetch to
be overly slow when most of the data isn't in memcache.
Using buffered reads tries to read 4096 bytes at a time, drastically reducing
the cost of reading a missing path/node pair from memcache.
Reviewed By: ikostia
Differential Revision: D14507063
fbshipit-source-id: e0910d7a303e15fe2d3c61fe2739e6c13370058f
Summary:
As part of the mononoke write path rollout we want to be
able to dynamically block writes to a repo. This is implemented as
a table in hgsql (repo_lock) that both hg and mononoke can read, but
will only be updated by scmadmin.
Expose a function that can be used by in-process hooks to check if a repo is
locked for mercurial.
Reviewed By: quark-zju
Differential Revision: D14279169
fbshipit-source-id: f8bb4afeeeda67796cf806ab7f3fe42f4089818f
Summary: This is removal of unused code that left from D14455360
Reviewed By: ikostia
Differential Revision: D14502837
fbshipit-source-id: df1912c7997847b0628b492b3fe735d5e3d7f201
Summary:
On connections that might experience higher latency the throughput for a clone
is easy to spike, raising the chunk size results in more consistent throughput.
Given that hardware these days has plenty of memory, this change shouldn't make
any significant difference either.
Reviewed By: farnz
Differential Revision: D14502443
fbshipit-source-id: 0e90f955304e9955df0ec69b5733db7c34b09e83
Summary:
Change `ancestor` to send 24 revs to the native ancestor logic at a time,
instead of 2 revs at a time.
This makes it much faster when calculating ancestor of many commits.
Before:
quark@devvm33994 ~/fbcode/scm/hg % hg.real log -r 'ancestor(limit(draft(),6000))' -T '{node}\n' --time --hidden
ca4beab06459e016ac1b4041400029165b701afa
time: real 66.620 secs (user 44.800+0.010 sys 21.840+0.000)
After:
quark@devvm33994 ~/fbcode/scm/hg % ./hg log -r 'ancestor(limit(draft(),6000))' -T '{node}\n' --time --hidden
ca4beab06459e016ac1b4041400029165b701afa
time: real 3.470 secs (user 2.380+0.000 sys 1.000+0.020)
This also affects `merge.preferancestor`. That's a dangerous config option and
seems fine to be ignored in the revset implementation.
Reviewed By: DurhamG
Differential Revision: D14461521
fbshipit-source-id: ec00921aece0adc6aaca49e5580bff52784c4ca5
Summary:
Use "interestingmaster()" to make it easier to see how "master" gets decided.
Change the type of "master" argument taken by "smartlog" revset from string to
revset. This is more consistent with other revset functions.
Reviewed By: DurhamG
Differential Revision: D14436003
fbshipit-source-id: 5aa166b523f36672f77dc4f161ae8d64c2b50579
Summary:
Similar to `interestingbookmarks()`, this exposes more smartlog logic to the
user.
Reviewed By: DurhamG
Differential Revision: D14436004
fbshipit-source-id: bd4ef1dcee8e7b29c43ce43fe6c1a3e7b5286774
Summary:
Make `heads` in `smartlog` customizable. This makes smartlog more flexible.
Instead of using the default selection, the user can choose draft branches, and
potentially pass in `interestingbookmarks()` to include bookmarks and remote
bookmarks. For example, `smartlog(.::)` shows the current branch and the public
commit it bases on.
Drop `recentdays` as it can now be expressed using `heads` directly. See the
test change.
This would also hopefully make test-fb-hgext-smartlog-hide-before.t stable,
as it no longer uses `time.time()`.
Reviewed By: DurhamG, sfilipco
Differential Revision: D14436007
fbshipit-source-id: 5e0a76e4424b01312fef02fae23a3abd74e863c6
Summary:
The old code basically selects ancestors of heads.
Rewrite the logic using revsets. Assuming we're only interested in ancestors
that are drafts, we can take advantage of `draft() & ::x` optimization.
The new logic also assumes master rev is public. Otherwise it can be slightly
different from the old logic.
The new code is much faster on my repo:
New code:
quark@devvm33994 ~/fbcode/scm/hg % ./hg log -r 'smartlog()' --hidden -T . --time | wc -c
time: real 0.630 secs (user 0.550+0.000 sys 0.030+0.000)
6716
Old code:
quark@devvm33994 ~/fbcode/scm/hg % hg.real log -r 'smartlog()' --hidden -T . --time | wc -c
time: real 5.470 secs (user 3.920+0.000 sys 1.550+0.000)
6716
This might make the ancestorcache hack (D5135746) unnecessary.
Reviewed By: DurhamG, sfilipco
Differential Revision: D14436008
fbshipit-source-id: 3c3bf47ccb67ea0e238542995009da9b9250b43b
Summary:
The `smartlog()` revset does a lot of things.
Add a new revset `interestingbookmarks()` to expose part of the smartlog features.
Reviewed By: DurhamG, sfilipco
Differential Revision: D14436006
fbshipit-source-id: 15b8d203b6547e63f8d062660ad27bdbc25b2c1c
Summary:
The code falls back to head() if there are no remotenames. We don't need that
behavior. Therefore just simplify it by always using `heads(draft())`.
Reviewed By: DurhamG
Differential Revision: D14436009
fbshipit-source-id: 25c2d245ed64a29e3e1677ededb4c2ba7b4a3ceb
Summary: D14387734 added 2 new arguments to the `httprequestpacks` function, but didn't update the callsite to pass those arguments. This diff fixes the problem.
Differential Revision: D14468592
fbshipit-source-id: 7e573838916067fc2cc12204ea1da460eb3955c8
Summary:
Currently archive is almost useless because it fetches each file
one-by-one. Let's add prefetching.
Differential Revision: D14460880
fbshipit-source-id: 1f06e1ac9d03aae3ab27d3064f9fe6141051be06
Summary:
In preparation to support Mononoke clean up the features that are Mercurial
specific and Mercurial infinitepush implementation specific.
For Mononoke migration we will to write a whole new set of logic what to do if
the "infinitepush" path has been changed. So, clean up is useful before
writing this logic.
Reviewed By: singhsrb
Differential Revision: D14455360
fbshipit-source-id: d15c3a9032b4888a1aa391da34ad5e499aba9a15
Summary: See the linked task for details.
Reviewed By: quark-zju
Differential Revision: D14448505
fbshipit-source-id: fc2efa71510b718c25a2cea3acf39663d280f19a
Summary:
After going over the code review for D14332967, I have decided to keep
things simple for now and only allow making commits to same target parent as
the original parent. This was already the intention with the existing code.
Therefore, this commit just further enforces the requirement.
Reviewed By: quark-zju
Differential Revision: D14422351
fbshipit-source-id: 2f786fc3596b17c5020de9906adf8f22b50be4dd
Summary:
These was probably introduced by moving to black.
The changes in the diff were generated by script.
Reviewed By: mitrandir77, singhsrb
Differential Revision: D14439667
fbshipit-source-id: 54f6e0bdcc59c1c6deb4eea46dc6f865bcd48cf8
Summary:
The code currently assumes that the target parent is the same as the
original parent by totally ignoring the original parent which can seem
surprising and more importantly, hinders supporting behaviors such as allowing
commits to a new parent. Therefore, this commit fixes the code to identify the
original and target parent separately.
Reviewed By: quark-zju
Differential Revision: D14422352
fbshipit-source-id: bc175f2fe636f9bf47a68f64c8efd52660e3b1b7
Summary:
D14183009 made commit cloud accept cloud bookmarks for hidden commits, rather
that omitting them. However, this only works for future bookmarks. If the
bookmark was already omitted, then `_checkomissions` would not recover the
situation for the same reason.
Update `_checkomissions` to also allow cloud bookmarks on hidden commits.
Reviewed By: liubov-dmitrieva
Differential Revision: D14437656
fbshipit-source-id: 2372323022a59bfd4210bc76f39b9a74872d5efe
Summary:
Now that hg_memcache_client will voluntarily exit when hg terminates, we no
longer need to wait for hg_memcache_client to finish before exiting hg.
Reviewed By: DurhamG
Differential Revision: D14396510
fbshipit-source-id: 7e73d9b70d481e58a0c47cd0f408580e6d548fd9
Summary:
This is useful for testing the `memcommit` command as the clients can
specify different destination parents than the original parents of the commit.
Differential Revision: D14410213
fbshipit-source-id: 846e0d764b9606f00aed95995c694f379457eec7
Summary:
Earlier message suggested the feature is not supported when in fact it
is allowed via a configuration.
Differential Revision: D14410214
fbshipit-source-id: 0ec2a22920417c378cf3ac596565f9d2fa5f6d5c
Summary:
Operations like `hg log -p` will inherently cause many requests to
hg_memcache_client. This will force many small packfiles to be created which
will significantly slow down future invocation of hg.
Now that `hg repack --incremental --packsonly` is fast, we can afford to run it
when mercurial exit after a prefetch operation was run.
Differential Revision: D14387735
fbshipit-source-id: 45f89f1120458c8b2471a1c55cafb6bc87263dd0
Summary:
When using packfiles, history and data are in different files, and thus it's
possible to only fetch one.
For now, besides the requests coming from contentstore and metadatastore, both
will be fetched, as the code hasn't been audited to know whether we only want
history or data.
Differential Revision: D14387734
fbshipit-source-id: 6aafd477ff486b9316458ce0e80636152db45b89
Summary:
For pushbackup it is needed to make hg rage more informative because we store
states for different paths separately anyway.
For cloud sync we will have to write some migration logic: if the remotepath
has been changed, we have to check what the server has to make sure everything
is backed up as cloud sync would expect.
Differential Revision: D14420713
fbshipit-source-id: 2046e9d7b16291a49d1bc40da5285de58017f4f2
Summary:
Corrupted packfiles, or background removal of them could cause repack to fail,
let's simply ignore these transient errors and continue repacking.
Reviewed By: DurhamG
Differential Revision: D14373901
fbshipit-source-id: afe88e89a3bd0d010459975abecb2fef7f8dff6f
Summary:
Fixes the below type error when pulling from a git repo using hg-git on my laptop:
File "edenscm/hgext/remotenames.py", line 225, in expull
pullremotenames(repo, remote, bookmarks)
File "edenscm/hgext/remotenames.py", line 314, in pullremotenames
path = activepath(repo.ui, remote)
File "edenscm/hgext/remotenames.py", line 1464, in activepath
rpath = _normalizeremote(remote.url)
File "edenscm/hgext/remotenames.py", line 1439, in _normalizeremote
u = util.url(remote)
File "edenscm/hgext/hggit/__init__.py", line 164, in _url
if not (path.startswith(pycompat.ossep) and ":" in path):
AttributeError: 'function' object has no attribute 'startswith'
Basically, `peer.url()` is the API, `peer._url` is a private field that does
not always exist somehow.
Besides, further remove named branches that can crash hg-git with
NotImplementedError:
File "edenscm/hgext/remotenames.py", line 225, in expull
pullremotenames(repo, remote, bookmarks)
File "edenscm/hgext/remotenames.py", line 322, in pullremotenames
for branch, nodes in remote.branchmap().iteritems():
File "edenscm/hgext/hggit/gitrepo.py", line 73, in branchmap
raise NotImplementedError
Reviewed By: DurhamG
Differential Revision: D14144462
fbshipit-source-id: 2e886c639cf6689480f84626eaf0d5ec25512ea0
Summary:
Since state files have been changed, now it is one per path, we should dump
them in a correct way
Reviewed By: markbt
Differential Revision: D14406311
fbshipit-source-id: 8d74a51e63028ec81bcf5e55ad117d3c960b4651
Summary:
To make "draft()" bounded and train users to hide unused commits manually,
change smartlog to "hide commits before <a static date>", instead of
"hide commits that are older than <a static time span>". Then we can
incrementally decrease the static date, and eventually show everything, and
force the user to think about what commits to keep or hide.
Reviewed By: singhsrb
Differential Revision: D13993584
fbshipit-source-id: 1a2b56f50d7f014a589f798cd2feaa6931e64fe3
Summary:
Subrepo is another unloved features that we don't want to support.
Aggressively remove it everywhere, instead of just turning off configs.
I didn't spend much time to split this commit so it's smaller and more friendly
to review. But it seems tests are passing.
Reviewed By: sfilipco
Differential Revision: D14220099
fbshipit-source-id: adc512a047d99cd4bafd0362e3e9b24e71defe13
Summary:
If an infinitepush bundle contains flat manifests and is served from a
treemanifest repository, it can potentially fail to send all the needed data to
the client.
Understanding the bug requires two bits of context:
1. When sending commits from a tree server to a tree client, we generally don't
send any trees because they can be fetched by the client ondemand later. The one
exception to this is for infinitepush bundles, where the trees inside the bundle
cannot be served ondemand, and therefore must be served at pull time. To do this
we check if a given manifest node exists in the repositories permanent storage,
and if it doesn't, we assume it came from an infinitepush bundle and serve it
with the pull.
2. When we lookup a manifest and fail to find a tree, our last resort is the
ondemandstore which knows how to convert a flat manifest into a tree manifest.
On the server, this is responsible for converting each of the flat bundle's
manifests into treemanifests before we serve the bundle to the client. As part
of converting the flat manifests into treemanifests, it writes the new tree
data into a pack file.
The bug is then, when serving a stack of commits, if we try to package up the
top tree first (i.e. the most recent tree), we end up converting the entire
stack from flat into trees, which inserts the bottom most trees into the
temporary pack file. Because they exist in the temporary pack file, when we
later check if they are part of the repositories store we end up finding them,
which causes us to treat them as not-infinitepush-trees which means we don't
serve them to the client.
The fix is to change the infinitepush tree-serving code to not consider the
mutable packs when checking if it should send trees.
Reviewed By: mitrandir77
Differential Revision: D14403925
fbshipit-source-id: 38043dfc49df5ff9ea2fae1d3cac341c4936509c
Summary:
When calculating bundleroots (the commits that are the common ancestors for the
infinitepush bundle), we currently include all the `nullid` parents (i.e. p2 for
most commits). This bloats the size of the list, and is unnecessary.
Reviewed By: quark-zju
Differential Revision: D14385912
fbshipit-source-id: c518b8b1aa27cff8562c2358a024b8a08ced8cba
Summary: Make it possible to override dates in commits.
Reviewed By: singhsrb
Differential Revision: D13993585
fbshipit-source-id: 59a72302d7ed0cb22f4eff84c1325e167963508c
Summary:
The feature was completed by Phil in D9816270. It's handy and can probably
reduce user support burden like: https://fb.intern.facebook.com/groups/scm/permalink/2039619916087618/
Therefore let's enable it.
Reviewed By: DurhamG
Differential Revision: D14293405
fbshipit-source-id: 54e934e0bf495c090109462e4f743d427df39380
Summary:
Some dependant libraries may only care about the serialization logic. As an
example, see D14332987 where the `hgrepo.py` only needs to depend on the
serialization. Therefore, its cleaner to extract out the serialization from the
`memcommit` data.
Reviewed By: quark-zju
Differential Revision: D14388474
fbshipit-source-id: 6f049dcc596b66b9ad72905f133529bdc9092382
Summary:
The shared code can be potentially called by clients using python 3. Therefore,
let's be compatible with python 3.
Reviewed By: quark-zju
Differential Revision: D14387005
fbshipit-source-id: 2ffb359d4d2762ffcba4a26a3ae5a7b45e89572b
Summary:
Eventually we'd like "default" to be not a special name. This is one step
torwards that.
Differential Revision: D14233455
fbshipit-source-id: 739091a124bc667c607c283bf00abc66b4081d25
Summary:
Some hooks (ex. mergedriver) are checked in old release branches. Provide
"mercurial" module compatibility so they won't break.
Reviewed By: mitrandir77
Differential Revision: D14366343
fbshipit-source-id: d47cc4fd512f63e4f6cdc5d7e5ab2c4398216b2f
Summary:
Introducing new command for Mercurial only to support check of existing commits in both the repo and the infinitepush storage.
For Mononoke case, just the standard 'known' works fine.
We can not overload the standard 'known' in mercurial case, because it is used in discovery and having there infinitepushcommits breaks some commands.
Next diff is replacing isbackedupnodes with isbackedupnodes2
Reviewed By: markbt
Differential Revision: D14370603
fbshipit-source-id: 8d7b64b4d556c0f1caa7f797dba360501571daad
Summary:
If Mercurial detects `ENOTCONN` when trying to open the current directory,
there is a high likelihood this is a disconnected eden virtual checkout.
Provide some advice as to what to do: run `hg fs start`.
Reviewed By: strager
Differential Revision: D13888873
fbshipit-source-id: 7619df0681d15b862d1a6f86d90491aa873bf86b
Summary: Include the output from edenfs rage in the hg rage output.
Reviewed By: chadaustin
Differential Revision: D14007061
fbshipit-source-id: fe0baf6c19dd4f2afd470ba70304c78582dfe879
