Summary:
D8560049 made `{reviewers}` work with commit template, but it breaks empty
`committemplate.reviewers` with `abort: "" not in template map`. This diff
fixes it.
Reviewed By: DurhamG
Differential Revision: D8596389
fbshipit-source-id: e1c046d59097181ef5c8d56a1cffc7c2407f0ba5
Summary:
Treestate tracks untracked files while treedirstate doesn't. The
treedirstate importmap logic changed by D7912557 will remove untracked files
from the tree. But it missed other places like nonnormalset, otherparentset
and copymap. This patch changes it to make sure paths in those sets are
tracked.
Reviewed By: DurhamG
Differential Revision: D8560022
fbshipit-source-id: 8ac39bfe634804ea94125fe42e11382acae35d94
Summary:
Previously `state._usetreestate` is a value set in `state.__init__`. In the
treestate migration case, `state._usetreestate` needs to change accordingly.
Therefore let's use a property to always calculate that value from the
source of truth.
Reviewed By: DurhamG
Differential Revision: D8560019
fbshipit-source-id: a313f603275321b959502e7f2c528c4b54a11ce9
Summary:
Revset predicates that work substantially through `subset.filter` should have a
higher weight than the default value of 1, to prevent the optimizer incorrectly
optimizing operations like:
```
reverse(ancestors(node, limit)) & filterpredicate()
```
in a way that filters all the revisions in the repo according the predicate
before performing the dag search.
Reviewed By: ikostia
Differential Revision: D8588027
fbshipit-source-id: 57be642761af013be8c8aa27ec126b3919c34256
Summary:
The expression `::revs & age("<x")` can be slow, as all ancestors must be
analyzed in case there are commits with ancestors that have later dates. Most
of this time this is unnecessary, as commits are usually monotonic in their
dates.
The `ancestorsaged` revsetpredicate evaluates to all ancestors within an age
range, optimized as if commit dates are monotonic.
Reviewed By: ikostia
Differential Revision: D8489319
fbshipit-source-id: 9b9055019c2afa70497a9df02c6fe3504e2c381a
Summary:
In the recent refactor we changed the definition of "is this tree
local" on the server to also include if the tree didn't exist in the store,
since then it might be an ondemand conversion from a flat manifest in a bundle.
This caused issues on peer-to-peer pulls, since most trees don't exist locally.
Let's fix it by only considering a missing tree to be a local tree if we're on
the server.
Reviewed By: quark-zju
Differential Revision: D8586779
fbshipit-source-id: ea8c8ee3e2a4dc054fae9caa122cfc733df1ab81
Summary:
The treemanifest manifest revlog store was keeping all the revlogs it
had ever opened. This was causing unnnecessary memory usage. Let's use an lru
dict instead.
Reviewed By: singhsrb
Differential Revision: D8579992
fbshipit-source-id: a0df30401714c93eebe0448b0ce92571f704bd96
Summary:
It seems the datapackstore c++ implementation is very leaky, which is
causing memory issues on the server. We're in the process of rewriting the
datapack logic to be pure rust, so let's just disable this feature for now.
Disabling this slows down tree downloads by 25%.
Reviewed By: quark-zju
Differential Revision: D8580519
fbshipit-source-id: 7e42e7ff211008d5461f95386da019ba5696e07b
Summary:
The template keyword only works for committed changesets. That conflicts
with committemplate. So special case the working copy case and make it work
if committemplate.reviewers is set.
Reviewed By: liubov-dmitrieva
Differential Revision: D8560049
fbshipit-source-id: 1427d0b72e023741be7012dd6eb4d51b0b595f39
Summary:
The `repos` file contains paths to repos so `hg gc` can scan them. The
current implementation basically writes a line per `hg pull`, and `repos`
can grow unbounded. Let's do a check before appending to that file.
Reviewed By: ryanmce
Differential Revision: D8416210
fbshipit-source-id: 0624bfbd6eb7d71dd749626e96063cb10f55bb7f
Summary: Commit cloud state is stored in the store, so the wlock is not necessary.
Reviewed By: liubov-dmitrieva
Differential Revision: D8121600
fbshipit-source-id: f96f4764e3cf0637cd45a0bc83d1227318ac9083
Summary:
`/` in this case is semantically not a dir seaparator, but a bookmark namespace
separator, so let's use `posixpath`, rather than `os.path` to combine parts.
Reviewed By: StanislavGlebik
Differential Revision: D8477029
fbshipit-source-id: 964b7d3f8a560fb73dfaaa39757c091908af77c5
Summary:
Flat manifest backfilling (which is used when reverting from treeonly
mode) was failing when the client had public heads that didn't exist on the
server (for instance, if they pulled from svn directly or if they had old
commits that had been stripped from the server). This was breaking the pull.
Let's fix it so we never send unknown heads to the server during the backfill
pull.
Reviewed By: phillco
Differential Revision: D8526027
fbshipit-source-id: 8d5bf67d2022dbc24facd0b96e001c8c81a0a2da
Summary: This adds a merge tool to only rebase if there aren't conflicts, which I think will be widely useful. It's for IMM only at the moment.
Reviewed By: DurhamG, quark-zju
Differential Revision: D8493065
fbshipit-source-id: 32ccc0e53af27d275a56bcac80861a421d7abfc6
Summary:
This was added to ease the IMM rollout, but today IMM rarely raises new exception, and this code fires on real errors that should be raised -- you can see here this lets us remove many hacky workarounds.
Some of the exceptions that didn't have carve-outs were:
- interrupted update
- source and destination form a cycle
- rebase in progress
Reviewed By: DurhamG, quark-zju
Differential Revision: D8522466
fbshipit-source-id: b407807ea059a0861fb3caca1db8699242342075
Summary:
This is a quick bug fix for the situation xixixao discovered:
You rebase commit B onto C while having A checked out, and the rebase creates merge conflicts.
In that case, the command aborts complaining of the uncommitted changes. But, the rebasestate isn't properly removed. The user might logically run `hg rebase --abort` to clear it, but that does an
`update` with `force=True` which can cause work to be lost.
This is just a quick bug fix. The right solution is to make an inmemory `mergestate` and not write it at all if rebasing in-memory.
Reviewed By: DurhamG
Differential Revision: D8517289
fbshipit-source-id: a62c99ccb2de1be102f9c23cd737794184c4fde8
Summary:
When building a bundle that will be used for a pull that can't use the linkrev
fastpath, we need to process the manifests of the commits for which we will
send files so that we can correctly set up their linkrevs. Currently this is
sometimes happening for all commits, which can be very slow when pulling a
large number of commits.
First, make it clearer what `LocalFiles` and `LocalTrees` means, and make it
easier to test for trees being local.
Second, when processing manifests looking for files that will be included in
the bundle, don't process non-local trees if we are only interested in local
trees.
Third, correctly determine whether to send all, local, or no trees or files
depending on what combinations of treemanifest and remotefilelog the client and
server support, and what operation is taking place.
Finally, we aren't passing `b2caps` through to `changegroup.makestream` in the
changegroup part generator for bundle2 parts. This means the above test for
whether the client supports treemanifest always failed.
With these combined changes, a pull of a specific public revision (so not
following the fastpath) where the client supports both remotefilelog and
treemanifest, will result in the server using `LocalFiles` and `LocalTrees`,
and therefore not processing the manifests of public commits looking for which
files to send.
Reviewed By: DurhamG, ikostia
Differential Revision: D8446137
fbshipit-source-id: 9a4247af75fe79c1962a29fcb8bf0181868699f4
Summary:
recent refactoring broke user experience if token is not found
not found is actually an error, rather than an empty result for security command
Reviewed By: markbt
Differential Revision: D8489287
fbshipit-source-id: 5ee6e821ce79877b43f93cb86b6935294b9e4a67
Summary: Move the log statements up so it becomes effective for treestate.
Reviewed By: markbt
Differential Revision: D8458183
fbshipit-source-id: af943c46dc3a3b0bd403476112473095d33b3650
Summary:
Let's make it clear for the user what background process is running and how long
I think it is really nice because the user can see if it is stuck and also have direct opportunity to kill it
Reviewed By: markbt
Differential Revision: D8476817
fbshipit-source-id: fefaef91e62b5e6199801a9e010a360e26ee6824
Summary:
Implement gc for treestate. Dirstate writes are protected by wlock. So it's
fine to do a gc inside the write method.
Reviewed By: markbt
Differential Revision: D8394736
fbshipit-source-id: 672087a2c14b855d1ca0449eb5b7d92d65bba418
Summary: Similar to treedirsate, if the file size exceeds configured limit, do a repack.
Reviewed By: markbt
Differential Revision: D8394735
fbshipit-source-id: 21a73cd32b33ed462bab8050c3d42a5e7545d3a4
Summary:
cloud sync:
when it is executed manually it doesn't have any sense to let user wait 30 seconds and look at nothing (there were no any message before 30 sec)
in the world of scm daemon, it is a normal case that cloud sync is already running,
let just tell it to the user and exit
cloud sync:
for background / scm daemon runs timeout is still the same
Reviewed By: markbt
Differential Revision: D8449219
fbshipit-source-id: 2b5c0c6874e23bcd71ca74b2a5231df3ae581e3b
Summary:
inform user that backupdisable command doesn't affect the current running
process if any exists
Reviewed By: markbt
Differential Revision: D8471761
fbshipit-source-id: 338b6d9e0332f5c845f26cbfddbf1d1c89f0ac41
Summary:
cloud sync can trigger another cloud syncs from itself when pull commits
disable it, this is the same setting we use for pullbackup
Reviewed By: markbt
Differential Revision: D8477217
fbshipit-source-id: 656fff11870d05c33aabf284fc9b4070bd5a29ea
Summary:
This fixes a subtle bug related to mergedrivers and IMM.
If your DAG looks like this:
```
dest (new mergedriver code)
|
| A (current commit)
|/
base (old mergedriver code)
```
And if you use IMM to rebase `A` onto `dest`, the wrong thing can happen because parts of `base`'s mergedriver are still loaded.
This only affected certain types of driver changes -- adding or removing parameters between files in the driver, or a driver change that also mutated the working copy (e.g. renaming a file checked by `preprocess()`). And, it only occurred if you had `A` (or some other change between `base` and `dest) checked out.
So, evict the old driver files, and all is well.
The right long-term fix is to probably implement our own Python loader for the mergedriver that reads all driver files from either the `source` or `dest` changectx not `.`. But, that's a bigger change.
Reviewed By: quark-zju
Differential Revision: D8464848
fbshipit-source-id: 8f03a4c99418aa26c3890989d19dd908b026c328
Summary:
implement in the easiest way, we will implement in a better way later
this will reduce confision when we will launch scm daemon
Reviewed By: farnz
Differential Revision: D8450341
fbshipit-source-id: 1c6bedbcf68224dd1e61ef4ec5cd04f26e76453a
Summary:
Regression caused by D7909172. This could trigger slow "status" calls 100%
for treestate, and cause race condition for non-treestate cases.
Reviewed By: phillco, singhsrb
Differential Revision: D8458182
fbshipit-source-id: 6a2e9939d83f55be4b0597b6e15932c9219aac4b
Summary:
This commit simply adds a method for checking if the hgsql extension
is bypassed. This will be used in a later commit for validating the repository
configuration.
Reviewed By: DurhamG
Differential Revision: D8364386
fbshipit-source-id: 70cc08c55339dc4a47ae17b30c131bbd2167e144
Summary:
This commit moves the logic concerning creation of commit during the
graft step in pushrebase to a separate method. This allows for easily wrapping
around the commit to add custom logic. In particular, this can be used to add
extras for the commits that will be created using the pushrebase.
Reviewed By: DurhamG
Differential Revision: D8364383
fbshipit-source-id: 60f845ec3f4f0b4d3799f4385ce49c6fd78cdda2
Summary:
This commit just introduces a method to check whether non pushrebase
commits are blocked. This will be used in later commits.
Reviewed By: DurhamG
Differential Revision: D8364384
fbshipit-source-id: ca0fb39e7925ce8dc7374bf9eb26f9c47506a67c
Summary:
The new treestate was designed to store fsmonitor state. Use it to make
fsmonitor and dirstate state consistent, and avoid fsmonitor state
invalidation.
The "fsmonitor identity" check was removed as we now rely on the dirstate
identity check - dirstate and fsmonitor state must be updated consistently -
both updated or neither updated.
Since this is the first dirstate that tracks "untracked" files, several places
are adjusted (ex. dmap.dropfile, dmap.keys) to take the new untracked files
into consideration.
Reviewed By: wez
Differential Revision: D7909172
fbshipit-source-id: 05fd64b25c67ae4b07bc8cfee2731c748205975e
Summary:
The previous logic restarted the *entire* rebase if one commit had merge conflicts or needed a mergedriver run. This was unnecessarily slow, plus the abort triggered a full repo scan.
Also, this logic is actually simpler and easier to reason about when done here, vs. `rebase()`. We can delete all the wrapping that function does after this rolls out.
This is config-gated, so we can roll it out gradually. Old config = old behavior.
Reviewed By: quark-zju
Differential Revision: D8272566
fbshipit-source-id: 9e598eea10c8a634497cd211a468dd2f58c5ff47
Summary: This just moves the logic around rebasing a single commit into a function, _rebaseone, so we can call it twice in the next diff.
Reviewed By: quark-zju
Differential Revision: D8272564
fbshipit-source-id: 38a15c099bb75d3a6b015fd22453c3732a6b21f5
Summary:
security command normally show a GUI prompt to allow you to type your password if the keychain is locked
if it is unable to do so, it returns 36 and empty stderr
it is ok that it is failed, because it happens due to security reasons,
the problem is only that stderr is empty, so the reason of failure is not clear to user.
I didn't find any option in the man of security command itself. So, I checked the status code.
Reviewed By: markbt
Differential Revision: D8418363
fbshipit-source-id: 0c8f0bf55f588822e50395c3ec8bfa8d3188ecf7
Summary:
D8410266 introduces the use of `chrono_literals` namespace, which requires
C++14. Since we don't use C++14 otherwise, let's not use it here as well.
Reviewed By: farnz
Differential Revision: D8418398
fbshipit-source-id: c2243c2cdf51587cda698bf7c2e96691cea2d305
Summary:
Avoid transient partially synced stated by updating the bookmarks and
obsmarkers as part of the pull command, if there is one. Additionally,
prevent remotenames bookmarks from being updated in cloud-sync pulls that
happen to include public commits. That's preferable, as usually this will only
be a subset of remote bookmarks, and this can give confusing results.
Reviewed By: DurhamG
Differential Revision: D8401117
fbshipit-source-id: 5ee646d8c7e49ea70d3ec5fd7df5c9009de7bcdd
Summary:
We get the list of obsmarkers to sync after we have pushed the commits. In
that time, new obsmarkers could have shown up, which shouldn't be synced until
the next sync operation.
Improve this by getting the obsmarkers just after we get the heads and
bookmarks. This still has a race, but the window is now much smaller.
It also means we can ensure we sync even when all we have is some extra
obsmarkers.
Reviewed By: DurhamG
Differential Revision: D8397739
fbshipit-source-id: e29890aa250d58bfc3c3c2a056a01d287c053605
Summary:
This fixes how DatapackStore::refresh() computes if it needs to perform a
refresh.
Previously this code was using `clock()`, which returns the amount of CPU time
used by the program. This seems like an undesirable metric to use: the amount
of CPU used by the current program doesn't seem like it should be relevant to
whether the files on disk need to be re-examined or not. If the program has
slept for an hour we probably still want to refresh the files even though no
CPU time has been consumed.
As a result of how the computation was performed, markForRefresh() also would
not trigger a refresh if the program has used less than 100ms of CPU time so
far.
This was breaking some of the eden tests which create a repository pull some
packs into it, and then immediately refresh the store. If all of this happens
in less than 100ms the DatapackStore code would not actually refresh the
store.
Reviewed By: chadaustin
Differential Revision: D8410266
fbshipit-source-id: f08d317b261ccfe89cff45bf3e511a2b87cf9fb5
Summary: This allows us to dogfood the new treestate.
Reviewed By: markbt
Differential Revision: D7912559
fbshipit-source-id: bfb30cd9989460b0eae859aeed402bd92776ab0a
Summary: This makes it easier to support format version 2 - treestate.
Reviewed By: markbt
Differential Revision: D8394003
fbshipit-source-id: ef341fc6d5dd6ba34ad57a98dc3ed63e03d5c1c4
Summary:
Previously, there are 2 configs: `treedirstate.useinnewrepos` and
`format.usetreestate`. They are both related to dirstate format and conflict
with each other. This patch unifies them into a single config
`format.dirstate`.
As we're here, merge `test-fb-hgext-treedirstate-x.t` to `test-dirstate-x.t`
if they were previously copied from `test-dirstate-x.t`
Reviewed By: markbt
Differential Revision: D8393878
fbshipit-source-id: 57abeea22ce732d93205e4d4308923afa90693f4
Summary:
When requesting trees from the server we try to provide a base tree
that we already have. In the hybrid manifest days, when we were fetching
manifest X, we would look up the linknode for X in the flat manifest revlog,
then scan the changelog up and down from that linknode. In the treeonly world we
don't have that revlog anymore though, so the current algorithm just searches from
the tip. This ended up being a really bad algorithm, since if you have tip and you
request tip~1000, then tip~2000, then tip~3000, you end up basing them all
against tip and redownloading a lot of data. This makes hg blame and hg log -p
super slow.
Let's change the algorithm to base off of a hint linknode, and let's change
changectx to set that hint every time it attempts to read a manifest. In manual
testing this significantly sped up hg log -p
Reviewed By: phillco
Differential Revision: D8399613
fbshipit-source-id: 771a94ee8b82be682ea0091b8d6c0fcd5f4e6646
Summary: as titled, so that we can `hg log -T {reviewers} --rev .`
Reviewed By: quark-zju
Differential Revision: D8389944
fbshipit-source-id: b3b194b7027bc831d5eca42dfcb4300745d6eb12
