Summary:
The `repos` file contains paths to repos so `hg gc` can scan them. The
current implementation basically writes a line per `hg pull`, and `repos`
can grow unbounded. Let's do a check before appending to that file.
Reviewed By: ryanmce
Differential Revision: D8416210
fbshipit-source-id: 0624bfbd6eb7d71dd749626e96063cb10f55bb7f
Summary: Commit cloud state is stored in the store, so the wlock is not necessary.
Reviewed By: liubov-dmitrieva
Differential Revision: D8121600
fbshipit-source-id: f96f4764e3cf0637cd45a0bc83d1227318ac9083
Summary:
`/` in this case is semantically not a dir seaparator, but a bookmark namespace
separator, so let's use `posixpath`, rather than `os.path` to combine parts.
Reviewed By: StanislavGlebik
Differential Revision: D8477029
fbshipit-source-id: 964b7d3f8a560fb73dfaaa39757c091908af77c5
Summary:
Flat manifest backfilling (which is used when reverting from treeonly
mode) was failing when the client had public heads that didn't exist on the
server (for instance, if they pulled from svn directly or if they had old
commits that had been stripped from the server). This was breaking the pull.
Let's fix it so we never send unknown heads to the server during the backfill
pull.
Reviewed By: phillco
Differential Revision: D8526027
fbshipit-source-id: 8d5bf67d2022dbc24facd0b96e001c8c81a0a2da
Summary: This adds a merge tool to only rebase if there aren't conflicts, which I think will be widely useful. It's for IMM only at the moment.
Reviewed By: DurhamG, quark-zju
Differential Revision: D8493065
fbshipit-source-id: 32ccc0e53af27d275a56bcac80861a421d7abfc6
Summary:
This was added to ease the IMM rollout, but today IMM rarely raises new exception, and this code fires on real errors that should be raised -- you can see here this lets us remove many hacky workarounds.
Some of the exceptions that didn't have carve-outs were:
- interrupted update
- source and destination form a cycle
- rebase in progress
Reviewed By: DurhamG, quark-zju
Differential Revision: D8522466
fbshipit-source-id: b407807ea059a0861fb3caca1db8699242342075
Summary:
This is a quick bug fix for the situation xixixao discovered:
You rebase commit B onto C while having A checked out, and the rebase creates merge conflicts.
In that case, the command aborts complaining of the uncommitted changes. But, the rebasestate isn't properly removed. The user might logically run `hg rebase --abort` to clear it, but that does an
`update` with `force=True` which can cause work to be lost.
This is just a quick bug fix. The right solution is to make an inmemory `mergestate` and not write it at all if rebasing in-memory.
Reviewed By: DurhamG
Differential Revision: D8517289
fbshipit-source-id: a62c99ccb2de1be102f9c23cd737794184c4fde8
Summary:
When building a bundle that will be used for a pull that can't use the linkrev
fastpath, we need to process the manifests of the commits for which we will
send files so that we can correctly set up their linkrevs. Currently this is
sometimes happening for all commits, which can be very slow when pulling a
large number of commits.
First, make it clearer what `LocalFiles` and `LocalTrees` means, and make it
easier to test for trees being local.
Second, when processing manifests looking for files that will be included in
the bundle, don't process non-local trees if we are only interested in local
trees.
Third, correctly determine whether to send all, local, or no trees or files
depending on what combinations of treemanifest and remotefilelog the client and
server support, and what operation is taking place.
Finally, we aren't passing `b2caps` through to `changegroup.makestream` in the
changegroup part generator for bundle2 parts. This means the above test for
whether the client supports treemanifest always failed.
With these combined changes, a pull of a specific public revision (so not
following the fastpath) where the client supports both remotefilelog and
treemanifest, will result in the server using `LocalFiles` and `LocalTrees`,
and therefore not processing the manifests of public commits looking for which
files to send.
Reviewed By: DurhamG, ikostia
Differential Revision: D8446137
fbshipit-source-id: 9a4247af75fe79c1962a29fcb8bf0181868699f4
Summary:
recent refactoring broke user experience if token is not found
not found is actually an error, rather than an empty result for security command
Reviewed By: markbt
Differential Revision: D8489287
fbshipit-source-id: 5ee6e821ce79877b43f93cb86b6935294b9e4a67
Summary: Move the log statements up so it becomes effective for treestate.
Reviewed By: markbt
Differential Revision: D8458183
fbshipit-source-id: af943c46dc3a3b0bd403476112473095d33b3650
Summary:
Let's make it clear for the user what background process is running and how long
I think it is really nice because the user can see if it is stuck and also have direct opportunity to kill it
Reviewed By: markbt
Differential Revision: D8476817
fbshipit-source-id: fefaef91e62b5e6199801a9e010a360e26ee6824
Summary:
Implement gc for treestate. Dirstate writes are protected by wlock. So it's
fine to do a gc inside the write method.
Reviewed By: markbt
Differential Revision: D8394736
fbshipit-source-id: 672087a2c14b855d1ca0449eb5b7d92d65bba418
Summary: Similar to treedirsate, if the file size exceeds configured limit, do a repack.
Reviewed By: markbt
Differential Revision: D8394735
fbshipit-source-id: 21a73cd32b33ed462bab8050c3d42a5e7545d3a4
Summary:
cloud sync:
when it is executed manually it doesn't have any sense to let user wait 30 seconds and look at nothing (there were no any message before 30 sec)
in the world of scm daemon, it is a normal case that cloud sync is already running,
let just tell it to the user and exit
cloud sync:
for background / scm daemon runs timeout is still the same
Reviewed By: markbt
Differential Revision: D8449219
fbshipit-source-id: 2b5c0c6874e23bcd71ca74b2a5231df3ae581e3b
Summary:
inform user that backupdisable command doesn't affect the current running
process if any exists
Reviewed By: markbt
Differential Revision: D8471761
fbshipit-source-id: 338b6d9e0332f5c845f26cbfddbf1d1c89f0ac41
Summary:
cloud sync can trigger another cloud syncs from itself when pull commits
disable it, this is the same setting we use for pullbackup
Reviewed By: markbt
Differential Revision: D8477217
fbshipit-source-id: 656fff11870d05c33aabf284fc9b4070bd5a29ea
Summary:
This fixes a subtle bug related to mergedrivers and IMM.
If your DAG looks like this:
```
dest (new mergedriver code)
|
| A (current commit)
|/
base (old mergedriver code)
```
And if you use IMM to rebase `A` onto `dest`, the wrong thing can happen because parts of `base`'s mergedriver are still loaded.
This only affected certain types of driver changes -- adding or removing parameters between files in the driver, or a driver change that also mutated the working copy (e.g. renaming a file checked by `preprocess()`). And, it only occurred if you had `A` (or some other change between `base` and `dest) checked out.
So, evict the old driver files, and all is well.
The right long-term fix is to probably implement our own Python loader for the mergedriver that reads all driver files from either the `source` or `dest` changectx not `.`. But, that's a bigger change.
Reviewed By: quark-zju
Differential Revision: D8464848
fbshipit-source-id: 8f03a4c99418aa26c3890989d19dd908b026c328
Summary:
implement in the easiest way, we will implement in a better way later
this will reduce confision when we will launch scm daemon
Reviewed By: farnz
Differential Revision: D8450341
fbshipit-source-id: 1c6bedbcf68224dd1e61ef4ec5cd04f26e76453a
Summary:
Regression caused by D7909172. This could trigger slow "status" calls 100%
for treestate, and cause race condition for non-treestate cases.
Reviewed By: phillco, singhsrb
Differential Revision: D8458182
fbshipit-source-id: 6a2e9939d83f55be4b0597b6e15932c9219aac4b
Summary:
This commit simply adds a method for checking if the hgsql extension
is bypassed. This will be used in a later commit for validating the repository
configuration.
Reviewed By: DurhamG
Differential Revision: D8364386
fbshipit-source-id: 70cc08c55339dc4a47ae17b30c131bbd2167e144
Summary:
This commit moves the logic concerning creation of commit during the
graft step in pushrebase to a separate method. This allows for easily wrapping
around the commit to add custom logic. In particular, this can be used to add
extras for the commits that will be created using the pushrebase.
Reviewed By: DurhamG
Differential Revision: D8364383
fbshipit-source-id: 60f845ec3f4f0b4d3799f4385ce49c6fd78cdda2
Summary:
This commit just introduces a method to check whether non pushrebase
commits are blocked. This will be used in later commits.
Reviewed By: DurhamG
Differential Revision: D8364384
fbshipit-source-id: ca0fb39e7925ce8dc7374bf9eb26f9c47506a67c
Summary:
The new treestate was designed to store fsmonitor state. Use it to make
fsmonitor and dirstate state consistent, and avoid fsmonitor state
invalidation.
The "fsmonitor identity" check was removed as we now rely on the dirstate
identity check - dirstate and fsmonitor state must be updated consistently -
both updated or neither updated.
Since this is the first dirstate that tracks "untracked" files, several places
are adjusted (ex. dmap.dropfile, dmap.keys) to take the new untracked files
into consideration.
Reviewed By: wez
Differential Revision: D7909172
fbshipit-source-id: 05fd64b25c67ae4b07bc8cfee2731c748205975e
Summary:
The previous logic restarted the *entire* rebase if one commit had merge conflicts or needed a mergedriver run. This was unnecessarily slow, plus the abort triggered a full repo scan.
Also, this logic is actually simpler and easier to reason about when done here, vs. `rebase()`. We can delete all the wrapping that function does after this rolls out.
This is config-gated, so we can roll it out gradually. Old config = old behavior.
Reviewed By: quark-zju
Differential Revision: D8272566
fbshipit-source-id: 9e598eea10c8a634497cd211a468dd2f58c5ff47
Summary: This just moves the logic around rebasing a single commit into a function, _rebaseone, so we can call it twice in the next diff.
Reviewed By: quark-zju
Differential Revision: D8272564
fbshipit-source-id: 38a15c099bb75d3a6b015fd22453c3732a6b21f5
Summary:
security command normally show a GUI prompt to allow you to type your password if the keychain is locked
if it is unable to do so, it returns 36 and empty stderr
it is ok that it is failed, because it happens due to security reasons,
the problem is only that stderr is empty, so the reason of failure is not clear to user.
I didn't find any option in the man of security command itself. So, I checked the status code.
Reviewed By: markbt
Differential Revision: D8418363
fbshipit-source-id: 0c8f0bf55f588822e50395c3ec8bfa8d3188ecf7
Summary:
D8410266 introduces the use of `chrono_literals` namespace, which requires
C++14. Since we don't use C++14 otherwise, let's not use it here as well.
Reviewed By: farnz
Differential Revision: D8418398
fbshipit-source-id: c2243c2cdf51587cda698bf7c2e96691cea2d305
Summary:
Avoid transient partially synced stated by updating the bookmarks and
obsmarkers as part of the pull command, if there is one. Additionally,
prevent remotenames bookmarks from being updated in cloud-sync pulls that
happen to include public commits. That's preferable, as usually this will only
be a subset of remote bookmarks, and this can give confusing results.
Reviewed By: DurhamG
Differential Revision: D8401117
fbshipit-source-id: 5ee646d8c7e49ea70d3ec5fd7df5c9009de7bcdd
Summary:
We get the list of obsmarkers to sync after we have pushed the commits. In
that time, new obsmarkers could have shown up, which shouldn't be synced until
the next sync operation.
Improve this by getting the obsmarkers just after we get the heads and
bookmarks. This still has a race, but the window is now much smaller.
It also means we can ensure we sync even when all we have is some extra
obsmarkers.
Reviewed By: DurhamG
Differential Revision: D8397739
fbshipit-source-id: e29890aa250d58bfc3c3c2a056a01d287c053605
Summary:
This fixes how DatapackStore::refresh() computes if it needs to perform a
refresh.
Previously this code was using `clock()`, which returns the amount of CPU time
used by the program. This seems like an undesirable metric to use: the amount
of CPU used by the current program doesn't seem like it should be relevant to
whether the files on disk need to be re-examined or not. If the program has
slept for an hour we probably still want to refresh the files even though no
CPU time has been consumed.
As a result of how the computation was performed, markForRefresh() also would
not trigger a refresh if the program has used less than 100ms of CPU time so
far.
This was breaking some of the eden tests which create a repository pull some
packs into it, and then immediately refresh the store. If all of this happens
in less than 100ms the DatapackStore code would not actually refresh the
store.
Reviewed By: chadaustin
Differential Revision: D8410266
fbshipit-source-id: f08d317b261ccfe89cff45bf3e511a2b87cf9fb5
Summary: This allows us to dogfood the new treestate.
Reviewed By: markbt
Differential Revision: D7912559
fbshipit-source-id: bfb30cd9989460b0eae859aeed402bd92776ab0a
Summary: This makes it easier to support format version 2 - treestate.
Reviewed By: markbt
Differential Revision: D8394003
fbshipit-source-id: ef341fc6d5dd6ba34ad57a98dc3ed63e03d5c1c4
Summary:
Previously, there are 2 configs: `treedirstate.useinnewrepos` and
`format.usetreestate`. They are both related to dirstate format and conflict
with each other. This patch unifies them into a single config
`format.dirstate`.
As we're here, merge `test-fb-hgext-treedirstate-x.t` to `test-dirstate-x.t`
if they were previously copied from `test-dirstate-x.t`
Reviewed By: markbt
Differential Revision: D8393878
fbshipit-source-id: 57abeea22ce732d93205e4d4308923afa90693f4
Summary:
When requesting trees from the server we try to provide a base tree
that we already have. In the hybrid manifest days, when we were fetching
manifest X, we would look up the linknode for X in the flat manifest revlog,
then scan the changelog up and down from that linknode. In the treeonly world we
don't have that revlog anymore though, so the current algorithm just searches from
the tip. This ended up being a really bad algorithm, since if you have tip and you
request tip~1000, then tip~2000, then tip~3000, you end up basing them all
against tip and redownloading a lot of data. This makes hg blame and hg log -p
super slow.
Let's change the algorithm to base off of a hint linknode, and let's change
changectx to set that hint every time it attempts to read a manifest. In manual
testing this significantly sped up hg log -p
Reviewed By: phillco
Differential Revision: D8399613
fbshipit-source-id: 771a94ee8b82be682ea0091b8d6c0fcd5f4e6646
Summary: as titled, so that we can `hg log -T {reviewers} --rev .`
Reviewed By: quark-zju
Differential Revision: D8389944
fbshipit-source-id: b3b194b7027bc831d5eca42dfcb4300745d6eb12
Summary: sunshowers hit an exception (either an upgrade race or a module deinitialization) and while it was spurious, it can't hurt to run this logic in a try/catch anyway.
Reviewed By: ikostia
Differential Revision: D8360101
fbshipit-source-id: e581bdeb69e22571f547633b7219d6886cab1aa7
Summary:
If the repo has treemanifests, we can use this to do the case check faster.
Note that with treemanifests we only find the first path component that
conflicts (subsequent conflicts won't be found), as these conflicts occur in
different trees.
Reviewed By: farnz
Differential Revision: D8332426
fbshipit-source-id: 243dab9bf711c6fdb62a0e0ba4346faea3b6b0ad
Summary:
Add a new function to `cstore.treemanifest` to list the contents of a directory
in the tree manifest.
Reviewed By: ryanmce
Differential Revision: D8332425
fbshipit-source-id: db74f11a691cd6b8ac7d51aec6b7da8149c22027
Summary: This will allow us to have json output
Reviewed By: quark-zju
Differential Revision: D8332945
fbshipit-source-id: 3ad39c4df4fe0fbe038d277eabeb18fed10a4bb1
Summary:
Detect the case when we're writing an commit that already exists and skip
writing the history entries.
Reviewed By: DurhamG
Differential Revision: D8363935
fbshipit-source-id: d8db6233eb0261b49e3903c0345c898f2abffa12
Summary:
It's legit for a file name to contain `%s` or `%r` strings. Previously,
blackbox always expect the first argument to be the "format string". That's
inconvenient and it's easy to just pass a string containing the troublesome
`%s` to `ui.log`.
Let's just do not even try formatting strings if there is only one argument,
and if there are multiple arguments, fallback to concatenate them if they
cannot be formatted.
Reviewed By: DurhamG
Differential Revision: D8364130
fbshipit-source-id: 75b2d0e0a460b9a86d4ecd6ecfbb77c0c0fbe98c
Summary:
Add a config option `format.usetreestate`, which would turn on the new
treestate for new repos.
Treestate does not need manually maintaining "copymap", "otherparentset",
"nonnormalset". The one single tree is the source of truth of everything.
So some code like `copymap.pop` etc. are disabled on treestate path.
Reviewed By: markbt
Differential Revision: D7909175
fbshipit-source-id: 96a405fccd4099cefc152ec17204cb5407efbe7e
Summary: return code check was not implemented before but it is important
Reviewed By: markbt
Differential Revision: D8348689
fbshipit-source-id: 69948091832d4c954284e0c999da05c4980d8228
Summary:
Let's roll this out gradually with opsfiles instead.
Also fix an inverted logic test.
Reviewed By: singhsrb
Differential Revision: D8339739
fbshipit-source-id: 2700d4a16ee38280d127e2a4ee7458bdff32f38c
Summary:
Some folks might attempt to load this extension despite it
not being useful with eden. The extension knows not to hook in to
the repo when eden is in used, but the various commands to manipulate
the sparse config do not.
This diff adds a little helper function to show a more reasonable
error message than an ugly stack trace when attempting to use them.
Reviewed By: simpkins
Differential Revision: D8328229
fbshipit-source-id: d7cc4c7047b11f8da96e26ab70bd522b52546151