Commit Graph

13521 Commits

Author SHA1 Message Date
Jun Wu
03e4494e3d lfs: verify sha256 before writing them to local store
Summary:
Verify SHA256 before writing content to the local blob store. This avoids
issues when the remote store does not response with the correct content.

Differential Revision: D10212567

fbshipit-source-id: 5683058670487df53fb14653d83c27da11367e76
2018-10-04 19:49:44 -07:00
Jun Wu
7f10a74837 lfs: add a test demostrating SHA256 hash check does not happen
Summary: As the title.

Differential Revision: D10212568

fbshipit-source-id: 3ad60f93909f5f76c07489bcb046a32f2f93d8f8
2018-10-04 19:49:44 -07:00
Mark Thomas
171b310857 fbamend: move predecessor/successor revsets to core
Summary:
Move the `predecessors` and `successors` revsets to core, and unify the
variants as a single revset.

Before:
* `predecessors` (provided by fbamend) only returned the immediate
  predecessors.
* `successors` (provided by core) returned the given commit and all eventual
  successors.
* `allpredecessors` and `allsuccessors` (provided by fbamend) returned all
  eventual predecessors and successors, not including the original commit.

Now all these revsets work like the old core `successors`, and additionally
have `depth` and `startdepth` parameters which can be used to customize the
behaviour.  This makes them analogous to `ancestors` and `descendants`.

To maintain compatibility with existing users of `allpredecessors` and
`allsuccessors`, these revsets default to a start depth of 1, so they
do not include the original commits.

Reviewed By: quark-zju

Differential Revision: D10028043

fbshipit-source-id: e006700ccf430b7c5d6bd9588253872bce9a18ed
2018-10-03 13:32:56 -07:00
Durham Goode
b3e328c903 treemanifest: don't store repo objects in the manifestrevlogstore
Summary:
Previously the manifestrevlogstore kept a copy of the changelog, the
store, and the svfs from the initial repo object. It's possible that these will
change or be replaced over time, so it's not safe to cache them. Instead let's
just fetch them again each time we need it.

Reviewed By: quark-zju

Differential Revision: D10156425

fbshipit-source-id: ac1e310b51be0377ba9bea75aa1806d256cc5b87
2018-10-03 08:52:19 -07:00
Durham Goode
85a676e406 treemanifest: add test for pushing treeonly commits to a treeonly repo via pushrebase
Summary:
This flow has a bug, so let's introduce a test to expose the break. The
next diff will fix the bug.

Reviewed By: quark-zju, singhsrb

Differential Revision: D10156424

fbshipit-source-id: 51d1d8da6728ee930dc1440cd3e685d7c5b5df2a
2018-10-03 08:52:18 -07:00
Stanislau Hlebik
d93c5269be pushrebase: record pushrebases failed on hooks
Summary:
At the moment they are not recorded because setrecordingparams may not be set
before hooks are called.

Also hooks weren't recorded because pushrebasereplacements weren't correctly
set and KeyError was thrown. This diff fixes it

Reviewed By: quark-zju

Differential Revision: D10147933

fbshipit-source-id: ef33d3e4efa7f3c42e7fcc22ee5f6e54068bf1dc
2018-10-03 04:20:22 -07:00
Stanislau Hlebik
056dad1e1b wireproto: log reponame
Summary:
Let's log reponame as well so that we can tell requests to different repos
apart.

I named config value `common.reponame` so that we can start using it in different extensions that also need reponame (for example, hgsql and infinitepush)

Reviewed By: farnz

Differential Revision: D10127734

fbshipit-source-id: 031c7393dfce43bf47474ac0878c1e68ddabec50
2018-10-02 05:05:41 -07:00
Stanislau Hlebik
96510842e5 clienttelemetry: use clienttelemetry when recording wireproto requests
Summary:
It's useful to know because we can track down slow requests and potentially fix
them on the client.

Reviewed By: farnz

Differential Revision: D10127735

fbshipit-source-id: d21088b50065f322b0c312f048f5828b3d96f99f
2018-10-02 05:05:41 -07:00
Mark Thomas
06a462bb89 fbsparse: add switchprofile command
Summary:
Add the `sparse switchprofile` command, which lets users specify a set of
sparse profiles they would like to switch to.  This is equivalent to `hg sparse
reset` followed by `hg sparse enableprofile`, but in a single step.

Reviewed By: farnz

Differential Revision: D10141938

fbshipit-source-id: 5a1e547d21e3246237fffb17f3b512da18fac804
2018-10-02 03:35:49 -07:00
Zsolt Dollenstein
64d45ccdb6 Format with black 18.9b0
Summary: Reformat all opted-in python code with version `18.9b0` of Black.

Reviewed By: ambv

Differential Revision: D10126605

fbshipit-source-id: 82af0d645dd411ce8ae6b8d239e151b3730cd789
2018-10-01 07:21:42 -07:00
Mark Thomas
c6703f295a fbamend: show a hint when split commits have the same Phabricator Diff
Reviewed By: quark-zju

Differential Revision: D10105739

fbshipit-source-id: 7f2c7664a2e8eb4704ff2e5800c1c24cbedb3fdc
2018-10-01 06:21:59 -07:00
Stanislau Hlebik
5b12e92cc2 pushrebase: record failed pushrebase conflicts
Summary:
In D10023543 stackpush was added, and conflict check can happen earlier. In
that case failed pushrebase wasn't recorded. This diff fixes it.

Reviewed By: quark-zju

Differential Revision: D10119679

fbshipit-source-id: ef12e00a43375151f81a95eec2d9f02db0a9b12b
2018-10-01 04:05:57 -07:00
Jun Wu
df06404eb0 hgsql: correct sqlstrip
Summary: When stripping rev x, revlog revisions with linkrev >= x should be deleted.

Reviewed By: phillco

Differential Revision: D10108592

fbshipit-source-id: 2f9f5663327c4494bd7e836ab24ffc7e507530f4
2018-09-28 16:58:37 -07:00
Jun Wu
d0d8a6a74b hgsql: rename variables to make things more obvious
Summary:
* Rename takelock to dbwritable
  The word "lock" is unclear what lock (local or SQL) it is. Make it clear it's
  all about database writes.

* Rename waitforlock to enforcepullfromdb
  Again, unclear what lock it is. It's also unclear what it does. Rename to
  "enforce pull from db" to make it obvious.

* Rename syncdb to pullfromdb
  "sync" is unclear about what direction to sync. Use "pullfromdb" to make it
  clear.  The hook name is unchanged for compatibility.

Reviewed By: phillco

Differential Revision: D10108594

fbshipit-source-id: fff405e2df9e926f5db436ef74cb5a9aacaebdb4
2018-09-28 16:58:36 -07:00
Mark Thomas
a77d98c8c9 obsshelve: update after shelve transaction has completed
Summary:
Delay updating the working copy until after the transaction that created the
obsshelve commit has completed.  This means we won't update away from the
user's changes until after their work is safe, so if the update is interrupted
for any reason, the user will be able up do a clean update and then unshelve to
get their work back.

Reviewed By: liubov-dmitrieva

Differential Revision: D10102089

fbshipit-source-id: 5709d3915a6c458ba7cfb37ba5e0be5c6e8fcbb2
2018-09-28 09:50:10 -07:00
Mark Thomas
3bfae82702 obsshelve: add test demonstrating dataloss in interrupted shelve
Summary:
If the `update` step is interrupted, then the transaction that created the
obsshelve commit is rolled back, but the update can't be rolled back, so the
files that have been shelved are lost.

Reviewed By: liubov-dmitrieva

Differential Revision: D10102088

fbshipit-source-id: f5bcac5c92069cc6ff3d1d9b7fb7ee507003d8eb
2018-09-28 09:50:10 -07:00
Liubov Dmitrieva
f4858c3c1c add to commit and amend commands ability to show information about the
Summary:
This is for sandcastle and other automated tools.

They will not need to run extra commands like hg log after amend to learn new
hash.

Reviewed By: markbt

Differential Revision: D10101887

fbshipit-source-id: 7c9931776a03f4335bdfe0a19e7d569e3dc4c4ba
2018-09-28 07:23:03 -07:00
Mark Thomas
78e2b44b06 localrepo: devel-warn when accessing repo.vfs directly
Summary:
Callers should prefer repo.localvfs (or perhaps repo.sharedvfs) rather than
calling repo.vfs directly.  Warn when repo.vfs is accessed directly.

Reviewed By: quark-zju

Differential Revision: D9699167

fbshipit-source-id: 83b43fd347da6e68376eaab081b7f6e2d8c5d044
2018-09-28 07:23:02 -07:00
Mark Thomas
372aef5b2d hgsubversion: util develwarn should work without config key
Summary:
In the hgsubversion tests, a develwarn from another component (which doesn't
have a `config` key) crashes with a `KeyError`.  Change to `get` so that
it doesn't crash.

Reviewed By: quark-zju

Differential Revision: D9699155

fbshipit-source-id: c360d5fb5ab7daf7f609d41e0ddfb7456b022666
2018-09-28 07:23:02 -07:00
Mark Thomas
99b3d7c398 share: remove shareutil and _sharedprimaryrepo
Summary:
These are no longer necessary.  Callers should use the sharedvfs to access the
shared repo's .hg directory.

Reviewed By: quark-zju

Differential Revision: D9699163

fbshipit-source-id: 9b9cd584d721c174a7eab06f6abcedc3a943233b
2018-09-28 07:23:02 -07:00
Mark Thomas
b439719a17 hgsubversion: use repo.sharedvfs instead of srcrepo.vfs
Summary:
Update hgsubversion to consistently use repo.sharedvfs, rather than using the
vfs of the srcrepo.

Reviewed By: quark-zju

Differential Revision: D9699165

fbshipit-source-id: 1f4dacdd23ebc3baaa0d09b65e45d4ceeb067559
2018-09-28 07:23:01 -07:00
Mark Thomas
483e3a915c hg: use repo.localvfs instead of repo.vfs
Summary: Update references in core mercurial to use repo.localvfs instead of repo.vfs.

Reviewed By: quark-zju

Differential Revision: D9699162

fbshipit-source-id: 0401677d2b0a1340e66cffb7ee907a0d93aa6717
2018-09-28 07:23:00 -07:00
Mark Thomas
29c915a4c1 bookmarks: use repo.localvfs and repo.sharedvfs instead of repo.vfs
Summary:
Update bookmarks to work correctly when shared.  Rather than copying the
bookmarks file over after transaction handling, update the file in both
locations when bookmarks are shared.

Reviewed By: quark-zju

Differential Revision: D9699158

fbshipit-source-id: 2f75aaac364ffe02e59441ac0f39fb7d8e5d5d2e
2018-09-28 07:23:00 -07:00
Mark Thomas
0b425b5f11 txnutil: add HG_SHAREDPENDING and trysharedpending
Summary:
Add `HG_SHAREDPENDING` which contains the path to the shared primary repository,
similar to how `HG_PENDING` contains the path to the local repository.

Repositories that are not shared check whether either of these refer to the
local repository path.  Repositories that are shared check whether the pending
directory matches their own path, or the shared-pending directory matches their
shared path, via the new `trysharedpending` function.

This fixes the asymmetry in shared repos where pending changes made in a shared
repo were not visible in the primary repo, even though they were visible the
other way around.

Reviewed By: quark-zju

Differential Revision: D9699164

fbshipit-source-id: 31bc5fb2df6e9b9468b6ef39aabf877045c2a011
2018-09-28 07:23:00 -07:00
Mark Thomas
0fa782e83d localrepo: split vfs into localvfs and sharedvfs
Summary:
Split the `repo.vfs` object into two.  When a repo is not shared, these are
both the `.hg` directory of the repo.  When it is shared:

* `repo.localvfs` represents the `.hg` directory of the local repository.
* `repo.sharedvfs` represents the `.hg` directory of the shared primary
  repository.

The old `vfs` is an alias for `localvfs`.  In the future, access through
this name will be deprecated to force callers to think whether they want
the local or shared hg directory.

Reviewed By: quark-zju

Differential Revision: D9699160

fbshipit-source-id: 6600df855c59b6df13e919399192789a873231c6
2018-09-28 07:23:00 -07:00
Jun Wu
77f7a24e4f statprof: stop early if profiling ends early
Summary:
Previously, statprof uses a naive `time.sleep`, which cannot be interruptted
easily (considering Windows compatibility). Change it to `threading.Event.wait`
so it can end early without waiting for the full cycle of the sampling period.

Reviewed By: wez, kulshrax

Differential Revision: D10067234

fbshipit-source-id: 24e5fc0ab05491cb3e7ff34024402842ac3d7d44
2018-09-26 17:52:27 -07:00
Jun Wu
9cf0c87149 hgsql: log rows read and write
Summary:
This gives us more information about how large the hgsql operation is.

As we're here, move `sqlwriteunlock` to an earlier place so the logging is outside
the sql lock. Also document `unbundle` a bit so it's clear `repo.transaction()` only
takes sql lock in `unbundle` context.

Reviewed By: phillco

Differential Revision: D10056307

fbshipit-source-id: 5d3361b4044e6fcf01e60409ef1ecb34da34ccac
2018-09-26 16:35:14 -07:00
Jun Wu
4ddedaf7d2 tests: remove most rocksdb test cases
Summary:
RocksDB and InnoDB are highly compatibile. There is no need to test RocksDB
engine for every hgsql related tests. Only use rocksdb for 2 of the tests.

Reviewed By: phillco

Differential Revision: D10055068

fbshipit-source-id: f9b7ef546fe7d457b0390e49014ebbe56d3c12c1
2018-09-26 14:20:15 -07:00
Phil Cohen
5f9bd1a7da pushrebase: run hgsql sync after running prepushrebase hooks
Summary:
Our hypothesis is that if the prepushrebase hooks take a significant amount of time, the repo state will get out date, causing hgsql to degrade under load.

This uses the previous test to simulate new commits coming in to the database while a single server is busy running prepushrebase hooks. The new code causes a second sync to occur just after running the hooks.

Reviewed By: quark-zju

Differential Revision: D9999683

fbshipit-source-id: 43d2390b476d090a66353555247c9a623386e75a
2018-09-25 18:23:12 -07:00
Phil Cohen
95a45ebb7c pushrebase: refine the "cannot rebase public changesets" test
Summary:
This test case is a bit simpler given the problem -- there's no need for the holding push hook.

Instead, you just need a server that's missing a public commit. (Simulate an `hg strip` on one of the servers. `hg strip` isn't allowed, so create another repo with the extra public commit instead.)

Reviewed By: quark-zju

Differential Revision: D10043378

fbshipit-source-id: 532d8a2791abe5aaa6b6932747c7e0145202e8fe
2018-09-25 18:23:11 -07:00
Jun Wu
83559cfb29 pushrebase: introduce a stackpush fast path
Summary:
The fast path avoids recreating the bundlerepo in the critical section, which
is like a reliable 0.8s win.

See the docstring in stackpush.py for details. It does not replace all use cases
that the old code path supports. So the old path is preserved.

Since it's a drop-in replacement, make it the default.

Reviewed By: phillco

Differential Revision: D10023543

fbshipit-source-id: eaceb9ae5067ab9040aa10cc65170ae54abd3331
2018-09-25 16:06:21 -07:00
Jun Wu
f7475e03c2 globalrevs: wrap repo.commitctx instead of wrapping extensions
Summary:
The next patch will change pushrebase code path so it does not have to go
through pushrebase._commit. Change globalrevs to wrap the lower-level
repo.commitctx to make sure commits have global revs assigned. Pushrebase
and hgsubversion wrappers are removed accordingly.

This also affects "hg commit" running in the server-side repo. Therefore
the test is changed to keep the change minimal.

It's also incorrect to reset the next revision counter at `repo.invalidate`.
It should only be reset at transaction abort. The original concern was
to get an up-to-date view of the revision number. Doing it in `repo.invalidate`
is risky. Enforce hgsql lock when reading the number to make it safer.

As we're here, change the transaction code so it does not wrap `_abort`
unnecessarily for nested transactions.

Reviewed By: singhsrb

Differential Revision: D10023541

fbshipit-source-id: 82d4b57dc2eafa8bc3cdf553e891db6e8c5ff741
2018-09-25 16:06:21 -07:00
Jun Wu
3311dab0ec hgsql: implement a faster version of needsync
Summary:
Inside the critical section, "needsync" runs twice and each time it scans
O(bookmarks + heads), which could be slow.

If the repo is actually up-to-date, there could be a faster path for testing -
just hashing all bookmarks and tip. Note: "heads" is skipped to save some time.
Since commits are append-only, "heads" change implies "tip" movement. So
checking "tip" is enough.

As we're here, fix the empty repo case so syncing is skipped for an empty repo.

Since this is a drop-in replacement for `needsync()[0]`, turn it on by default.

Reviewed By: phillco

Differential Revision: D10022866

fbshipit-source-id: f9e304865db3575515d66444762f21071d5665a3
2018-09-25 16:06:21 -07:00
Jun Wu
3ee1c205cd hgsql: sync repos before acquiring the SQL write lock
Summary:
When entering the critical section (creating a transaction), hgsql might wait
for SQL lock for a long time (minutes). And it currently does nothing during
that wait time. If the lock was acquired after a long time, the work of
catching up with what the database already has might be too much. That is a
waste within the precious hgsql lock held critical section.

Instead of waiting for the lock for N seconds and doing nothing, wait for k
(k < N) seconds and also try to sync with SQL periodically. This makes the
repo closer to what the database has and can reduce work needed to pull from
the database in the critical section.

This adds CPU pressure to the MySQL tier. I added a config option to control
it.  Worse case, we can change `hgsql.syncinterval` to make the sync less
frequent.

As we're here, try document the configs.

Reviewed By: phillco

Differential Revision: D10002578

fbshipit-source-id: bd72d8225c919aa2bc62743de1e1d3f27cba606a
2018-09-25 16:06:20 -07:00
Phil Cohen
a8879d2b36 pushrebase: add a test case demonstrating the "cannot rebase public commit" issue
Summary: We want to add a test for this case before fixing it.

Reviewed By: quark-zju

Differential Revision: D10034448

fbshipit-source-id: 9244c57311d71a0bf1d75643caa31960a2f3519f
2018-09-25 13:21:45 -07:00
Phil Cohen
25bb829e65 pushrebase: add a new test for hgsql cached manifest reads
Summary: We need an in-depth test that tests both pushrebase and hgsql and shows how often we are reading uncached manifests. This is such a test.

Reviewed By: quark-zju

Differential Revision: D10023492

fbshipit-source-id: cf9e76e505c3bc478a7fc13d8d1adf1adeb6c1e4
2018-09-25 13:21:44 -07:00
Mark Thomas
08567ee311 localrepo: don't add storerequirements by default
Summary:
This allows versions that don't know about storerequirements still access newly
created repos with this version.  We will turn this on at a later date.

Reviewed By: singhsrb

Differential Revision: D10033964

fbshipit-source-id: e1065e05c33544d0287eda5eb852baff07c13147
2018-09-25 12:37:57 -07:00
Mateusz Kwapich
28fefbaa65 abstract the write lock
Summary:
I want to centralize knowledge about specific sqllocks so I can change the
locking mechanism in the next commits of this stack

Reviewed By: quark-zju

Differential Revision: D9993477

fbshipit-source-id: 9398476b0ba8c3175ce84a7e0a809bbb8e60b7df
2018-09-25 11:07:04 -07:00
Jun Wu
99245c48eb hgsql: add hgsql.verbose for printing what the server is doing
Summary:
With hgsql.verbose turned on, print useful messages like how many commits the
server is syncing, how long the lock was held or waited, to stderr so they are
visible to the client.  This is useful for the client to understand why the
server "hangs" for a couple of seconds, and is useful in tests.

As we're here, drop no-flake8 comment.

Enable the flag for the hgsql-sync test to verify it.

Reviewed By: phillco

Differential Revision: D10019937

fbshipit-source-id: 8d304ce5208dbc5b92ed20f69daba02e9040c73f
2018-09-24 20:53:08 -07:00
Jun Wu
051e7a99f0 fsmonitor: limit file list being logged to blackbox
Summary:
Sometimes watchman can return a very long list of files. That would make
blackbox rotate too frequently. Truncate that list.

Reviewed By: phillco

Differential Revision: D9997658

fbshipit-source-id: 1314b7c299a5f50ed344e0b85befd4c0525b1da7
2018-09-21 16:51:44 -07:00
Jun Wu
d9394d90ff profiling: allow profiling config to be specified by other sections
Summary:
Previously, the `[profiling]` config section defines the profiler. Now it can be
defined by other sections, `[profiling:1]`, `[profiling:2]`, etc. Those sections
are sorted by name, and the first one with `enabled = true` is effective.

This makes it possible to specify secondary profiling configs, like having a
low-overhead profiler to be always enabled and logged if the command exceeds
certain threshold:

  [profiling:background]
  enabled = 1
  freq = 1
  minelapsed = 30
  output = blackbox
  type = stat

The config will only be used when the primary profiling (`--profile`, or
`profiling.enabled`) is disabled.

Reviewed By: markbt

Differential Revision: D9828548

fbshipit-source-id: 301a2b049d569d6fc065d05ab5b02e85b4ea8cc5
2018-09-21 14:37:23 -07:00
Jun Wu
27c2bc6c0b profiling: make the section name a variable
Summary:
Make "profiling" a variable. No logic change.

The next patch changes the config section name dynamically.

Reviewed By: markbt

Differential Revision: D9828551

fbshipit-source-id: b6b76012462f2dc1a76cd162381d949a94dc498b
2018-09-21 14:37:23 -07:00
Jun Wu
d61bfcb84f legacyui: remove it since the Rust config parser is working fine
Summary:
The Rust config parser is working well. Therefore remove the legacy config
support.

Reviewed By: markbt

Differential Revision: D9840431

fbshipit-source-id: cebb98f74f60a1848dacf9622607249efc7d2099
2018-09-21 14:37:22 -07:00
Jun Wu
1a9c73c079 profiling: add minelapsed config option
Summary:
This config allows us to only log the profiling result if it exceeds a given
threshold. It's helpful for always enabling profiling and only log meaningful
data.

Reviewed By: markbt

Differential Revision: D9828549

fbshipit-source-id: f7e8f7e1e1f834d980519c2a639f3c7fd02f7f42
2018-09-21 14:37:21 -07:00
Jun Wu
8c33c0736a pushrebase: handle tree commits correctly for merges and renames
Summary:
In pushrebase code path, `_getmanifest` needs to be used to obtain the manifest
correctly for pushes from treeonly repos. The merge commit code path missed this
change. Fix it.

Besides, the rename checking code patch can also hit flat manifest errors via
repo.commitctx -> ... -> fctx._copied -> ctx.filenode -> ctx._fileinfo. Patch
that path too.

Reviewed By: phillco

Differential Revision: D9977904

fbshipit-source-id: 8c51aa9cc6bd85b0147d88c61a5af074e36cdb61
2018-09-21 10:26:40 -07:00
Phil Cohen
580e131809 merge: print conflicting paths when rejecting update
Summary: Part of tech debt week; let's make it so any time we reject an update due to conflicts, we print what the conflicts are.

Reviewed By: quark-zju

Differential Revision: D9784659

fbshipit-source-id: 1ee232b728bb602dbeaa4d2ecbc11016edd48dce
2018-09-20 17:21:37 -07:00
Saurabh Singh
a72db7de63 globalrevs: add a configuration option to run in read only mode
Summary:
This commit adds a way to run the `globalrevs` extension only in read
only mode. This is useful in cases where repository `x` pulls from repository
`y`, both the repositories have `globalrevs` and `hgsql` enabled, we only
want to embed `globalrevs` in the commits for repository `y`, and we want to be
able to read the `globalrevs` in repository `x`.

Reviewed By: phillco, quark-zju

Differential Revision: D9978146

fbshipit-source-id: b9530e8e9cfadc28e2cd30e51649cee9e6e144bf
2018-09-20 14:28:07 -07:00
Mark Thomas
d2c1764909 store: encode even longer names further
Summary:
The long filename encoding method introduced in D8527475 is insufficient for
files that have very long names that include some underscores.

For these files, we encode the underscores as `:`.  This character is valid for
filenames on Linux, but won't work on Windows.

Reviewed By: mitrandir77

Differential Revision: D9967059

fbshipit-source-id: 67ea662f48f9fcc40e00b36c86697f3be6aac978
2018-09-20 08:35:09 -07:00
Liubov Dmitrieva
75817a809e commitcloud: make port and TLS options configurable to connect to Commit
Reviewed By: markbt

Differential Revision: D9942698

fbshipit-source-id: b0426498d67cea84d3831ee46a29b32145edb46e
2018-09-20 05:50:53 -07:00
Harvey Hunt
b9debb9853 Add python bindings to BookmarkStore
Summary:
Create Python bindings so that the BookmarkStore can be interacted with
from Python.

Reviewed By: quark-zju

Differential Revision: D9768565

fbshipit-source-id: 00d75b5250d8bc7dbeddd90d80ff4a23c60d00f9
2018-09-20 05:05:08 -07:00
Mark Thomas
a533c07518 commitcloud: revive commits that are visible in the cloud
Summary:
When syncing with the commit cloud, if we have commits that are hidden locally,
but visible in the cloud workspace, and no local obsmarkers waiting to be
synced, revive the cloud-visble commits.

This prevents two repos from oscillating between two views when they have
a conflicting set of obsolescence information that has not been cloud shared.

Reviewed By: liubov-dmitrieva

Differential Revision: D9845984

fbshipit-source-id: 2f44ff8dbc636e2afa56d5efbb3ea5114472c41c
2018-09-19 02:34:56 -07:00
Mark Thomas
216c7fc2c5 commitcloud: add test demonstrating workspace oscillation
Summary:
If two repositories have conflicting obsmarker information that has not been
shared through commit cloud, then they can enter a state where they fight over
the visibility of the commit, and the cloud workspace oscillates between two
views.  This adds a test to demonstrate this occurring.

Reviewed By: quark-zju

Differential Revision: D9845983

fbshipit-source-id: d5c74bed25da5721f273e53df10bf7b88efe2850
2018-09-19 02:34:56 -07:00
Saurabh Singh
c1e276b215 globalrevs: do not wrap extensions if hgsql is bypassed
Summary:
In the current implementation, the `globalrevs` extension complains if
the `hgsql` extension is enabled but bypassed. This creates issues in cases
where the `hgsql` extension is bypassed intentionally as in D5012225. This
commit changes the logic in `globalrevs` extension to not wrap any extensions
if the `hgsql` extension is bypassed intentionally instead of raising an
exception.

Reviewed By: quark-zju

Differential Revision: D9883231

fbshipit-source-id: c57eaad4f661b69752a6457e6ecdd43f377285c1
2018-09-17 13:22:05 -07:00
Mark Thomas
ab55bd31c6 newdoc: fix test failures and warnings
Summary:
Remove newdoc from test-check-code - it's a Sphinx project, so MiniRST
checks don't apply.  Also remove the unused imports from the Sphinx config
file.

Reviewed By: farnz

Differential Revision: D9851834

fbshipit-source-id: c17e83fd96d39423eeedfb8a6cfae382f88c4d5f
2018-09-17 08:56:40 -07:00
Mark Thomas
0ca4acd250 localrepo: add storerequirements feature
Summary:
Add the `storerequirements` feature to the repo.  This means the store may have
a `requires` file, and clients must check it for any store features that they
may be missing.  This allows new requirements to be added that affect the store
when the repo is shared.  Currently there are no store features.

This commit adds support for the feature, and only new repos have the
requirement added.  A future commit will optimistically upgrade repos to
include the requirement.

Reviewed By: quark-zju

Differential Revision: D9699156

fbshipit-source-id: 95c1ab6973d44c02abc69b78a15311fe6a8696fd
2018-09-15 03:22:34 -07:00
Mark Thomas
1b115babba share: add a test demonstrating bad behaviour with requirements
Summary:
The share extension doesn't currently check the requirements of the shared
repo.  This means the shared repo's requirements can't be used to stop older
versions of Mercurial modifying future store versions that they don't
understand.

Reviewed By: quark-zju

Differential Revision: D9699169

fbshipit-source-id: 5a3a6d3389ecd682168f2660b425f4d762771f99
2018-09-15 03:22:34 -07:00
Jun Wu
a9ed32f839 tests: set maxdifflines to 1000 for buck test
Summary:
Buck test does not print outputs directly to the terminal. So it's better to
not limit the diff size.

Reviewed By: singhsrb

Differential Revision: D9842709

fbshipit-source-id: f17c7321d75761fdac21b9def52dec38f7e0b60c
2018-09-14 19:05:35 -07:00
Jun Wu
fb02b06bab fsmonitor: add 'debugrefreshwatchmanclock' command
Summary: See the command docstring for the use-case.

Reviewed By: phillco, strager, singhsrb

Differential Revision: D9836447

fbshipit-source-id: 22ff603ce3e0cd95ea4e400ed24a96c0ea636e6b
2018-09-14 18:25:52 -07:00
Saurabh Singh
e6f4ea31d3 namespaces: make specifying namespace priority mandatory
Summary: This would lead to less surprises than using the default priority.

Reviewed By: quark-zju

Differential Revision: D9818647

fbshipit-source-id: 59214eff84ea0b485ab8ff42b86df1c605a319f5
2018-09-13 16:53:54 -07:00
Saurabh Singh
1030d2b7e5 namespaces: remove redundant name specification in namespace object
Summary:
This is unnecessary after the recent modifications in the namespace
code.

Reviewed By: quark-zju

Differential Revision: D9815973

fbshipit-source-id: d7c6b07226be35ebbbe5354d5c520c9807cc0aa6
2018-09-13 13:59:38 -07:00
Jun Wu
a58e75fae8 lock: unlink temporary symlink placeholder on POSIX
Summary:
On POSIX, a lock is created in these steps:

  1. Take the directory lock to make "test stale lock & unlink" atomic.
  2. Test and remove stale lock.
  3. Create a legacy symlink (placeholder) lock.
  4. Create a temporary file, flock it.
  5. Rename the temporary file to override the symlink lock.

Step 3 is to be compatible with legacy hg programs. However, the error handling
is not great - in case 4 or 5 errored out (ex. Ctrl+C), the symlink file is left
untouched. This diff removes it.

Reviewed By: ikostia

Differential Revision: D9770936

fbshipit-source-id: a004d6326381e304552088bcf322c052254321dd
2018-09-13 12:20:50 -07:00
Mark Thomas
4df24e1861 treemanifest: don't truncate list of candidate manifests
Summary:
`treemanifest._generatepackstream` truncates the list of possible base
manifests to length 2, because `cstore.treemanifest.walksubdirtrees` can only
support two base trees to compare against.  However, it does this too soon, we
can select a different 2 base manifests for each comparison out of the possible
pool.

Reviewed By: quark-zju

Differential Revision: D9788298

fbshipit-source-id: 4ba8753ccc34e09563bebdf90602211a999a02ea
2018-09-13 03:51:34 -07:00
Mark Thomas
361c96f1d3 treemanifest: add test demonstrating compare trees problem
Summary:
This test arranges it so that `treemanifest._generatepackstream` is called with
three possible base manifests, each of which is appropriate for a different
commit.  Because this function truncates the list of candidate base manifests
too soon, we end up with too much data in the bundle.

Reviewed By: quark-zju

Differential Revision: D9788299

fbshipit-source-id: 2147a63108e11750083862abc73c386867549ba2
2018-09-13 03:51:34 -07:00
Jun Wu
1b3e623616 entrypoint: add missing "#!" header
Summary: Otherwise `./entrypoint.py` will be executed as a shell script.

Reviewed By: singhsrb

Differential Revision: D9800130

fbshipit-source-id: 656fb37aae97f1d5757a3f5bba0af701abc84d03
2018-09-12 16:44:28 -07:00
Jun Wu
67db9ced8b test-check-execute: make it a fast test
Summary: `hg files .` is fast. Therefore use it.

Reviewed By: singhsrb

Differential Revision: D9800126

fbshipit-source-id: d22c42599f7a4e923d0261f6cd670dd111965ef9
2018-09-12 16:44:27 -07:00
Jun Wu
aab14173e3 helpers-testrepo: fix the script
Summary:
Broken by D7563731.  Running `exec` in a test will replace the test process.
That means the tests are basically doing nothing. Wrap `exec` in a subshell to
fix it.

Reviewed By: singhsrb

Differential Revision: D9800124

fbshipit-source-id: d4f7f29502dcd2db542f5dd03827a06cc9abd783
2018-09-12 16:44:27 -07:00
Saurabh Singh
6187668183 globalrevs: add namespace to lookup based on global revision numbers
Summary:
We need to look up commits in the repository based on the strictly
increasing global revision numbers. This commit adds the following way of doing
so:

 - `m<number>`: `hg log -r m<number>` would match the commit having `number` as
   the strictly increasing global revision number.

Reviewed By: quark-zju

Differential Revision: D9764819

fbshipit-source-id: b9d3a80852c7acebd0c939a4d4f43c3c9a6cb0ab
2018-09-12 15:09:30 -07:00
Alexandre Marin
cdd06e9ef2 importer - correctly deal with p4 changelist description encoding
Summary:
This fix uses mercurial's encoding.tolocal on the Perforce changelist description
instead of trying to decode it as ascii and ignoring/throwing away non-ascii
characters.

Reviewed By: singhsrb

Differential Revision: D9777614

fbshipit-source-id: 713bd63831c965687b7328de06016b4f351b0469
2018-09-12 14:21:48 -07:00
Jun Wu
b398eaaa6e namespaces: define builtin namespaces with priorities
Summary:
Migrate builtin namespace definition to use the registrar. So we can deprecate
`addnamespace`.

`after` is not expressive enough in certain cases, although it's easier to
understand. Add a `priority` parameter for more explicit ordering handling.

Reviewed By: singhsrb

Differential Revision: D9780363

fbshipit-source-id: 85b9cfb91a7c937eea95f2a279a43c11e561b5fc
2018-09-12 12:35:28 -07:00
Jun Wu
acc7039436 configparser: permit spaces in more cases
Summary:
Be more permissive about spaces. Namely:
- Spaces after a section name like `[foo]    ` are allowed.
- Spaces in config names are allowed.
- Spaces at trailing lines are ignored and no longer insert an `\n` to the previous config.

This makes it closer to the older config parser behavior. But it's still
different on some cases, like `[foo]]`, `[foo] # bar`, `[foo]]` still do not
parse.

Benchmark shows no obvious (within 10%) slowdown. So this is probably fine.

Reviewed By: strager

Differential Revision: D9620253

fbshipit-source-id: 8489ef8e83606d0557db56e8da0a017d55ff1514
2018-09-12 12:05:32 -07:00
Jun Wu
1dc9519dda registrar: add a namespace predicate
Summary:
Previously, there is no easy API to add a namespace with desired order. There
is `extensions.afterloaded`, but that only works for `ui/extsetup`, not for
`reposetup`.

This diff add a namespace predicate so registering a namespace is similar to
registering a revset. It also provides an optional `after` parameter to define
the desired order, like:

  from mercurial import registrar, namespaces

  namespacepredicate = registrar.namespacepredicate()

  namespacepredicate("svn", after=["remotenames"])
  def svn(repo):
      return namespaces.namespace("svn", ...)

Existing namespaces will be migrated by upcoming changes.

This is useful for things wanting to resolve custom names, like globalrevs and
commit cloud.

Reviewed By: singhsrb

Differential Revision: D9775703

fbshipit-source-id: f15827b042d605b9cebc1ee1af6479903f47ee9c
2018-09-11 19:51:56 -07:00
Jun Wu
8a42f77192 test-pushvars: add a test about Python hook
Summary:
The Python hook got "kwargs" without "HG_" prefix. Add an explicit test for it.

Also remove unnecessary configs and change the shell to bash, since we don't
run tests using vanilla "sh".

check-code is updated so it no longer complains about the use of `bash`.

Reviewed By: markbt

Differential Revision: D9561962

fbshipit-source-id: 76a190dde1b0aeb0032a65c173ce6368a28e8cf6
2018-09-11 18:06:13 -07:00
Jun Wu
02ea73b1d5 test-fileset: fix the test on Windows
Summary:
The previous pattern coverts "\\" to "/" too aggressively. Fix the following
test:

  --- test-fileset.t
  +++ test-fileset.t.err
  @@ -50,7 +50,7 @@
     a2
     $ fileset 'a_b'
     $ fileset '"\xy"'
  -  hg: parse error: invalid \x escape
  +  hg: parse error: invalid /x escape
     [255]

Reviewed By: markbt

Differential Revision: D9550579

fbshipit-source-id: db3e32ba90f0c356e5837c692528d546fdd57be4
2018-09-11 18:06:12 -07:00
Saurabh Singh
b0e2154df6 globalrevs: add revsets to look up based on global revision numbers
Summary:
We need to look up commits in the repository based on the strictly
increasing global revision numbers. This commit adds two ways of doing so:

 - `globalrev(number)` revset: `hg log -r 'globalrev(number)'` would match the
   commit having `number` as the strictly increasing global revision number.
 - `m<number>` stringset: `hg log -r m<number>` would also match the commit
   having `number` as the strictly increasing global revision number.

Reviewed By: quark-zju

Differential Revision: D9756146

fbshipit-source-id: db9ecf8f8244634d71957ddb645dfda675cab662
2018-09-11 15:22:26 -07:00
Jun Wu
6f58fc79d4 resolve: enable --root-relative if HGPLAIN is set
Summary:
Some automation actually reads `hg resolve -l` output. D9607987 caused surprise
for them.

Reviewed By: singhsrb

Differential Revision: D9693108

fbshipit-source-id: 68b6efe26d184391924219d56d05f06032441bd2
2018-09-06 16:48:49 -07:00
Phil Cohen
7409181240 tests: remove test-platform.t
Summary: It's still been failing and we probably don't need to worry about maxrss failing any time soon.

Reviewed By: singhsrb

Differential Revision: D9671400

fbshipit-source-id: e847a240285c2aa2fa15d49ec4fb010483278e9d
2018-09-05 19:35:33 -07:00
Liubov Dmitrieva
e638e652e9 commitcloud: add option to select only commits related to the user in the default workspace
Summary:
this is usefull when a user have a lot of other users commits in the default workspace

the set of commits includes all the stacks where at least one commit is user's own

the option is for the default workspace only, it doesn't apply for other workspaces a user may collaborate in

the idea is that the client code will never offer other user's commits to sync, so cloud sync will perfectly work with some extra commit might present on any machine.

this allow users to continue have other users commits in their workspace for review or other purposes even if it is very very large, but provides reliable backup and sync solution for user's own work.

the option works for all commands `hg cloud join` `hg cloud rejoin` `hg cloud sync` and `hg cloud recover`

Reviewed By: simpkins

Differential Revision: D9645571

fbshipit-source-id: 9adb68568083c87c22bb86c19fb7a5bb0f639265
2018-09-05 17:39:31 -07:00
Phil Cohen
a0358711b6 tests: try to make test-platform a bit less flaky
Summary: This test fails from time to time. Let's widen the upper bound for maxrss a bit. If it continues to fail we can just delete the test.

Reviewed By: quark-zju

Differential Revision: D9659411

fbshipit-source-id: ef65d96919ddc809a29c5cd811dfa0bec780e722
2018-09-05 12:10:12 -07:00
Phil Cohen
e5f1622604 tests: add glob to test-fb-hgext-obsshelve.t
Summary:
This test has been flaking out and breaking the build when the shelve was made >1s ago.

cc @[1266426728:Kostia Balytskyi]

Reviewed By: singhsrb

Differential Revision: D9645529

fbshipit-source-id: 575cb4272710d60c27e61aa519da32f4d2291eff
2018-09-04 13:36:25 -07:00
Kostia Balytskyi
4a7c319c2e hg: make shelve not abort on every transaction with treemanifest
Summary:
Before this commit, `treemanifest` registered a post-transaction hook to
prefetch data for the parents of new commits. Shelve produces a hidden
commit, so this logic failed, since it used a filtered repo instance.

Let change it to use unfiltered repo and remove all of the hidden commits
from the revset to avoid surprises.

Reviewed By: markbt

Differential Revision: D9613454

fbshipit-source-id: 5caddd2948e5d2e982462bebdb950b359995baa1
2018-09-03 04:51:49 -07:00
Ryan McElroy
5edc0d3281 resolve: use relative paths with option to override
Summary:
Previously, we would show root-relative paths. Now we show cwd-relative
paths unless the --root-relative flag is passed.

Reviewed By: phillco, farnz

Differential Revision: D9607987

fbshipit-source-id: 6c24c05317c165f88d2a83d6921d1520044a9b6e
2018-09-03 03:05:10 -07:00
Ryan McElroy
92e30ff38c merge: allow "always" continuing when on-failure=prompt
Summary:
When I first rolled this feature out, many people liked it but enough
people complained that it would prompt after EVERY file when sometimes they
wanted to always continue (although they were fine with the initial prompt).

This adds the new "always" option to the prompt.

Reviewed By: quark-zju

Differential Revision: D9607680

fbshipit-source-id: 76989b346ed9dd88048a37bcad6dcddd8325f314
2018-09-03 03:05:10 -07:00
Matt Glazar
aa0316fa51 Fix test-lfs-test-server.t
Summary:
test-lfs-test-server.t is failing because of spurious output from Bash:

```
======================================================================
FAIL: test_lfs_test_server_t (scm.hg.tests.unittestify.hgtests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/users/strager/fbsource/fbcode/buck-out/dev/gen/scm/hg/tests/hg_run_tests#binary,link-tree/scm/hg/tests/unittestify.py", line 96, in runsingletest
    raise self.failureException(err + out)
AssertionError:
 --- test-lfs-test-server.t
+++ test-lfs-test-server.t.err
@@ -158,6 +158,7 @@
 (Restart the server in a different location so it no longer has the content)

   $ $PYTHON $RUNTESTDIR/killdaemons.py $DAEMON_PIDS
+  $TESTTMP.sh: line 116: 2128544 Terminated              lfs-test-server &>lfs-server.log
   $ rm $DAEMON_PIDS
   $ mkdir $TESTTMP/lfs-server2
   $ cd $TESTTMP/lfs-server2
@@ -187,3 +188,4 @@
 Clean up:

   $ $PYTHON $RUNTESTDIR/killdaemons.py $DAEMON_PIDS
+  $TESTTMP.sh: line 140: 2130121 Terminated              lfs-test-server &>lfs-server.log  (wd: ~/lfs-server2)

ERROR: test-lfs-test-server.t output changed
!
Failed test-lfs-test-server.t: output changed
```

If this this test wasn't broken, it would have caught a bug introduced by D9478744 (fixed by D9619464). Fix the failures to make the test pass again.

Reviewed By: quark-zju

Differential Revision: D9619647

fbshipit-source-id: 989db431a28ab2d491ced70dc9113998b0ddb5f5
2018-09-01 13:20:34 -07:00
Jun Wu
610acb415f directaccess: fix revset travesal
Summary:
Previously, revset like

  ('or',
   ('list',
    ('string', '00ff93e1af4dc4ce7effb69afd0b91551cf1fe72'),
    ('func', ('symbol', 'smartlog'), None)))

won't be processed because len(tree) == 2. Change the logic to recursively
enter the tree. I think this is a regression since [revset: remove order information from tree](mercurial-scm.org/repo/hg/rev/1b28525e66982a50c33a7163228afdc785e8ca58).

Reviewed By: ryanmce

Differential Revision: D9615771

fbshipit-source-id: ea3c11b291743ade32357652f3c6790d65b83e5d
2018-08-31 14:36:36 -07:00
Kostia Balytskyi
e86c24cc01 hg: make shelve --patch work without arguments
Summary: Seems like a sensible thing to do

Reviewed By: lukaspiatkowski

Differential Revision: D9613215

fbshipit-source-id: ed7bfa436edcaa988e77c568a3a5c74bacff63a8
2018-08-31 08:20:57 -07:00
Stanislau Hlebik
782da103f1 fix pushrebase recording
Summary:
It was trying to add a recording for non-pushrebase pushes. It did not result
in failure, but it did result in an annoying warning.

Reviewed By: farnz

Differential Revision: D9570530

fbshipit-source-id: de999a60c146cc49a1a5fd057b6037cf9f73e367
2018-08-31 03:52:07 -07:00
Jun Wu
1782010568 directaccess: also deal with strings
Summary:
In revset AST, there are 2 types - string and symbol.

We had users hitting filtered node error by using revset like:

  'only('\''83f5'\'','\''a9ee'\'')'

The extra quoting made the commit hashes "string" instead of "symbol",
and directaccess didn't handle them.

As we're here, also clean up the comments to make it easier to understand.

Reviewed By: singhsrb

Differential Revision: D9564685

fbshipit-source-id: 567e3a1d0b88c3aab269a537f2db6671c4dda95d
2018-08-29 19:20:45 -07:00
Jun Wu
20a01dee08 run-tests: improve reliability killing hang tests
Summary:
Write a test that hangs forever, like:

  $ hg init
  $ hg serve -p 0

`run-tests.py --timeout 3` cannot handle it (on both Windows and Linux). It
only tries to kill the bash process, and the `hg serve` process stays running.

For some reason, existing tests are more likely hang on Windows. Therefore we
need a way to reliably kill processes and their descendants. Windows provides
Job Object [1] which reliably solves the problem.

On POSIX, process group is used. It's less reliable because processes can
escape from the group by using setpgid, etc. But it's already better than
before.

[1]: https://docs.microsoft.com/en-us/windows/desktop/procthread/job-objects#managing-processes-in-jobs

Reviewed By: phillco

Differential Revision: D9516042

fbshipit-source-id: 970fce7ba24d0e9bafd66438354de2e6df761edf
2018-08-27 13:05:30 -07:00
Jun Wu
1fe370b4f9 run-tests: add progress bars
Summary:
Similar to buck. Show progress for running tests.

This also improves Ctrl+C handling. It's now more
responsive since we no longer wait for the entire
output.

Reviewed By: phillco

Differential Revision: D9516040

fbshipit-source-id: 98b29228177510855e29da341198697da940884c
2018-08-27 13:05:30 -07:00
Jun Wu
b14232f56c run-tests: truncate diff output
Summary:
`.t` tests are stateful. This means if some early parts do not execute
as the test expect, the the remaining parts can break in funny ways,
and generate lots of uninteresting output.

Limit the length of the diff output so the "likely" boring parts won't show
up by default. The change can be overridden by `--maxdifflines`.

Reviewed By: phillco

Differential Revision: D9516041

fbshipit-source-id: 6c998cdc73ae4a45e2f4167c22c04df51392da29
2018-08-27 13:05:30 -07:00
Jun Wu
783b7ae902 windows: use MSYS binaries to support run-tests.py
Summary:
MSYS provides binaries that tests need so we no longer
depend on system binaries, which might be broken.

MSYS can provide `make.exe` to make `make local` working,
so we don't depend on other things to provide `make`.

As we're here, change `run-tests.py` to use the right Python
interpreter and escape Windows paths.

Reviewed By: phillco

Differential Revision: D9505926

fbshipit-source-id: f17e77dfc599069d4fe849057b7a198758c2dd42
2018-08-27 11:36:42 -07:00
Liubov Dmitrieva
b4b2a00950 diff allow remote obsoleted revisions to be pulled if missing
Summary:
The goal is to improve backup experience, make repo after recovery to behave as it was for the most popular commands.

This is only if backup is backed by cloud sync because we have more meta information in the repo.

the commands:

`hg diff --since-last-submit`
`hg log -r 'lastsubmitted(.)'`
`hg unamend`

for example, are not working for fresh repos that are recloned because the repo will only contains the latests snapshot of commits and also all obsmarker history but not obsoleted commits.

those are popular commands, so let's make them work.

the standard error there is 'unknown revision' and it is very confusing for users

there is no reason not to pull a remote obsoleted revision into the repo if needed, and there is no reason to ask user to do that manually.

the obstore is still lazy loaded, so shouldn't be perf problems

this diff supports good range of commands, so should make experience for our backup use case users significantly better.

pulling the hidden revisions is also relatively fast and doesn't affect commit cloud state of the repo in any way

Reviewed By: quark-zju

Differential Revision: D9437518

fbshipit-source-id: 01065c642aa9a194f2d321f03c1bd747f57c74b1
2018-08-24 05:51:45 -07:00
Jun Wu
a987ecb9b2 ui: use Rust config parser in Mercurial
Summary:
Switch `ui.load` and `ui.readconfig` to use the Rust config parser.
`ui` now no longer depends on `config.config` or `rcutil`.

Pest's error messages are fancier, thus most test changes.

For the fbsource repo, debugshell shows the new code is 10+x faster:

On laptop:

Before:

  In [1]: %timeit m.ui.ui.load()
  10 loops, best of 3: 27.8 ms per loop

After:

  In [1]: %timeit m.ui.ui.load()
  100 loops, best of 3: 1.85 ms per loop

On devserver:

Before:

  In [1]: %timeit m.ui.ui.load()
  100 loops, best of 3: 16.8 ms per loop

After:

  In [1]: %timeit m.ui.ui.load()
  1000 loops, best of 3: 1.28 ms per loop

Since `ui._rcfg` is no longer copy-on-write, there is concern about `ui.copy()`
performance. It is faster too (on devserver):

Before:

  In [1]: %timeit ui.copy()
  1000 loops, best of 3: 198 µs per loop

After:

  In [1]: %timeit ui.copy()
  10000 loops, best of 3: 157 µs per loop

The old `ui.py` was copied to `legacyui.py` and can replace the new `ui.py` if
a config file exists on the system.  This provides a way to switch back to the
old config parser in case of emergency.

Reviewed By: mitrandir77

Differential Revision: D8887375

fbshipit-source-id: 2951ca622c77bf41187ad5c5cab3445cda0dc519
2018-08-23 17:52:52 -07:00
Jun Wu
07f3d004cc summary: remove branch related logic
Summary:
I noticed `hg summary` takes 32 seconds running in my local repo. Profiling
shows 30 seconds spent on `changelog.findmissing`. We don't use branches and
heavily patched other places to get rid of branch heads logic. So let's remove
them from `hg summary` too.

Reviewed By: phillco

Differential Revision: D9477205

fbshipit-source-id: 17b07190b6dcc96bc3a5f3c2b5ff4aa1366f4904
2018-08-23 17:35:01 -07:00
Saurabh Singh
9f912dccda test-lfs-localstore: make test more robust
Summary:
This test fails on `OSX` with the following error:

```
  $ find $TESTTMP/remote | grep '/../' | sort
+  $TESTTMP/remote
+  $TESTTMP/remote/96
   $TESTTMP/remote/96/85eb765661ea3b95f31e1bb3c3b5501d0c2acdf353feeaa4d8fe32f95f77fb
+  $TESTTMP/remote/ab
   $TESTTMP/remote/ab/861dc170dc2e43224e45278d3d31a675b9ebc34c9b0f48c066ca1eeaed8ee6
```

This commit addresses that issue by only checking for the important paths.

Reviewed By: quark-zju

Differential Revision: D9491589

fbshipit-source-id: 42941b9a7b8ca0f9ac6df1ef85598e78d14e1355
2018-08-23 17:05:39 -07:00
Jun Wu
af0a2ecb8b lfs: add a config to skip writing to localstore
Summary:
We'd like to enable `lfs.url` on hg servers to make blob verification work
without downloading actual blobs. Previously, the LFS local blob store is
always backed by the local filesystem. D6699099 added a simple in-memory store
for another purpose. That in-memory local store fits the
"avoid downloading to disk" purpose. So let's add a config option to use it.

Reviewed By: zhh95

Differential Revision: D9478744

fbshipit-source-id: 69b04cd719cedb77f01b90b4666676c8f38584a1
2018-08-23 11:01:07 -07:00
Jun Wu
d5391483e7 lfs: add a config option to disable LFS object verification
Summary:
The verification also runs client-side and that can cause issues with complex
treemanifest setups.

Therefore turn the verification off by default. It would be enabled on the
server-side.

Reviewed By: phillco

Differential Revision: D9478745

fbshipit-source-id: 299b9180c661023d8e2358eedeae7b5c7d36ab79
2018-08-23 00:35:16 -07:00
Jun Wu
3a1a154516 lock: suggest running debugprocesstree to figure out what's blocking
Summary: This would help people unblock themselves.

Reviewed By: markbt

Differential Revision: D9443602

fbshipit-source-id: f05e2b2390a88a9280149d2164c2d7ab71c29600
2018-08-22 21:35:19 -07:00
Jun Wu
98c4b5d665 debugcommands: add a "debugprocesstree" command
Summary:
Recently there are a trend of questions about repo being locked. People usually
don't have enough debugging skills to find out what's going on. So let's add a
debug command to help them.

The implementation uses `osqueryi`, which perfectly solves the cross-platform
process handling headache. Note the well-known `psutil` Python library does not
seem to provide parent process information on Windows.

The `_rapply` function was backported from upstream mercurial.

Reviewed By: markbt

Differential Revision: D9443601

fbshipit-source-id: d26b3adfde1045ebd5bca1c6b1c93f0db147f9b0
2018-08-22 21:35:19 -07:00
Kaley Huang
61aa294522 fix buck test
Summary: fix flaky buck test

Reviewed By: quark-zju

Differential Revision: D9465687

fbshipit-source-id: d27f1b8ed5d8eb0ffe15aa92400afc92dfd8238a
2018-08-22 12:26:02 -07:00
Mark Thomas
283c1fd469 undo: show a hint when a user might have wanted uncommit or unamend
Summary:
If a user uses `hg undo` to undo a `commit` or `amend` operation, this discards
the changes.  If they wanted to keep them in the working copy, they should have
used `hg uncommit` or `hg unamend`.

Provide a hint telling the user how to get back their changes, and letting them
know that `hg uncommit` and `hg unamend` exist.

Reviewed By: quark-zju

Differential Revision: D9346061

fbshipit-source-id: 300b1ac851a12ccf2432ba2c11fceac619667701
2018-08-22 03:50:55 -07:00
Jun Wu
dc42feebe8 test-run-tests.t: remove warning about entry point
Summary:
The warning was removed by D9436487. This diff should fold into that change.
Unfortunately buck test didn't run with run-tests.py changes. So lets fix it.

Reviewed By: singhsrb

Differential Revision: D9444368

fbshipit-source-id: 8e27ae14d56e559fc063d5500ebb789c4fe26545
2018-08-21 20:04:52 -07:00
Jun Wu
e082921ddc run-tests: remove warning about entry point does not exist
Summary:
It breaks test-runtests.t when running with buck:
https://fburl.com/testinfra/7ui6uffp

Reviewed By: singhsrb

Differential Revision: D9436487

fbshipit-source-id: caa01f5d5c2d60cd0bd197a1d1b6269e64c98df7
2018-08-21 13:35:26 -07:00
Jun Wu
f3718fc48d util: skip checkexec or checklink for wider file systems
Summary:
D9328199 made them skip eden mounts. This diff expands the whitelist to other
common filesystems.

Reviewed By: wez

Differential Revision: D9416599

fbshipit-source-id: 5f1dddf5a181833a2fa6e0954c2579bb5adcd170
2018-08-21 12:21:07 -07:00
Jun Wu
25eb53f3e7 alias: make resolution unaffected by definition order
Summary:
Previously, the alias resolution is affected by the alias definition order. So
things like:

  [alias]
  myglog = log -G
  myvlog = myglog -v

works. But

  [alias]
  myvlog = myglog -v
  myglog = log -G

doesn't. D8767902 changed ordering semantics and broke some people's aliases,
because if both `myvlog` and `myglog` are defined in system hgrc, their order
cannot be changed from a user hgrc.

Instead of having subtle behavior here depending on the order. Let's just do
not rely on the order. This diff makes it so, by resolving aliases using latest
(alias-if-possible) definitions, with only one special case: "foo = foo ..."
uses the original non-alias "foo" command.

The "alias ... shadows command ..." debug message was removed as it's
no longer accurate.

Reviewed By: simpkins

Differential Revision: D9417710

fbshipit-source-id: f4228eba3c8c728163a54bcf053c379fa86bd123
2018-08-21 12:06:18 -07:00
Liubov Dmitrieva
a51873a3fb cloud sync - add an option to specified a head to include
Summary:
this is an option to take into consideration only specified heads that run cloud sync

this is helpful to debug issues with individual stacks on push part of cloud sync

Reviewed By: quark-zju

Differential Revision: D9349821

fbshipit-source-id: c5220be3b30cfc300bf62bdcfc472368ce74104e
2018-08-21 10:51:39 -07:00
Kostia Balytskyi
0598171436 hg: fix linux test failure
Summary:
Generally, it seems the `mercurial/entrypoint.py` approach might need some
more thinking, but in this particular case I'd prefer to just fix the tests.

Here's what's going wrong:
- D9407009 intoduced a logic to expect `mercurial/entrypoint.py` to always
exist under the `hg` installation on a *run-tests* level
- `run-tests.py test-run-tests.t` however runs `run-tests.py` (yo dawg), while
`hg` is a symlink to `$(which true)`
- while the `mercurial/entrypoint.py` script cannot be found by the test
suite in this case, it's not important becuase no real Mercurial is run.
Therefore, I propose to change `SystemExit` to `log`. On one hand, the absense
of `mercurial/entrypoint.py` is a big deal, so we should know about it. On the
other, this case can run completely without it. In the dominant majority of the
other test cases, we'll learn about the absense of `mercurial/entrypoint.py`
by means of the failing Mercurial call.

Generally, this migth require some more thought, but for now let's fix tests.

Reviewed By: lukaspiatkowski

Differential Revision: D9421433

fbshipit-source-id: e39628015ff96f27368b99ec1f99145c0cb1ae7b
2018-08-21 07:51:24 -07:00
Jun Wu
0e88e00ceb commit: auto revive committed commits
Summary: It makes no sense for newly committed commits to be obsoleted.

Reviewed By: DurhamG

Differential Revision: D9236660

fbshipit-source-id: 95e4076bb9acde67aaab95f25a6de27d761cd960
2018-08-20 23:24:50 -07:00
Jun Wu
005a60c6b5 histedit: do not always use real strip on --abort
Summary:
Similar to rebase, do not use unsafe strip for histedit.

This makes `histedit --abort` safer, faster, undo-able, and also solves
potential crashes using real strip with clindex.

Note `test-histedit-obsolete.t` shows a case that is suboptimal. It'll be fixed
by the next commit.

Reviewed By: phillco

Differential Revision: D9236662

fbshipit-source-id: e436b0aa9b4a1b155dcfb111c5618371b68232eb
2018-08-20 18:35:56 -07:00
Jun Wu
159468b58f run-tests: set HGPYENTRYPOINT explicitly
Summary:
The run-tests script symlinks hg to a place and sets `PATH` so `hg` can be
called from anywhere without requiring the local shell settings, including
"dummy-ssh" server side.

The Rust `hgmain` uses
`env::current_exe().parent().join("mercurial/entrypoint.py")` to locate
the `entrypoint.py` file. However, `env::current_exe()` does not follow
symlink on OSX, unlike Linux. Therefore the entrypoint.py cannot be located on
OSX.

`hgmain` also reads `HGPYENTRYPOINT`. So let's set it explicitly from
run-tests.py.

Reviewed By: singhsrb

Differential Revision: D9407009

fbshipit-source-id: db9a55e11b424625a6ab58afdbebf0944c7012f5
2018-08-20 17:05:24 -07:00
Kaley Huang
9d54d78caa verify LFS blobs were uploaded before push
Summary:
per @[1070541862:quark]'s suggestion, I attempt to add a verification hook that checks LFS blobs were uploaded before push.

The code execution path looks like this:
`command.py: push()` -->
`exchange.py: push()` -->
 ......
 `exchange.py: _pushbundle2()`
 `exchange.py: _pushcheckoutgoing()`
 ......

And I insert the hook in `exchange.py: _pushcheckoutgoing()`

Reviewed By: quark-zju

Differential Revision: D9212199

fbshipit-source-id: 98e016da81f9df85731771b7d486d03848624bcb
2018-08-20 16:51:07 -07:00
Jun Wu
b04dda41f1 hint: point people to gitignore QA page
Summary:
As requested by https://fburl.com/qw620t12.

Didn't add another option to make the message configurable as it'll eventually
be removed.

Reviewed By: singhsrb

Differential Revision: D9405366

fbshipit-source-id: 1aef69b4d01be976c3c4465cd542642e586599ca
2018-08-20 13:51:32 -07:00
Pavel Aslanov
8ed850dc2b fixes recording of: 1. correct repo_id. 2. full ontorev
Summary:
This change fixes two problems in pushrebase recording, that will be used to test mononoke pushrebase:
- record correct `repo_id`, it used to be always 0
- `ontorev` truncation, as we store rev as hex string we need 40 bytes instead of 20

Reviewed By: ikostia

Differential Revision: D9398009

fbshipit-source-id: 91add2d75fbb9acf32b99dc952c467d44c2ccbc9
2018-08-20 06:23:20 -07:00
Aida Getoeva
57eaddb3b9 loose files: packing loose files for local data
Summary: Packing loose files for local data into `.hg/store/packs`. Existing pack files for local data can also be repacked.

Reviewed By: DurhamG

Differential Revision: D9181106

fbshipit-source-id: 4d044e97178b4263b87404357445b43eb99017b3
2018-08-20 04:34:48 -07:00
Jun Wu
c601e242af rebase: do not always use real strip on --abort
Summary:
The real strip was used on purpose [1] as a workaround of the old obsstore
design where commits cannot be revived.

Since reviving commits is now possible. Let's avoid dangerous strip here.

This makes `rebase --abort` safer, faster, undo-able, and also solves an
issue where clindex crashes with real strip reported by @[839419353:jeroenv] at
https://fburl.com/uqfglmu5.

[1]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-March/095572.html

Reviewed By: DurhamG

Differential Revision: D9236658

fbshipit-source-id: 9ae7089a8991d8d21ddc5e107c15b8374c7d7339
2018-08-18 10:51:57 -07:00
Jun Wu
51c92b569d hgext: use in-core revive API
Summary: No longer need to check inhibit.

Reviewed By: DurhamG

Differential Revision: D9236656

fbshipit-source-id: d05ec2ed2b2314ef7608c54df4f1653ca8b1b53d
2018-08-18 10:51:57 -07:00
Jun Wu
4791303cd2 obsolete: move commit reviving logic to core
Summary:
Reviving commits is an essential feature. So move it to core.

Most test changes are caused by the auto-date-bump behavior, which is needed to
revive commits automatically.

Note the existing code is not fully ready for the change. For example,
`precursors.get(node)`, `successors.get(node)` do not filter out markers that
are suppressed. For example, some code paths treat node as obsoleted if
`precursors.get(node)` is non-empty. That's no longer true.  markbt's
planned visibility change might clean up this area a bit.

Regarding on tests, most changes are because of the "auto bump date" feature.
The graphlog change in `test-obsmarker-template.t` is because it creates a
cycle, which behaves differently in the new code. Half of obsmarker exchange
tests break. Given the fact that we do not use and will probably rewrite the
exchange algorithm, related tests are deleted, including
`test-obsolete-distributed.t`.

Reviewed By: DurhamG

Differential Revision: D9236661

fbshipit-source-id: 85b983f8bd46dece908c05f56bea2abbc8ffbaf6
2018-08-18 10:51:56 -07:00
Jun Wu
16ae892817 debugrebuilddirstate: setparent early
Summary:
The old code was counting on `dirstate.rebuild` to setparent. However, that
could be too late.

Namely, the sparse extension would expect dirstate p1 to be available to load
the correct sparse profile. So let's set dirstate parent early before calling
dirstate rebuild.

Reviewed By: DurhamG

Differential Revision: D9375196

fbshipit-source-id: baed4a5ab329344ff3bb0b323046b05aece65f53
2018-08-17 11:38:26 -07:00
Jun Wu
b62ff85087 debugrebuilddirstate: read p1 directly
Summary:
The dirstate file consists parents and other metadata. Other metadata might
fail to load but p1 might still be readable. Let's bypass metadata reading logic
and read p1 directly, so we won't get a "null" rev in case only metadata (or
related treestate) gets corrupted.

Reviewed By: DurhamG

Differential Revision: D9375197

fbshipit-source-id: f44d622440125f373ebe09cd0032f05c69417aec
2018-08-17 11:38:26 -07:00
Jun Wu
f5f5c3107f blackbox: ignore more exception from dirstate
Summary:
Dirstate can raise IOError and maybe other errors. Update blackbox to ignore
them.

Reviewed By: DurhamG

Differential Revision: D9375198

fbshipit-source-id: bb81f3560b8fcdca5856977971753f56bd401cc8
2018-08-17 11:38:26 -07:00
Jun Wu
35b4a2b556 test-dirstate-rebuild: demonstrate dirstate rebuild problems
Summary:
With treedirstate and treestate, dirstate becomes harder to recover. The test
demonstrates related problems.

Reviewed By: DurhamG

Differential Revision: D9375195

fbshipit-source-id: a42ea84e810fe6cd9db8412c2a75fffbebdb6dc7
2018-08-17 11:38:26 -07:00
Kostia Balytskyi
a2307a1009 hg: make test-devel-warnings.t compatible with Rust binary
Reviewed By: quark-zju

Differential Revision: D9368559

fbshipit-source-id: 875fadbd4ba4cadb45af697af3f9ab4502d7c2b0
2018-08-17 10:51:47 -07:00
Kostia Balytskyi
ff36c006e6 hg: move hg entrypoint to the mercurial dir
Summary:
We want to have two possible Python entry points for Mercurial:
- pure-python one (`hg/hg` script, the one which is renamed into `hg.real` during FB installation)
- the one to be called from a Rust binary

The reason we can't reuse the `hg/hg` to be called form the Rust binary is because this script will eventually not exist. We also need something in the known location, so we chose to put `entrypoint.py` into `mercurial/`.

This means that both the `hg` script and the `hg.rust` binary need to be able to find the `mercurial/` directory on the system. Traditionally, `hg` contained a `LIBDIR@` string, which was replaced with the parent directory of a `mercurial/` dir at install time. Thus we could install Mercurial in non-standard locations if we needed to. We keep this functionality for the `hg` script.

`mercurial/entrypoint.py` however knows that it lives under `mercurial/` and that `hgext/` and `hgdemandimport/` live alongside `mercurial/`. Thus, the parent dir of `mercurial/` is added as a first item of `sys,path` in `entrypoint.py`.

In case `entrypoint.py` is called from a Rust binary, Python import logic also adds the entire `mercurial/` dir to `sys.path`. This is not desired, since we use `absolute_import` and want to only be able to import things as `from mercurial import smth`. A good example is `json`: even with `absolute_import` enabled `import json` loads `mercurial/json.py` if hg is called as a Rust binary. Therefore, we explicitly remove `mercurial/` from `sys.path`.

Reviewed By: quark-zju

Differential Revision: D9336157

fbshipit-source-id: 22f69e7782d549915c91ef9a0ad0ed29f62a9304
2018-08-17 10:51:47 -07:00
Durham Goode
ef44953866 historypack: make version 1 the default
Summary:
We already have the config set to make version one used in all our main
repositories, but it wasn't being used in all of our tests. Let's change
mutablehistorypack to default to creating version 1 historypacks now.

Reviewed By: quark-zju

Differential Revision: D9367017

fbshipit-source-id: 7d4055455cd8f8e6b2378c5f3a9993b2a40221a8
2018-08-17 09:39:36 -07:00
Durham Goode
c7ff9e5572 pyrevisionstore: add known to getancestors
Summary:
The python getancestors implementation expects a known optional
argument. Unfortunately I can't get rust cpython to support optional arguments,
so for now we make it required.

Reviewed By: singhsrb

Differential Revision: D9367480

fbshipit-source-id: 47dd74e6cbd4fe5a774ebf0949e2114bae15844d
2018-08-17 09:39:36 -07:00
Phil Cohen
3d3d84bea3 dispatch: log raised exception messages and types with ui.log
Reviewed By: DurhamG

Differential Revision: D9363420

fbshipit-source-id: 08fe04b7a72bc14b8f3fccbe1c40ad15d896e5b4
2018-08-16 17:51:25 -07:00
Saurabh Singh
743490c386 globalrevs: allow specifying custom repository name for globalrevs
Summary:
We can have situations where using the repository name from the
`hgsql.reponame` configuration as key for `globalrevs` data in the database can
lead to issues. As an example, for some repo `x`, we usually have a
`x-from-svn` Mercurial repository which gets the commits from svn and then
repository `x` pulls that data from `x-from-svn`. In such a setup, the
`hgsql.reponame` is usually set to `x-from-svn` for the corresponding
repository and similarly for the repository `x`. For the purpose of getting rid
of the backing Subversion repository, we will need to use the `globalrevs`
extension in the `x-from-svn` repository and it makes sense for us to have the
repository name as `x` from the beginning instead of `x-from-svn` from a
`globalrevs` perspective.

This commit just allows specifying a custom repository name to help with
aforementioned situation.

Reviewed By: quark-zju

Differential Revision: D9372695

fbshipit-source-id: 080e08a4a3ab3a4ba7a213e5177e40deb5dd5d88
2018-08-16 16:21:25 -07:00
Bennett Magy
e31dcc3efe Updated parsing logic for .arcrc
Summary:
For some arcrc formats, hg commands prefer a deprecated blob in the .arcrc file which does not contain an oauth token, and only contains a phabricator cert (which is deprecated in favor of oauth)

This updates the logic to look for the phabricator.intern.facebook.com blob in the .arcrc to get an oauth token.

Reviewed By: mitrandir77

Differential Revision: D9351851

fbshipit-source-id: 11efe30b077480caa7b147c45a8d0eb78189277e
2018-08-16 13:50:53 -07:00
Phil Cohen
917604f2f5 tests: fold tglog into tinit; standardize
Summary:
Like D9323267, hg tests commonly reinvent common aliases to render the DAG, and they often differ very slightly. This makes adding a test require more boilerplate, and reading a test in a foreign new area slightly more overhead.

Let's standardize these to reduce the copypasta.

It's necessary to define this as a shell function instead of an hgrc alias to prevent tests that list aliases from printing it. Plus that enforces a nice separation of test/stdlib logic.

Bookmarks and branches are easy enough to add since they're empty if not used. A good number added `{phase}` -- I renamed this to `tglogp`.

Reviewed By: quark-zju

Differential Revision: D9347072

fbshipit-source-id: 6aac7de3e65d2295a7ebecd2ab30901709af3ff1
2018-08-16 10:53:43 -07:00
Durham Goode
8a4d897fc2 historypack: fix check code by using util.safehasattr
Summary: A previous diff broke check code. This fixes it.

Differential Revision: D9355573

fbshipit-source-id: 0eb35678817bab76088a18e976438d5cab7f795a
2018-08-15 18:20:21 -07:00
Durham Goode
3a3cc3e4da pyrevisionstore: add HistoryPack to python unit tests
Summary: Runs the existing historypack unit tests on the rsut implementation.

Reviewed By: markbt

Differential Revision: D9273269

fbshipit-source-id: 14dd3142e6f0ce68ef3c1ee53a42897a2850fdf9
2018-08-15 15:24:40 -07:00
Phil Cohen
d8239f3323 restack: don't exclude obsolete commits from restackonce()
Summary:
This was added by D7067121 by @[1093711665:mitrandir] before @[1070541862:quark] landed the copying of obsmarkers in D7121487. The copied obsmarker means we can rebase obsolete commits correctly now, so don't exclude them.

This fixes the restack test added in the previous commit.

Reviewed By: quark-zju

Differential Revision: D8908156

fbshipit-source-id: 229c8099cbd3e7559e106fd6abb653b387f12469
2018-08-15 11:51:07 -07:00
Phil Cohen
d1d25de9fb restack: add test case for rebasing obsolete children
Summary:
Subsequent changes will fix this.

This test case should assert the bug @[413595:durham] noticed. [1]

Reviewed By: DurhamG

Differential Revision: D9310229

fbshipit-source-id: b033608b78a305b9a12196ab845b7fe9bf23a6f7
2018-08-14 23:20:24 -07:00
Phil Cohen
c6317e40fa tests: add {node|short} to showgraph()
Summary:
I hit a bug where commit hashes were useful, but it only showed up in the `hg rebase` output late in a test.

I think adding hashes to log output is generally a best practice in tests, since it prevents accidental regressions.

Reviewed By: quark-zju

Differential Revision: D9323265

fbshipit-source-id: a76a01ddd416c68fe8a1c440584b21e5d88dc82a
2018-08-14 23:20:24 -07:00
Phil Cohen
d56ff5d1de tests: move showgraph() to tinit.sh
Summary:
It's duplicate in all the fbamend tests, and I want to make a change to it. Rather than duplicate it, let's fold it into `tinit`.

In the future, I'll unify this with tglog, printdag, etc.

Reviewed By: quark-zju

Differential Revision: D9323267

fbshipit-source-id: c1db0f2f22ffadbcb8ad3c0bde329798db06f089
2018-08-14 23:20:23 -07:00
Mark Thomas
0ba172a19e arcdiff: add lastsubmitted revsetpredicate
Summary:
Add a new revsetpredicate, `lastsubmitted(set)`, which maps commits to the last
version that was submitted to phabricator.

Note that this will fail if any commit doesn't have a diff associated with it,
or if the last submitted commit is not in the local repository.

Reviewed By: quark-zju

Differential Revision: D9282952

fbshipit-source-id: 9de8e3196197b2ed106779bb2f58b1f208525e91
2018-08-14 19:27:41 -07:00
Jun Wu
2682422a3e uncommit: fix "restoring deletion" case for treestate
Summary:
The uncommit command does some low-level changes to dirstate - it changes the
parents without updating the tree. Then it uses traditional dirstate APIs to
update the tree.

Treestate and traditional dirstate differ on the "remove" API - treestate would
make it a no-op-ish if the file is not already tracked. Traditional dirstate
would create such an entry blindly and then mark it as "removed".

Treestate cannot simply create the entry because it needs "EXIST_P1" or
"EXIST_P2" information, which is outside the "remove" API itself.

To fix this, teach "uncommit" to write a treestate entry in this case.

The longer term fix would probably be removing the low-level "setparents"
calls.

Reviewed By: DurhamG

Differential Revision: D9282452

fbshipit-source-id: 9a0f0ac2f1bc2a4296c10f95819457f4f9585a82
2018-08-13 08:35:43 -07:00
Jun Wu
c66764acec test-uncommit: add a case where treestate behaves wrong
Summary: Reported by bzawada at https://fburl.com/ji58tnyq.

Reviewed By: DurhamG

Differential Revision: D9282451

fbshipit-source-id: e320383bcb64662ee0e7129fc436aecbb88055c6
2018-08-10 18:25:21 -07:00
Jun Wu
94c114e021 test-configparser: fix failure on Windows
Summary:
The test fails like:

```
    --- test-configparser.t
    +++ test-configparser.t.err
    @@ -21,7 +21,7 @@
       >>> from mercurial.rust import config
       >>> cfg = config.config()
       >>> cfg.readpath("a.rc", "readpath", None, None, None)
    -  ['"$TESTTMP/broken.rc":\n --> 1:2\n  |\n1 | %not-implemented\n  |  ^---\n  |\n  = expected include or unset']
    +  ['"\\\\\\\\\\\\\\\\?\\\\\\\\C:\\\\\\\\cygwin\\\\\\\\data\\\\\\\\sandcastle\\\\\\\\temp\\\\\\\\hgtests.ta2wor\\\\\\\\child187\\\\\\\\test-configparser.t\\\\\\\\broken.rc":\\n --> 1:2\\n  |\\n1 | %not-implemented\\n  |  ^---\\n  |\\n  = expected include or unset']\r (esc)
       >>> cfg.parse("[c]\nx=1", "parse")
       []
       >>> cfg.set("d", "y", "2", "set1")
```

Not sure how `$TESTTMP` works. But let's repalce all of them to glob pattern as
an attempt to fix the test.

See https://fburl.com/sandcastle/pmirvy8a for full log.

Reviewed By: DurhamG

Differential Revision: D9281760

fbshipit-source-id: bb7319511bc8ffecba139bd608f47d7204853076
2018-08-10 18:25:21 -07:00
Jun Wu
b8b7c7a11c test-shelve: stabilize tests
Summary:
They can fail like:

```
    --- test-shelve.t
    +++ test-shelve.t.err
    @@ -1711,7 +1711,7 @@
       rebasing 2:3fbe6fbb0bef "changes to: 1" (tip)
       unshelve of 'default' complete
       $ hg shelve --list
    -  default         (*s ago)    changes to: 1 (glob)
    +  default         (11s ago)   changes to: 1
       $ cd ..
```

Due to space changes.

Reviewed By: phillco

Differential Revision: D9236542

fbshipit-source-id: 4b360454b5eab497cf224fa6eaa4b32ff92f4096
2018-08-10 11:50:03 -07:00
Jun Wu
6422602e0b config: change order semantics
Summary:
Previously, the following config will generate "b", "a" order.

  [section]
  a=1
  b=2
  a=3

It actually makes implementation harder and less efficient (ex. removing an
element from a list is O(N)). The behavior traces back to 2009 [1] where
there is no clue about the above behavior is more desirable.

Let's just change it so the above config will generate "a", "b" order, to
make implementation easier and more efficient.

The old `sortdict` behavior is moved to a different class and used by
http header logic to avoid break them.

[1]: https://www.mercurial-scm.org/repo/hg/rev/fca54469480e

Reviewed By: DurhamG, mitrandir77

Differential Revision: D8767902

fbshipit-source-id: 9fa7baae3b89c3680314e70a674098dd073f5e2b
2018-08-09 21:21:51 -07:00
Jun Wu
709e5ae16c config: drop tweakdefaults in core
Summary: We don't use it and it adds complexity to the rust config migration.

Reviewed By: DurhamG

Differential Revision: D8887369

fbshipit-source-id: 6c3a41088495f869338e80c807a389131218b332
2018-08-09 21:21:51 -07:00
Jun Wu
527ee76ec7 config: remove trusted config handling
Summary:
The "trusted" feature refuses to load config files based on filesystem owner
information. It dates back to 2006 [1]. From the initial discussion [2],
it's mainly for hgweb use-case where `hg serve` runs inside other people's
repo, which is not an important day-to-day use case.

The feature belongs to "complex with little gain" category:

- It does not work for Windows currently. It is difficult to implement on
  Windows.
- Git does not have any protection around this.
- It adds undesirable complexity to the config parser. Now loading order
  matters and the parser has to have a special state recording all trusted
  users and groups so far.
- It adds complexity to the caller - there are extra code handling untrusted vs
  trusted hoooks etc. printing messages.
- File based security does not provide good granularity - imagine a reporc
  enables both a system extension that provides a repo requirement, and an
  untrusted user extension - config file-based security will either load the
  untrusted extension, or fail to read the repo.
- The usecase it protects is relatively rare and whoever hits that would
  probably be a power user that knows things.

Therefore let's just remove the feature at least from the config parser layer.
If the requirement of protecting reading other people's repo is justified in
the future, we can implement repo ownership check properly (with Windows
support).

[1]: 494521a3f1
[2]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2006-October/000056.html

Reviewed By: DurhamG, mitrandir77

Differential Revision: D8767903

fbshipit-source-id: 60e4e0aded57b423e1b5cc5a80ebf873c6f28bcb
2018-08-09 21:21:50 -07:00
Jun Wu
3e4c952d62 configparser: translate byte range to line number at Python land
Summary:
The configsource line number is exposed in a bunch of UI and tests.  Therefore
translate the bytes range to line numbers in the Python wrapper.

The Rust config parser does not track line numbers explicitly, as that adds
complexity to the parser.

The implemenation is inefficient. However, configsource is not used in
common code paths, so it'd be fine.

Reviewed By: DurhamG

Differential Revision: D8960868

fbshipit-source-id: 55e9f5170177e7cb4c8ef55ed17c1f85eb70a192
2018-08-09 21:21:49 -07:00
Jun Wu
cb58cd0d26 configparser: return errors instead of keeping them
Summary:
Change the API to return parse errors directly, instead of keeping them in
ConfigSet struct. This makes it easier to get errors related to one of the
"parse" calls.

Reviewed By: DurhamG

Differential Revision: D8960869

fbshipit-source-id: fbd571f264415e788c5ac44961149d1498826a6d
2018-08-09 21:21:49 -07:00
Jun Wu
a4129f8d53 configparser: use pest to parse config files
Summary:
[pest](https://github.com/pest-parser/pest) is an elegant Rust library for
parsing text.

A navie benchmark on a 1MB config file shows pest is about 1.5 to 2x slower.
But the better error message and cleaner code seems worth it.

Practically, in a VirtualBox VM, parsing a set of our config files takes 3-7ms.
The overhead seems to be opening too many files. Reducing it to one file makes
parsing complete in 2-4ms.

Unfortunately the buck build has issues with the elegant syntax
`#[grammar = "spec.pest"]`, because "spec.pest" cannot be located by pest_derive.
Therefore a workaround is used to generate the parser.

The motivation behind this is because I noticed multi-line value can not be
taken as a plain Bytes slice. For example:

  [section]
  foo = line1
    line2

"foo" should be "line1\nline2", instead of "line1\n  line2". It does not make a
difference on configlist. But it affects templates. Rather than making the
parser more complex, it seems better to just adopt a reasonbly fast parsing
library.

Reviewed By: DurhamG

Differential Revision: D8960876

fbshipit-source-id: 2fa04e38b706f7126008512732c9efa168f84cc7
2018-08-08 17:20:00 -07:00
Jun Wu
4f6c9b1a5e configparser: add a way to clone configs
Summary: Mercurial's ui.py needs a way to copy configs.

Reviewed By: DurhamG

Differential Revision: D8886245

fbshipit-source-id: b936edf5e215ecae078d992a344bcecef7fcd7f3
2018-08-08 17:20:00 -07:00
Jun Wu
ec34cbf739 configparser: expose interface to Python world
Summary: This allows the Python world to access its features.

Reviewed By: DurhamG

Differential Revision: D8790922

fbshipit-source-id: e7e561c86596159c3766d6da8e0834c6fe441cbd
2018-08-08 17:20:00 -07:00
Jun Wu
79ce2d436f hgignore: show hgignore deprecation hint
Summary: This makes it cleaner that we're migrating to gitignore.

Reviewed By: DurhamG

Differential Revision: D9232209

fbshipit-source-id: a1accdb1935f553fd2e52b6367219860b62c9cec
2018-08-08 17:06:11 -07:00
Jun Wu
9740116626 lock: use flock on POSIX
Summary:
We recently ran into issues with locks in pid namespaces [1]. Let's fix that
by using flock.

flock is more reliable in Linux's pid namespace use-case than file-existence
test, because it works without a /proc filesystem and does not have deadlock
issue if an hg process is killed unexpectedly (ex. OOM or SIGKILL).

The transition should be transparent:
- If the new code saw a symlink lock file generated by the old code.
  `open(..., O_NOFOLLOW)` will fail and it's considered lock taken by the old
  process correctly.
- If the old code saw a new lock file. It will treat it as system without
  symlink support and it's considered lock taken by the new process correctly.

A non-symlink stale lock (regardless of whether it contains pid information or not)
will be confidently removed automatically by the new code.

The change is complicated because it works when both new and old hg
run at the same time. If we have migrated most users to the new code path,
the code can be cleaned up significantly.

[1]: https://fburl.com/85fxjisi

Reviewed By: DurhamG

Differential Revision: D9004614

fbshipit-source-id: d501c4f3a7bc8ad73c9556be1c6a265ffd0d0686
2018-08-08 16:20:21 -07:00
Jun Wu
4ed4bcf69a lock: rename locker to lockinfo and clarify its usage
Summary:
The `locker` name is a bit confusing since it has nothing to do with actual locks.
It's just a read-only description.

Rename `locker` class to `lockinfo` and rename `lockerdesc` variable to `info`.
The `info` word comes from `util.makelock(info, path)`.

As we're here, also clean up the docstring.

Reviewed By: DurhamG

Differential Revision: D9004612

fbshipit-source-id: bc57cdd5d3196d0fb80de16a56c1cee0a8fd8486
2018-08-08 16:20:20 -07:00
Jun Wu
2b42573eaa tests: stablize treemanifest-server test by sending stdout to null
Summary:
The order of stdout and stderr is kind of racy. Here is an example build
failure: https://fburl.com/sandcastle/zqt7weh0.

Reviewed By: mitrandir77

Differential Revision: D9203507

fbshipit-source-id: 084c532bc1d7b01c4daebc98e0b4930ba0da3763
2018-08-08 09:20:19 -07:00
Liubov Dmitrieva
1acaf11226 cloud smartlog: fix error with empty result
Summary: we crashed on empty result before this check

Reviewed By: markbt

Differential Revision: D9216307

fbshipit-source-id: 1f5ce5b0c783b25b45d51dbd3bb9f91a2eeee8b9
2018-08-08 06:24:31 -07:00
Durham Goode
e348c3c98c hggit: use transactions during git import
Summary:
Previously hggit used a single transaction for each commit it imported.
This added a lot of over head, such as writing the fncache every time and
committing to sql every time.

Instead, let's use a transaction that lasts a while. We'll tie it to the
existing gitmap save logic, so that we commit commits to disk before writing the
map file (so the map file never references commits that don't exist).

Reviewed By: quark-zju

Differential Revision: D9210162

fbshipit-source-id: 9b1302dfc824055742c4194dfaea7e172e5db6b4
2018-08-07 18:07:08 -07:00
Mark Thomas
a0e7e9f5c2 perfdatapack: fix tests
Summary: Recently-added perfdatapack breaks `test-contrib-perf` and `test-check-code`.

Reviewed By: pkaush

Differential Revision: D9195509

fbshipit-source-id: 023a89bb25ca3b387d6bf5684282910dd62611fe
2018-08-07 08:49:47 -07:00
Jun Wu
a95d4c0118 shallowbundle: fix parent dependency check during unbundling
Summary:
Apparently, the `continue` should apply to the `while` loop, not `for`.
This removes some random unnecessary SSH file fetches during unbundle.

Reviewed By: DurhamG

Differential Revision: D9174383

fbshipit-source-id: 9dad5c9e47d2d59fdc6421efd4bc303e791a6a9f
2018-08-06 23:05:28 -07:00
Jun Wu
394e0c8630 shallowbundle: break cycles when applying deltas
Summary:
There are 2 kinds of dependencies between nodes:

  1. Parents (or copyfrom parent).
  2. Delta-base.

Either of them cannot form cycles. But together they can. Previously they are
considered together, therefore infinite loops can occur. This patch removes
the dependency check for "delta-base" chains to solve the problem.

It hurts unbundle performance in theory as some deltas might be reapplied
multiple times. Practically, I guess nobody unbundles huge bundles in a
shallow repo, so it's probably fine.

Reviewed By: DurhamG

Differential Revision: D9174384

fbshipit-source-id: bbaaddaef4fce942337c64c5b72741eeff7a7a53
2018-08-06 23:05:27 -07:00
Jun Wu
18fc405471 shallowbundle: demonstrate circular node dependency happens
Summary: Without the previous fix, this would be an infinite loop.

Reviewed By: DurhamG

Differential Revision: D9174385

fbshipit-source-id: a2529c687768f73382f9e23984ada7ebcb384b2b
2018-08-06 23:05:26 -07:00
Durham Goode
229cc3de75 treemanifest: add config option to bypass flat block
Summary:
A previous diff added a config option to block serving flat manifests.
We still need flat manifest serving for some of our mirroring infrastructure
(like git and svn mirroring), so let's add a bypass for those.

Reviewed By: quark-zju

Differential Revision: D9182861

fbshipit-source-id: 42d38000509c189dc5c85c5afeffe95f416726e3
2018-08-06 12:51:58 -07:00
Jun Wu
7d346e6bc2 ignore: support global gitignore configs
Summary:
Change the Rust ignore matcher to accept an extra list of gitignore files.
Parse "git:" entries of "ui.ignore" to be git ignore files.

Reviewed By: DurhamG

Differential Revision: D8863905

fbshipit-source-id: 0cd5e29e01f01496ff61c81b89f7876202f18a98
2018-08-02 20:22:47 -07:00
Liubov Dmitrieva
7b92b09010 commitcloud: prototype for hg cloud sl command
Summary:
`hg cloud sl`
`hg cloud ssl`

is the future commands to look at somebody's else smartlog

trying to reuse defined ssl and sl templates

the only real place it uses changeling if hash shortening, so I have to use 'crop' instead

Reviewed By: quark-zju

Differential Revision: D8749430

fbshipit-source-id: e808623ad01d9a04681363bd2ca6d206da0ba1ce
2018-08-02 12:50:44 -07:00
Liubov Dmitrieva
f3937b3937 cloud join - add workspace parameter
Summary:
Add this as an experimental option

We don't have switching of workspaces so far, but hg clone is very fast with eden, so it is easy to create a brand new repo and join to any workspace.

This is already useful for debugging and repro user's issues.

You just join to the user's workspace from a brand new repo and can see what is going on there.

Reviewed By: quark-zju

Differential Revision: D9082157

fbshipit-source-id: 561314cddfced07c81c4bb6c1c2ccf99e140b27c
2018-08-02 05:20:28 -07:00
Jun Wu
d9496d4ad2 racedetector: ignore ignored files
Summary: Changing ignored files should be fine if "-i" is not passed.

Reviewed By: wez

Differential Revision: D9121707

fbshipit-source-id: d11f41cba9d25d8945dff052dd4075150c2b3bfe
2018-08-01 12:21:29 -07:00
Mark Thomas
29cee6aed2 progress: keep cursor in column 1 for fancyrenderer
Summary:
If a user types when the fancyrenderer progress bar is showing, their extra
characters cause output to wrap to the next line, which looks bad.  Instead,
return the cursor to the start of the line each time.

Reviewed By: ryanmce

Differential Revision: D9117862

fbshipit-source-id: 6cd6bfb85ee0626fa1eaca07c7fafdde6961a597
2018-08-01 09:50:10 -07:00
Durham Goode
05a7f9f6e0 osx_tests: fix casefolding test
Summary:
In a previous diff a test was removed which broke the prerequisites for
this osx only test. Let's fix it up.

Reviewed By: mitrandir77

Differential Revision: D9119700

fbshipit-source-id: 9b9e880b9d1c2db73f3b498a20a5d27e666c1dee
2018-08-01 09:50:10 -07:00
Mark Thomas
7f7497d6c9 commitcloud: disambiguate usernames in default workspace names
Summary:
The default workspace name is `user/<shortname>/default`.  This is ambiguous,
as all of:
* fred.bloggs@example.com
* fred.smith@example.com
* fred@example.org
are rendered as `fred`.

This changes the default to use the full email, and provides a new config
option `commitcloud.email_domain`, which defines a single domain that is
dropped.

Reviewed By: liubov-dmitrieva

Differential Revision: D8998491

fbshipit-source-id: e108e116e82d3acc9f1ae1584c63266799fa8cc5
2018-08-01 08:02:35 -07:00
Jun Wu
3766bed594 treestate: migrate old config items
Summary: `format.usetreestate` is no longer used.

Reviewed By: DurhamG

Differential Revision: D9093126

fbshipit-source-id: a509810a4d8464ef4c0d43a19269d6e35e7aaefa
2018-07-31 17:04:50 -07:00
Jun Wu
cb617c8f23 test-rebase-missing-cwd: mark as requiring rmcwd
Summary:
Not all OSes support removing the current directory.
The test is failing on Windows because of that.

Reviewed By: singhsrb

Differential Revision: D9093397

fbshipit-source-id: 84f9f31fd00e769d878be5ddb75ce4c515af97ee
2018-07-31 15:21:55 -07:00
Kaley Huang
6bbd9fbbe5 lfs: upload during push
Summary: during `hg push`, LFS blobs are not uploaded with bundles. this diff adds the extra step [`uploadblobsfromrevs`](https://fburl.com/v05o1l6l) to upload LFS blobs. infinite push [uses a similar approach](https://fburl.com/hwk8yyq4).

Reviewed By: quark-zju

Differential Revision: D9039657

fbshipit-source-id: 2b1651970df1d3761536ef6433d1362756ea3811
2018-07-31 14:50:15 -07:00
Jun Wu
03a0270913 treestate: change getfiltered API to return all matched entries
Summary:
Case-folding could be more complex than what Mercurial currently handles.
Suppose the following paths are committed to a repo using a case-sensitive
filesystem:

  a/a/A
  a/A/a
  A/a/a

Then querying "a/a/a" with a "normpath" filter should ideally have access to
all the above paths.

Unfortunately, the API is changed to use copy instead of references, as it's
impossible to return multiple values borrowed from `&mut self`.

Changes are made on treestate Python land as well to use the new API.  This
solves issues about case-folding corner cases covered by test-eol.t and
test-casefolding.t.

Reviewed By: DurhamG

Differential Revision: D9092405

fbshipit-source-id: 49eb4511ff3c9e5400a522b37126e112c917d2d7
2018-07-31 13:49:35 -07:00
Durham Goode
9bfb2486a7 treemanifest: add treemanifest.blocksendflat config option
Summary:
In order to convert a server repo to treeonly mode, let's first add a
config option that will block even sending flat manifests from a server
repository. This will allow us to find clients who aren't using treeonly mode
before we stop producing flat manifests for good.

Reviewed By: quark-zju

Differential Revision: D9075308

fbshipit-source-id: 1bdce7b55ccb20c2c047939b89571b083d6fba8e
2018-07-31 10:23:21 -07:00
Jun Wu
4337aee636 mq: remove the extension
Summary:
mq is already somehow problematic at D8907646. Without bandwidth supporting
it, let's remove it.

Alternative to mq would be rebase, shelve, unshelve, histedit.

Maintain "--config extensions.mq=" compatibility by marking it builtin so hg4idea
won't break by this change.

Reviewed By: phillco

Differential Revision: D9039741

fbshipit-source-id: a3a1e48a2a982ff8e8b6a6ce659c906a4e2b2b36
2018-07-30 08:49:26 -07:00
Ryan McElroy
0a851d43ba dirsync: do not crash rebase when cwd goes missing
Summary: Replace an ugly hack with an less ugly hack for great winning

Reviewed By: quark-zju

Differential Revision: D9029088

fbshipit-source-id: 1cb4783fbe92ad122e540beb9e32caea0b418054
2018-07-30 08:05:07 -07:00
Ryan McElroy
88baf29979 rebase: show an interesting abort case
Summary:
When the current working directory goes missing during a rebase and
dirsync is enabled, hg aborts badly in the middle of the rebase. This test
reproduces the issue.

Reviewed By: quark-zju

Differential Revision: D9029087

fbshipit-source-id: 61e85205b3f026d6c37b82d9557e47e6a06bbc58
2018-07-30 08:05:07 -07:00
Jun Wu
ba0bb4495f pushrebase: use overlayfilectx
Summary: This allows pushrebase to avoid reading LFS content.

Reviewed By: DurhamG

Differential Revision: D8986782

fbshipit-source-id: 4ae6725d6fe901ec6b33d22d1f6c0f28c2ccfd39
2018-07-27 20:06:08 -07:00
Jun Wu
8b421e16c4 run-tests: do not set lfs.usercache
Summary:
Setting `lfs.usercache` means every LFS object is available globally
for the test. Some tests would like to test pushing LFS objects from
one repo to another. This config hides real issues.

Reviewed By: DurhamG, zhh95

Differential Revision: D8986784

fbshipit-source-id: 3f8d925b28acfe1c6b8ebcb02bd3815642747629
2018-07-27 20:06:07 -07:00
Jun Wu
e1e3cc79e7 filecommit: add a fast path to reuse raw revision data
Summary:
This is a resend of https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-May/097693.html,
with Yuya's comments addressed and some additional comments.

The original commit message is:

> The high-level idea is similar to metadataonlyctx.
> If filelog parents and metadata match, then raw revision data (rawtext,
> rawflags, hash) could be reused. This saves time calculating hash or going
> through flag processors.

Note in the original patch, the amend operation is not using the fast path.
But it uses the fast path today. That's because singhsrb's D636 change
(cmdutil: remove the redundant commit during amend):

```
                    # If the file being considered is not amongst the files
                    # to be amended, we should return the file context from the
                    # old changeset. This avoids issues when only some files in
                    # the working copy are being amended but there are also
                    # changes to other files from the old changeset.
                    if path not in filestoamend:
                        return old.filectx(path)
```

Reviewed By: DurhamG

Differential Revision: D8986780

fbshipit-source-id: 7d18957a713f90eafb786446972020f7939c3671
2018-07-27 20:06:07 -07:00
Jun Wu
af17261b14 treestate: enable by default
Summary:
Enable it by default so all tests run with it.

The test changes are mostly caused by repo requirement changes.

Reviewed By: DurhamG

Differential Revision: D8880991

fbshipit-source-id: f96cecfd85b8088098c3b55d06ab0374ee93437b
2018-07-27 18:04:56 -07:00
Jun Wu
5ee658c927 test-hgext-repogenerator: use treestate
Summary: The test change is caused by treestate having side effect on RNG.

Reviewed By: DurhamG

Differential Revision: D8907647

fbshipit-source-id: 71f4989a137454dd8717a734ee1905aa5b6a42de
2018-07-27 18:04:56 -07:00
Jun Wu
f237fcebc4 test-merge-remove: use treestate
Summary:
The test change indicates that treestate is more correct in flagging a removed
file exited in only p2.

Reviewed By: DurhamG, phillco

Differential Revision: D8842392

fbshipit-source-id: 84a8aba89a34bb52c53ff43b86dea717417dcd24
2018-07-27 18:04:56 -07:00
Jun Wu
f6532949bd test-mq: use treestate
Summary:
I couldn't figure out what's wrong here. Since we plan to eventually deparecate
mq, the test changes are left as-is.

Reviewed By: DurhamG

Differential Revision: D8907646

fbshipit-source-id: ec86d7ddcce6e066f08e50fc6e1f670e1ec96e26
2018-07-27 18:04:56 -07:00
Jun Wu
9dcf7b127d test-sparse: use treestate
Summary:
The test exposes a known issue with ignore handling - unignoring a file
won't get noticed.

Note the unignore handling is broken for a long time since [1].
Treestate+fsmonitor extends the breakage to sparse profiles.

The real fix would be tracking ignored files in treestate, which would
also speed up `status --ignored`. But that feature needs some time to
be implemented correctly.

[1]: https://www.mercurial-scm.org/repo/hg/rev/6dea1701f170

Reviewed By: DurhamG

Differential Revision: D8881046

fbshipit-source-id: 6809696366de98c58d493aa3f49a5db60625de6c
2018-07-27 12:20:24 -07:00
Jun Wu
68d30259d1 test-commit-amend: work with fsmonitor and treestate
Summary: fsmonitor started the transaction before checking commit message.

Reviewed By: DurhamG

Differential Revision: D8880987

fbshipit-source-id: 445628d5b9934baa6c61174f2356b0d564cd6eec
2018-07-27 12:20:24 -07:00
Jun Wu
c63edfcf9a test-locate: use treestate
Summary:
The test reveals the treestate matcher fastpath does not handle patterns like "path:" correctly.
Therefore fix the fastpath and enable treestate for the test.

Reviewed By: DurhamG

Differential Revision: D8839481

fbshipit-source-id: 4824bba83f3cd91f2e1eea965df26e4d02eeccd7
2018-07-27 12:20:24 -07:00
Jun Wu
8093138001 tinyproxy: find free port automatically
Summary:
Previously tinyproxy.py was using `$HGPORT1` as its port, which might be
already taken by another process. If the port was taken, that could cause the
.t test to wait forever therefore the whole test has to time out.

This patch changes tinyproxy to use a free port assigned by the OS, write
the port down to a file so the test knows what port it is.

As we're here, clean up tinyproxy's argv handling so it relies on the option
parser, does not do hacky things on sys.argv, and only listen on localhost.

Reviewed By: DurhamG

Differential Revision: D8884144

fbshipit-source-id: 650ed3c637b136cbaee8280e1033edd0eb3ddc22
2018-07-27 12:05:41 -07:00
Durham Goode
a3ab094cbe treemanifest: prevent pushrebase from processing trees during reply
Summary:
In a previous diff we stopped pushrebase from sending flat manifests
back to treeonly client, but we didn't stop pushrebase from analyzing from
analyzing the manifests to decide what files to send. It turns out that this can
be expensive, since it needs to diff two manifests.

Let's fix up the bundle2 capabilities so that pushrebase knows to not send files
back to the client for a remotefilelog client.

Reviewed By: markbt

Differential Revision: D9021066

fbshipit-source-id: ba422ce8b5ada7905e4900ec68957a84613897ff
2018-07-27 10:36:23 -07:00
Mark Thomas
0f6c096ef1 rage: add profiling of rage steps
Summary:
Some `hg rage` steps may be slow.  Add a profile so we can see which ones these
are.

Reviewed By: ryanmce

Differential Revision: D9027796

fbshipit-source-id: 6e478bee1f1265e4e80c067968da6821c7e071c9
2018-07-27 07:19:53 -07:00
Ryan McElroy
9e51fdef40 context: fix sanity check in metadataonlyctx
Summary:
Previously, this check was subtly wrong, causing it to fail for empty
commits (which have the same manifest as their parents). Update the check to
ensure the actual thing we want to ensure -- that the caller isn't creating
incorrect commits.

The failure would happen in this case:

```
  B  empty commit (manifest Am)
  |
  A  draft commit (manifest Am)
 /
M  master (manifest Mm)
```

When metaediting `A`, `metaedit` would then "rebase" `B` onto `A'` to create
`B'`, with both `A'` and `B'` reusing `Am` as their manifest. The old check
would fail here because the parent of `Am` is `Mm`, but the manifest of `A'`
(the parent of `B'`) is `Am`, so when checking the manifest of the parent
against the parent of the manifest, we would find that `Am != Mm` and fail.

The actual check we wanted was that the manifest of the parent of `B'` is the
same as the manifest of the parent of `B` -- which the old check approximated
as long as no manifests were reused in the stack. As this is not sufficient
with empty commits, we instead want to check that the manifest of the old
parent matches the manifest of the new parent, which we now do.

By avoid reading `manifest.parents`, this would also avoid some lazy tree
fetches, therefore speed up metaedit in certain cases.

Reviewed By: quark-zju

Differential Revision: D9013995

fbshipit-source-id: 4d0b05fc9bb81d115cd87ba2bf98aa253ae6f88b
2018-07-27 02:49:19 -07:00
Ryan McElroy
2b7e9e5e8b context: show how metaedit can crash with empty rebased commits
Reviewed By: quark-zju

Differential Revision: D9013996

fbshipit-source-id: f5d1cfc687981e07e51ef58f07fac17602c6233d
2018-07-27 02:49:19 -07:00
Jun Wu
8be07fec9a treestate: do not gc trees that are modified recently
Summary:
This could avoid issues where there are references to new trees. Trees
are written, but those references haven't been written to disk.

As we're here, also add `ui.log` to log what files are removed.

Reviewed By: markbt

Differential Revision: D8940827

fbshipit-source-id: fbe59a5d87b302c7074f618cbbae56fb1f3d4ff4
2018-07-25 18:21:11 -07:00
Durham Goode
50b2838bd2 check-help: don't write to the repo root
Summary:
This test was writing files 'topic' and 'topic_filtered' into the hg
root directory. Let's write them to $TESTTMP instead

Reviewed By: phillco, quark-zju

Differential Revision: D9005934

fbshipit-source-id: fde38d2ff16245fd529f50bb3bd6631462ed1668
2018-07-25 16:37:18 -07:00
Durham Goode
458dd9e060 packs: move packpath and indexpath into functions
Summary:
A future diff will expose this data from the rust pack implementation,
so we need them to be expoed as functions instead of attributes.

Reviewed By: quark-zju

Differential Revision: D8884705

fbshipit-source-id: 401e5c0d24039c68063d4e9171fefefaf7c2bdff
2018-07-25 11:07:34 -07:00
Durham Goode
2256494edc datapack: add rust datapack to the datapack unit tests
Summary:
Now that the rust data pack is usable from python, let's enable it in
our datapack unit tests. This requires fixing up a few tests.

Reviewed By: farnz

Differential Revision: D8775754

fbshipit-source-id: a6755ca8056169228d8ee2472980f25e3231883e
2018-07-25 11:07:33 -07:00
Stanislau Hlebik
def2302abb pushrebase: record duration as well
Summary: Useful to compare perf

Reviewed By: quark-zju

Differential Revision: D8932499

fbshipit-source-id: 4624222db49f95cdcbb347ffa4eade10f9128225
2018-07-25 02:34:49 -07:00
Kaley Huang
813ff213e6 pushrebase: add a test with lfs enabled
Summary:
enables lfs for pushrebase and adds tests.

using
```
  > [pushrebase]
  > rewritedates = True
```
in the tests so that pushrebase actually changes the hash of the commit

Reviewed By: DurhamG

Differential Revision: D8939281

fbshipit-source-id: a3b95962635bb1f7926889c2869ea79cfca8132d
2018-07-24 09:32:02 -07:00
Saurabh Singh
33ea4e3f9c globalrevs: add tests for commits created through hgsubversion
Summary:
This commit adds basic tests for testing `globalrevs` in the commits
created via the `hgsubversion` extension.

Reviewed By: quark-zju

Differential Revision: D8925893

fbshipit-source-id: 288042d9c93527e96f5ff50069cab98cd45b780b
2018-07-23 18:53:16 -07:00
Saurabh Singh
c5a2f465ca globalrevs: support global revs for commits created using hgsubversion
Summary:
This commit enables embedding global revisions in commits created via
the `hgsubversion` extension.

Reviewed By: quark-zju

Differential Revision: D8925897

fbshipit-source-id: 17f320dd7b6ca560acb78ba8e5ee26a8a3a6b045
2018-07-23 18:53:16 -07:00
Saurabh Singh
49a2709037 test_hgsubversion_util: add optional arguments to the dispatch method
Summary:
This commit just introduces `ui` and `repo` as optional arguments to
the `dispatch` method. This enables testing with custom `ui` and `repo` objects
during the tests.

Reviewed By: quark-zju

Differential Revision: D8925903

fbshipit-source-id: b720bf5fdb94cb08f83c69565b97b81e114a3327
2018-07-23 18:53:15 -07:00
Saurabh Singh
9a9ddae46a hgsubversion: support custom ui during testing
Summary:
Currently, the `hgsubversion` tests use the same `ui` object while
creating the test repository. This commit just refactors and introduces some
methods to ease testing with a custom `ui` object.

This is useful for enabling other extensions together with `hgsubversion`
during the testing.

Reviewed By: quark-zju

Differential Revision: D8925900

fbshipit-source-id: 7c70fe7c9a58b242ab0af10045ce1589c67231b5
2018-07-23 18:53:15 -07:00
Saurabh Singh
52b5158168 hgsubversion: introduce class for interacting with the test database
Summary:
This commit introduces a class for easily connecting with the test
database required for testing `hgsubversion` extension along with `hgsql`.

Reviewed By: quark-zju

Differential Revision: D8925907

fbshipit-source-id: ebf30a17ef71d62a8b95b52454a99d73fb6a1c0e
2018-07-23 18:53:14 -07:00