Summary:
The postincoming checks prints out advice of the following forms:
* `(run 'hg heads' to see heads)`
* `(run 'hg heads' to see heads, 'hg merge' to merge)`
* `(run 'hg heads .' to see heads, 'hg merge' to merge)`
* `(run 'hg update' to get a working copy)`
This advice is no longer useful, so remove it.
Reviewed By: DurhamG, farnz
Differential Revision: D15317185
fbshipit-source-id: 50ba576406c96715fa058399da53462be9b7a3bf
Summary:
We want to migrate the tests to run using treemanifest. As part of
that, we want to first transition to using treemanifest without actually
changing the hash, so we can check that the tests still work first, then update
the hashes second.
This diff adds the flatcompat mode and enables it by default. A future diff will
start enabling treemanifest for existing tests.
Reviewed By: quark-zju
Differential Revision: D15030252
fbshipit-source-id: 06c82be749282d62f1d9cfb43246695c427f8165
Summary:
The test frequently fails with spurious remote errors such as:
remote: ** Unknown exception encountered with possibly-broken third-party extension treemanifest.
remote: ** Please disable treemanifest and try your action again.
remote: ** If that fixes the bug please report it to the extension author.
remote: ** Python 2.7.11 (default, Feb 5 2019, 17:19:11) [GCC 5.x 20180625 (Facebook) 5.5.0+]
remote: ** Mercurial Distributed SCM (version 4.4.2_dev)
remote: ** Extensions loaded: remotefilelog, rebase, simplecache, treemanifest, conflictinfo, debugshell, errorredirect, githelp, mergedriver, progressfile, pushrebase
remote: Traceback (most recent call last):
remote: File "<string>", line 44, in <module>
remote: File "<string>", line 42, in __run
remote: File "/usr/local/fbcode/gcc-5-glibc-2.23/lib/python2.7/runpy.py", line 162, in _run_module_as_main
remote: "__main__", fname, loader, pkg_name)
remote: File "/usr/local/fbcode/gcc-5-glibc-2.23/lib/python2.7/runpy.py", line 72, in _run_code
remote: exec code in run_globals
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/hg.py", line 42, in <module>
remote: entrypoint.run(False)
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/entrypoint.py", line 80, in run
remote: dispatch.run()
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/dispatch.py", line 99, in run
remote: status = (dispatch(req) or 0) & 255
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/dispatch.py", line 393, in dispatch
remote: ret = _runcatch(req)
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/dispatch.py", line 575, in _runcatch
remote: return _callcatch(ui, _runcatchfunc)
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/dispatch.py", line 584, in _callcatch
remote: return scmutil.callcatch(ui, func)
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/scmutil.py", line 135, in callcatch
remote: return func()
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/dispatch.py", line 565, in _runcatchfunc
remote: return _dispatch(req)
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/dispatch.py", line 1353, in _dispatch
remote: lui, repo, cmd, fullargs, ui, options, d, cmdpats, cmdoptions
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/dispatch.py", line 1053, in runcommand
remote: ret = _runcommand(ui, options, cmd, d)
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/dispatch.py", line 1365, in _runcommand
remote: return cmdfunc()
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/dispatch.py", line 1350, in <lambda>
remote: d = lambda: util.checksignature(func)(ui, *args, **strcmdopt)
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/util.py", line 1337, in check
remote: return func(*args, **kwargs)
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/commands/__init__.py", line 5635, in serve
remote: s.serve_forever()
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/sshserver.py", line 107, in serve_forever
remote: while self.serve_one():
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/sshserver.py", line 139, in serve_one
remote: self.handlers[rsp.__class__](self, rsp)
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/sshserver.py", line 88, in sendstream
remote: for chunk in gen:
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/bundle2.py", line 710, in getchunks
remote: self._getcorechunk(), self._compopts
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/bundle2.py", line 732, in _getcorechunk
remote: for chunk in part.getchunks(ui=self.ui):
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/bundle2.py", line 1165, in getchunks
remote: pycompat.raisewithtb(exc, tb)
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/bundle2.py", line 1141, in getchunks
remote: for chunk in self._payloadchunks():
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/bundle2.py", line 1181, in _payloadchunks
remote: chunk = buff.read(preferedchunksize)
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/util.py", line 2062, in read
remote: for chunk in self.iter:
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/mercurial/util.py", line 2033, in splitbig
remote: for chunk in chunks:
remote: File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/scm/hg/hg#link-tree/edenscm/hgext/treemanifest/__init__.py", line 2255, in _generatepackstream
remote: for subname, subnode, subtext, x, x, x in subtrees:
remote: RuntimeError: std::exception
They don't seem to affect what the test expect (missing node on the server), so
let's disable these
Reviewed By: DurhamG
Differential Revision: D14446457
fbshipit-source-id: 0ecb114bfcdeef1a3fcfef5a927f725401f90d4e
Summary:
Move the strip extension to core. Rename the command to `hg debugstrip` as it
is not intended for use by users. Users should use `hg hide` instead.
Reviewed By: quark-zju
Differential Revision: D14185822
fbshipit-source-id: ef096488cb94b72a7bb79f5bf153c064e0555b34
Summary:
We're encountering an issue that I think is caused by invalid data
coming back from memcache, possibly due to our recently introduced connection
reusing. Let's add some checksums to verify that the data we put in is identical
to the data we get out for a given key.
Reviewed By: kulshrax
Differential Revision: D14141683
fbshipit-source-id: 206b51b862db7d54def02f5310b90f473d5a0d03
Summary:
simplecache extension support both memcache and local disk as a cache store.
we are going to use memcache on hg servers
getting rid of the caches on disks will save 966GB of disk space per machine.
P60876105
this will also allow hg machines to share the same cached data
Reviewed By: DurhamG
Differential Revision: D13927163
fbshipit-source-id: 3155c61da4a82dd7a790a8ccf8dbc701fe3957e1
Summary:
D13853115 adds `edenscm/` to `sys.path` and code still uses `import mercurial`.
That has nasty problems if both `import mercurial` and
`import edenscm.mercurial` are used, because Python would think `mercurial.foo`
and `edenscm.mercurial.foo` are different modules so code like
`try: ... except mercurial.error.Foo: ...`, or `isinstance(x, mercurial.foo.Bar)`
would fail to handle the `edenscm.mercurial` version. There are also some
module-level states (ex. `extensions._extensions`) that would cause trouble if
they have multiple versions in a single process.
Change imports to use the `edenscm` so ideally the `mercurial` is no longer
imported at all. Add checks in extensions.py to catch unexpected extensions
importing modules from the old (wrong) locations when running tests.
Reviewed By: phillco
Differential Revision: D13868981
fbshipit-source-id: f4e2513766957fd81d85407994f7521a08e4de48
Summary:
This fixes LFS compatibility with packlocaldata.
Also drop the config as the default version is 1 now.
Reviewed By: DurhamG
Differential Revision: D13469486
fbshipit-source-id: 1dc4a1051667419d7aab97bf95f93cacd166468a
Summary:
This makes tests closer to production setup and removes a bunch of "saved
backup bundle to ..." messages.
With D9236657, this should not hurt server-side performance.
Unfortunately a lot tests cannot be migrated easily, mostly because revision
numbers are used. They are left with a TODO.
Reviewed By: DurhamG
Differential Revision: D9237694
fbshipit-source-id: c993fce18f07aba09f6d70964e248af8d501575a
Summary:
On the server, once we enter treeonly mode we still want to be able to
strip trees from the revlog.
Reviewed By: quark-zju
Differential Revision: D10248895
fbshipit-source-id: 15faec870370d4c28f6a83058eb1bf19cb414d29
Summary:
There's a bug where strip in a treeonly server will not strip the tree
revlogs. Let's add a test so a later fix can show the impact.
Reviewed By: quark-zju
Differential Revision: D10248897
fbshipit-source-id: f45e5abc59eeb9398ba17dd927a5f8eb50bc51af
Summary:
As we transition to treeonly, let's be sure every push contains a
manifest for every commit. To do so, let's add a hook that checks this.
Reviewed By: quark-zju
Differential Revision: D10244912
fbshipit-source-id: 6c2023b1cba5663ed386888a2007a005bc02f688
Summary:
There's an issue now where if a client is treeonly but does not have
sendtrees enabled, it could push commits to the server without manifest data.
Let's add a test for this.
The next commit will introduce a hook to catch this case.
Reviewed By: quark-zju
Differential Revision: D10244913
fbshipit-source-id: 2d6c3081c7190831e0368b0dab472d14e893471f
Summary:
The order of stdout and stderr is kind of racy. Here is an example build
failure: https://fburl.com/sandcastle/zqt7weh0.
Reviewed By: mitrandir77
Differential Revision: D9203507
fbshipit-source-id: 084c532bc1d7b01c4daebc98e0b4930ba0da3763
Summary:
A previous diff added a config option to block serving flat manifests.
We still need flat manifest serving for some of our mirroring infrastructure
(like git and svn mirroring), so let's add a bypass for those.
Reviewed By: quark-zju
Differential Revision: D9182861
fbshipit-source-id: 42d38000509c189dc5c85c5afeffe95f416726e3
Summary:
In order to convert a server repo to treeonly mode, let's first add a
config option that will block even sending flat manifests from a server
repository. This will allow us to find clients who aren't using treeonly mode
before we stop producing flat manifests for good.
Reviewed By: quark-zju
Differential Revision: D9075308
fbshipit-source-id: 1bdce7b55ccb20c2c047939b89571b083d6fba8e
Summary:
When requesting trees from the server we try to provide a base tree
that we already have. In the hybrid manifest days, when we were fetching
manifest X, we would look up the linknode for X in the flat manifest revlog,
then scan the changelog up and down from that linknode. In the treeonly world we
don't have that revlog anymore though, so the current algorithm just searches from
the tip. This ended up being a really bad algorithm, since if you have tip and you
request tip~1000, then tip~2000, then tip~3000, you end up basing them all
against tip and redownloading a lot of data. This makes hg blame and hg log -p
super slow.
Let's change the algorithm to base off of a hint linknode, and let's change
changectx to set that hint every time it attempts to read a manifest. In manual
testing this significantly sped up hg log -p
Reviewed By: phillco
Differential Revision: D8399613
fbshipit-source-id: 771a94ee8b82be682ea0091b8d6c0fcd5f4e6646
Summary:
Previously we weren't including p2 when creating trees, this resulted
in incorrect conversions.
Reviewed By: phillco
Differential Revision: D8152253
fbshipit-source-id: c4f8c79b40532c5162b15032962fbc6a78d44b5f
Summary:
Make the (user-facing) log for prefetching trees say how many commits (or which
commit) it is prefetching for.
When remotefilelog.debug is enabled, print out the manifest identity when
fetching a tree during normal operation.
Reviewed By: quark-zju
Differential Revision: D8056555
fbshipit-source-id: dc0c7bc5c949a0674d5f553661e736bc545134b4
Summary:
We had a bug where the cache store had a value that didn't correspond
with it's key. Let's add the key to the value so we can ensure they are always
paired correctly.
Reviewed By: phillco
Differential Revision: D8065151
fbshipit-source-id: 01be3a3432d1b4f5e9a234b35b8315c37b1869c3
Summary:
Pushrebase previously defaulted to bundlev1. Now that the server can
support v2, let's default to v2.
Reviewed By: quark-zju
Differential Revision: D7591174
fbshipit-source-id: bc5448490ff319543baedf3f5a1aab160a73ed27
Summary:
We want to deprecate the bundlev1 format, so let's start by adding a
develwarn. Later diffs will update the tests to not use v1, then remove v1 as a
supported outgoing bundle entirely.
Reviewed By: quark-zju
Differential Revision: D7591166
fbshipit-source-id: 143ad029bfe4d141f91d6d5077342dfa44ad2944
Summary:
Previously we relied on the list of files created by strip to decide
what directories to strip. It turns out this list wasn't adequate though, since
it's possible for directories to change even when files do not. For example,
during a merge if different files were added on different sides of the merge,
the files don't change during the merge but their containing directory does.
This diff special cases merge commits to make sure all the directories affected
by the merge are included in the strip.
Reviewed By: quark-zju
Differential Revision: D7409156
fbshipit-source-id: 9bf67eefb70189300c29db60d9945a7f608dfdda
Summary:
A prior diff optimized push/pull to not send public trees between
peers, since those trees can be downloaded from the main server. Let's be
careful when sending data to the main server and always send everything.
In the future we should add validation on the server that the received data is
complete, but Mercurial doesn't currently do that today.
Differential Revision: D7296253
fbshipit-source-id: 49513685d19991a70d66da1d734ddae23491ed0c
Summary:
This allows pushing a treeonly pack to a server without using
pushrebase.
Differential Revision: D7295686
fbshipit-source-id: b0bfe4fbb04bc765e57f1db82909fa1ae7b3063b
Summary:
Previously pushrebase pushes would send a tree pack part as well as a
rebase pack part. The tree pack part was ignored. Let's make receiving tree pack
parts on the server an error, and instead make the client not send them if it's
doing a pushrebase push.
A future diff will come back and add support for non-pushrebase push of tree
packs. This diff just makes use sure that the new logic won't run during
pushrebase.
Reviewed By: ryanmce
Differential Revision: D7267802
fbshipit-source-id: cabff989c178afb3706b77a03a299bcf3195f289
Summary:
Previously the server would reject pushes that didn't use pushrebase,
since we relied on pushrebase to recreate the commits with the treemanifests.
Now that the flat-to-tree conversion logic is cleaner, we can use it to do the
conversion during a normal push.
Reviewed By: quark-zju
Differential Revision: D7143453
fbshipit-source-id: c13713e2ff59f1dff3ee1f44b7f8db7a92cfe1de
Summary:
Previously we weren't able to commit right after creating a treeonly
repository. This was caused by the code attempting to read the null tree from
the store, which doesn't exist. The fix is to handle the null case and return an
empty tree instead of trying to look it up in the store. We already do this in a
number of other cases, so this was just a missing one.
Reviewed By: singhsrb
Differential Revision: D6930919
fbshipit-source-id: e227612be2640282eb997f4d563102d86f0be43a
Summary:
A future diff will add tree prefetching during changegroup creation. This
requires access to the tree prefetch function from the shallowbundle class.
let's move the prefetch functions onto the repo object in preparation for that.
Reviewed By: quark-zju
Differential Revision: D6873055
fbshipit-source-id: 18de0ee0f6ab566587509f9e23ebb2e5779ed1c8
Summary:
Now they are unnecessary since `run-tests.py` will set up `PYTONPATH`
correctly.
Differential Revision: D6865042
fbshipit-source-id: ca95314f725968e14349a9d916434aa832c596f9
Summary:
When pushing a treeonly commit to a tree+flat hybrid server, the server
needs to execute the hooks in treeonly mode so we never try to access the flat
manifests. Previously we did this for shell hooks by setting some environment
variables, but we didn't do it for python hooks that ran in process.
This diff makes the python hooks run against a bundle repo that is instantiated
in treeonly mode. No changes to any hooks are required, as the repo object they
are given is already in the correct mode.
Reviewed By: quark-zju
Differential Revision: D6840971
fbshipit-source-id: 9fcb97d972076911b35bccf3f79b60972bcafc33
Summary:
We're seeing high load on the server when many clients are fetching treemanifest
entries. A lot of this stems from the cost of opening revlogs. Let's introduce a
simple on-disk cache that let's us avoid opening revlogs as much as possible.
Reviewed By: quark-zju
Differential Revision: D6797698
fbshipit-source-id: c131aa57be333dcb2e4453c6a1f9f4e24532a7ee
Summary:
Now that fastmanifest and treemanifest are moved, we can moved the
tests.
Test Plan: make local && ./run-tests.py
Reviewers: singhsrb, #mercurial
Reviewed By: singhsrb
Subscribers: singhsrb
Differential Revision: https://phabricator.intern.facebook.com/D6685334
Signature: 6685334:1515526455:57791d7f9d2a3f2a6119c0e79a81b31e63a11c60
