Summary:
It's just copy-paste from my mutable-history patch series that ended up as
persistent hotfix.
The wrapping logic is very simple: it redirects '--fold' calls to original
metaedit and handles all the rest.
Test Plan:
Included a big copy-pase from test-evolve.t preserving only cases that were
executing metaedit and modified for the smaller test repo (the metaedit tests
were at the end of evolve test file).
Reviewers: #sourcecontrol, quark, simonfar
Reviewed By: quark, simonfar
Subscribers: rmcelroy, quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4708017
Signature: t1:4708017:1489524494:d682aab857e8422bb1a23adc22dddfb9f505b6b3
Summary: New logging will have rotation and will separate different users and repos
Test Plan: arc unit
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4689191
Tasks: 16474976
Signature: t1:4689191:1489172091:d304d4b19ae6cf52f86c11f442fa760b50d1fdf9
Summary:
The upstream has changed how color works. `chistedit` seems to use a very hacky
way to disable color. Let's use a safer and simpler approach.
Test Plan: Run chistedit with hg-committed.
Reviewers: #sourcecontrol, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4702229
Signature: t1:4702229:1489446376:f0c7b4d31eb62bb6aea4cfeb13d78be8c219374c
Summary:
Upstream has switched the type here for better python 3 compatibility. Let's
match their change.
Test Plan: Tests now pass
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4699371
Signature: t1:4699371:1489429276:9da9f64489ecaf94cd77599a0558c081cfe792c2
Summary: The poorly formatted docstring was causing an extra blank line to be inserted both before and after the content
Test Plan: Changed a binary in an hg repo. Committed interactively using the editor. Made sure that there were no blank lines
Reviewers: rmcelroy, quark, simonfar
Reviewed By: simonfar
Subscribers: most, asriram, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4684549
Tasks: 15728734
Signature: t1:4684549:1489269897:faecd381c98e8c2b9884f9bacd18dfc32ee77758
Summary:
For now these command will be used in test because
`wait_for_background_backup.py` tied to the backup state file. And every change
to this file requires change to wait_for_background_backup.py. Let's create
simple internal backup command.
Test Plan: arc unit
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4667056
Tasks: 15389402
Signature: t1:4667056:1489024344:6c43241e42b7c418baa4e1542f303239ba887c45
Summary:
There can be multiple backup processes running at the same time. Since these
processes can be quite heavy it makes sense to limit them. Let's use lock file
to do that.
Test Plan: arc unit
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4667020
Tasks: 15389402
Signature: t1:4667020:1489170813:f2a685b1c224c553d3ee004d89d3eeeca816e824
Summary:
Previously, the rebase logic itself would unlink its state file. Now that the
state file is part of the transaction, it gets serialized at the end of the
transaction, even if it was unlinked midway through the transaction. This
affects fbamend because it wraps various rebases in a higher transaction.
Ideally we would add support to the mercurial transaction framework to allow
marking a file as deleted halfway through the transaction, but this patch will
fix the tests until we get that upstream.
Test Plan: Tests now pass
Reviewers: kulshrax, #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4696578
Signature: t1:4696578:1489348040:b12104b2519d69dab98d2076f640f9f3157036e2
Summary: Upstream has added a new attribute to dirstate. Let's add it too.
Test Plan: Tests now pass
Reviewers: mitrandir, #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4695923
Signature: t1:4695923:1489283439:c7876832af2764d2bbe9cd0f9e47199c84799609
Summary:
Upstream changed the format from ['diff ...\n', 'file1\nfile2\nchunks...'] to be
['diff ...\nfile1\nfile2\n', 'chunks...'] in 199440c7d1f49752. So grepdiff needs
to be updated.
Test Plan: Tests now pass
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4695911
Signature: t1:4695911:1489282635:920e88fecc5d6d4660d1d21289d1d07e26c93315
Summary:
Upstream has added a matcher arg to manifest.filesnotin, so let's support it in
our native treemanifest implementation.
Test Plan: Added unit tests for filesnotin with and without matchers.
Reviewers: #mercurial, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4677030
Signature: t1:4677030:1489076599:23ab67d85bb4fa689f05349253a4d2c4e7d496cc
Summary:
Upstream has added a matcher argument to the diff API which allows diff to avoid
traversing certain parts of the tree. This adds support for that to our native
treemanifest implementation.
Test Plan: Added tests for diff with matches
Reviewers: #mercurial, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4677023
Signature: t1:4677023:1489076627:dbcea209d300a68fa050f68c52b4fd9949b85302
Summary:
A bunch of if statements were doing the same thing every time. This moves that
logic out of the individual if statements, which makes it cleaner and will make
a subsequent patch that runs a matcher against the paths easier.
Test Plan: Ran the tests
Reviewers: #mercurial, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4677011
Signature: t1:4677011:1489075541:9379597b8866358ad5dad3f1f9ae00a0a0b523f9
Summary: Let's add option that contains a list of other config options to log.
Test Plan: arc unit
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4681336
Signature: t1:4681336:1489171115:dddd27032e945f37fcc846a511e3aaa9e83d29d3
Summary:
Upstream has added a new match argument to manifest.diff() and removed the
existing manifest.matches() function, so we need to update our internal usage.
A separate diff will update treemanifest to support the new diff() api.
Test Plan:
Ran the tests, some still fail because of the upstream changes, but
future patches fix those.
Reviewers: #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4677002
Summary: This diff adds a `--towards` flag to `hg next`. This flag tells `hg next` to always proceed in a linear fashion towards the specified commit. This is useful in situations where the user has a very branchy stack. For example, if the user has a local stack with several branches, each with a feature bookmark at the top, if the user wants to go up a particular stack they can do so with something like `hg next 3 --towards bookmark`.
Test Plan:
Here is an example of basic usage. Also see test file.
```
$ hg debugbuilddag "+5 *2 +2"
$ hg book -r 4 feature1
$ hg book -r 7 feature2
$ hg up 0
$ hg sl
o 92eaf3 debugbuilddag feature2
| r7
|
o 8bbe84 debugbuilddag
| r6
|
o 914970 debugbuilddag
| r5
|
| o bebd16 debugbuilddag feature1
| | r4
| |
| o 2dc09a debugbuilddag
| | r3
| |
| o 012414 debugbuilddag
|/ r2
|
o 66f7d4 debugbuilddag
| r1
|
@ 1ea734 debugbuilddag
r0
$ hg next 3 --towards feature1
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
[2dc09a] r3
$ hg sl
o 92eaf3 debugbuilddag feature2
| r7
|
o 8bbe84 debugbuilddag
| r6
|
o 914970 debugbuilddag
| r5
|
| o bebd16 debugbuilddag feature1
| | r4
| |
| @ 2dc09a debugbuilddag
| | r3
| |
| o 012414 debugbuilddag
|/ r2
|
o 66f7d4 debugbuilddag
| r1
|
o 1ea734 debugbuilddag
r0
```
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4695861
Signature: t1:4695861:1489325959:1145a29ba87f99d07ede1e81415804723dc48818
Summary:
We are relying on pushrebase to replay each of the incoming commits so they
produce both flat and treemanifests. This patch blocks any commits that don't
come through pushrebase, by intercepting manifest.addgroup calls (which is what
a normal push would use) and throwing an exception.
Test Plan: Added a test
Reviewers: #mercurial, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4093745
Signature: t1:4093745:1478100170:79b1eea6668cd3efb27742323e11a45bde930440
Summary:
When a server is a treemanifest enabled server, we want to ensure that every
commit that comes in will be added to the treemanifest as well as the flat
manifest. This patch accomplishes that by hooking into the addmanifest logic and
applying the new manifest to the tree as well.
Test Plan: Added a test
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4093738
Signature: t1:4093738:1478260442:58f77bd88ab09ccda4ac238a065388be8f4a271d
Summary:
This adds a command for producing tree manifests for existing manifests in the
repository. The tree manifests are stored similar to how vanilla mercurial
stores them, but the root revs are in 00manifestree.i instead of 00manifest.i
and we fake the root node to be the same as the flat manifest node, so we can
reference either the flat or the tree easily.
Test Plan: Added a test. Also began converting fbsource
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4092252
Signature: t1:4092252:1478260394:a55360a8ba83ad85b1694cb93f0788f1f672ef61
Summary:
As part of the migration, we need both flat and tree manifests to exist at once
on the server. To do this, we need to store the tree manifest root revisions
somewhere other than 00manifest.i.
This patch divides the treemanifest logic into client and server bits, and on
the server creates a repo.manifesttree property that contains the tree manifest.
Future patches will write to this to fill in new tree entries, and eventually
read from it to seed clients with tree data.
Test Plan:
Future patches add tests that for features that use this
functionality
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4092217
Signature: t1:4092217:1478260318:7edada2ea39a5de565955a4180bfad62f6126352
Summary:
The computehidden caching functionality doesn't seem to be correct, and breaks
my repository. "hg log -r head()" now incorrectly reports some obsolete
commits, which causes various mercurial commands to crash. (They incorrectly
think that these commits aren't obsolete, but their parents are obsolete, which
breaks things.)
This diff reverts the new functionality until it gets fixed.
Test Plan: Confirmed that "hg-dev pull" no longer crashes in my repository.
Reviewers: #mercurial, stash, durham, quark
Reviewed By: quark
Subscribers: net-systems-diffs@fb.com, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4676573
Tasks: 16558840
Signature: t1:4676573:1489008220:f730da69ec0f8d7220cad16ea6956944cfeb670d
Summary: Previously, we were relying on the `allowunstable` extension to wrap various instability-inducing commands and set the appropriate config option to cause the `tweakdefaults` extension to write the correct operation metadata into the obsmarkers produced by each respective command, causing commits to show up as "amended as", "rebased as", etc, in `hg sl`. Since the operation name was based on the command being run, if a user ran `hg amend --fixup` instead of `hg restack`, for example, smartlog would say "amended as" instead of "rebased as". This diff fixes the problem by always setting the operation to "rebase" during a restack.
Test Plan:
1. Create a stack of two commits.
2. Amend the bottom one without `--rebase`.
3. Run `hg amend --fixup` with the `fbamend.userestack` config option enabled.
4. Run `hg sl --hidden` and observe that the rebased commit is marked as "rebased".
See test file for an example.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4670066
Tasks: 16529478
Signature: t1:4670066:1488991076:58c139c50a1aab2aa4fc7e28b1efd57a67bdfb08
Summary:
Sometimes the hg process gets stuck, and we want to get the stack traces to
learn what it's doing. It's not a big issue if gdb can be used with python
debugging support [1]. However, that feature could not be easily set up on
OS X. This extension will make debugging on OS X easier by providing the
Python stack traces on SIGUSR1.
It's similar to OpenStack's "GuruMeditationReport" feature [2].
Unlike `contrib/showsstack.py`, this extension uses SIGUSR1 instead of SIGQUIT,
and writes to a file, instead of stderr, and prints all threads, instead of
just the current one. So it's more practically useful. We may want to
replace `showstack.py` eventually.
[1]: https://wiki.python.org/moin/DebuggingWithGdb
[2]: https://wiki.openstack.org/wiki/GuruMeditationReport
Test Plan: Added a new test
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4660938
Signature: t1:4660938:1488927878:c751856681816a739160c361ed5cc10bab325000
Summary:
Now that ctreemanifest no longer depends on python.h, let's move pythonutil over
to cstore where all the python code is.
Test Plan: Ran the build and the tests
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4663988
Signature: t1:4663988:1488895919:652b3fc35a2dd12c51a9f70e32997c7b4d037c95
Summary:
This is the last piece of removing the Python dependency from the core
treemanifest code. This replaces the old PythonObj diff dictionary with a new
DiffResult class that has a PythonDiffResult implementation.
Test Plan:
Ran the test suite, including the unit tests that explicitly cover
treemanifest diff from python.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4663984
Signature: t1:4663984:1488889615:62f064924b0d6b45dfa7e490d19418060c374f40
Summary:
As part of breaking the native cstore implementation away from Python, let's
create a Matcher class that can be used to perform path match testing. Initially
the only implementation is the PythonMatcher which just wraps a python match
object.
Test Plan: Covered by existing matcher tests in cstore-treemanifest.py
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4663354
Signature: t1:4663354:1488914384:2c33c7e0e7f2eade0786b6ff41503317989fd1e5
Summary:
The upstream matcher has a 'visitdir' function which indicates whether a walk
should traverse into a directory. Let's support this in treemanifest so we can
skip a ton of lookups for parts of the tree that aren't relevant to the match.
Test Plan:
Used gdb to verify commands like 'hg diff -r .^ -r .
--include foo' only traversed into directory foo, and not directory bar. I'm not
sure how to write a test for this since it doesn't actually affect user facing
output.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4648252
Signature: t1:4648252:1488882860:b5e1b738fef2f63336c80cd7c0eb59c2fcf16d0f
Summary:
Previously, tree repacks did not take into account tree history. It would just
look at the delta base and if the base existed, it would just reuse the delta.
This would A) result in very long chains, and B) result in chains where the full
text was the oldest version, instead of the newest (recent full texts means
faster access to recent versions).
This patch threads tree history into the repacker, which already knows how to
use history for repacks.
Test Plan:
Updated the tests, and inspected the new test results to ensure tree
entries that were not deltas before the repack became reverse deltas during the
repack.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4647359
Signature: t1:4647359:1488882710:dba72cf488766ce827b7641735164fa0efc9a303
Summary:
Previously, when treemanifest would create packs of trees during pull, we
allowed trees to be delta'd against trees in other packs. This resulted in
smaller packs, but if the other pack disappeared for some reason (since it's a
cache), the chain broke.
This patch ensures that the first version of every tree added to a pack is a
full text.
This temporarily makes repacks worse, since the repacker doesn't know about
history to produce deltas when combining packs. The next patch adds history
awareness which improves the repack deltafication.
Test Plan:
Updated the tests, and inspected the new test results to ensure that
all packs only had deltas within the pack.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4647348
Signature: t1:4647348:1488882214:e850622a853a534fc60caeef604c88c30740c60d
Summary:
Previously the treemanifest code itself would create the text deltas when
writing a tree out. This meant we couldn't make the delta decision based on
other data, like if the p1 commit was in the same pack file.
This patch removes treemanifest.write() and moves all calls over to
treemanifest.finalize() which gives the python/pack layer control over delta
choices. A future patch will use this to ensure tree packs always contain
complete delta chains.
Test Plan: All tests pass
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4645942
Signature: t1:4645942:1488880851:d0c8c902e7e849072a53344630a9184b6d8e1e7f
Summary:
Previously, to serialize a tree we would call treemanifest.write() which would
take the stores and call the appropriate add() functions. This meant the
treemanifest code controlled when deltas were used, which makes it hard to
decide deltas based on external factors, like which pack file the parent is in.
As a first step to fixing this, we're adding a new finalize function that
returns a python iterator that allows iterating over the new parts of the tree.
In a future diff we will use this to do serialization and delta decisions at the
python/pack layer instead.
The same future diff will also cover this in tests, as it moves all calls to
treemanifest.write() to use treemanifest.finalize().
Test Plan: The next diff switches all write() calls and tests to finalize()
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4645922
Signature: t1:4645922:1488880722:574a37ba800f4b78faf8bad3979b447f4278cc55
Summary:
Previously the treemanifest auto-tree-creation logic only produced data packs
containing the actual contents of the tree blobs. This lost history information
which is important for our ability to efficiently repack the data files.
This patch creates history packs during pull as well. A future patch will also
create history packs for the local tree blob store.
Test Plan: Updated the tests to cover this
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4638865
Signature: t1:4638865:1488449992:48b60961b50b90b6d0e75a64af1f36fb29944e7a
Summary:
To support treemanifests in history packs we need to support the empty filename
(i.e. the root of the repo). This removes some checks that prevented that from
working.
Test Plan:
A future patch will add history pack support for treemanifests,
including tests that cover this.
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4637917
Signature: t1:4637917:1488429178:c25d03b73eb379d4126ebbcee4bb5797f7b841b2
Summary:
Upstream changed the api of this function. Let's add args and kwargs to cover
future api changes.
Test Plan: Ran the tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4667282
Signature: t1:4667282:1488910879:7db5764704dd345946f7f933e7e2c4a8fd77b699
Summary:
Upstream has deprecated repo.*opener properties, so we need to switch to the
equivalent vfs apis to prevent us from being spammed with deprecation warnings.
Test Plan: Ran the tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4667252
Signature: t1:4667252:1488911019:efc3adf21f723151f1ee3b21b91fe62ea3c7b259
Summary: manifestctx's upstream now accept a manifestlog instead of a repo.
Test Plan: Ran the tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4667246
Signature: t1:4667246:1488911047:5340f4dc18183c68dfd47a0c5223b0f804e17806
Summary:
Command that checks that every head and bookmark node is present in the
bundlestore
Test Plan: arc unit
Reviewers: #mercurial
Subscribers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4658436
Tasks: 15389402
Summary: This functionality will be used in command to check backup consistency
Test Plan: arc unit
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4658433
Tasks: 15389402
Signature: t1:4658433:1488828061:49aa5bfeba922d617cf1aab393f5e9f598aa33a4
Summary:
Computing cache key can be quite slow (100ms and even more). Instead let's use
a simpler cache key: size + mtime of phaseroots, obsstore and changelod and
whether changelog write was delayed.
Test Plan:
1) arc unit
2) Run test-newhiddencache.t
3) Run upstream unittest with extension enabled.
python run-tests.py --extra-config-opt extensions.newhiddencachekey=/home/stash/facebook-hg-rpms/fb-hgext/hgext3rd/newhiddencachekey.py -j20
Verify that there are no failures except for those that show that new extension were added.
Note: also need to comment out debug output in the extension
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4604811
Signature: t1:4604811:1487958882:702ea8fdee16def674db97344832936d53f0cd9d
Summary:
This small change makes a way better user experience. Users usually have one
big repo per host, but they may have many hosts (for example, laptop and
devserver). If we ask for reporoot first then it's very likely that next
`pullbackup` invocation will fail because there will be ambigious hosts.
Let's switch the order.
Test Plan: arc unit
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4649946
Tasks: 15389402
Signature: t1:4649946:1488561645:330590fcf1dcd4af7fb572c5d4ccfd8a5ab78c60
