Commit Graph

3155 Commits

Author SHA1 Message Date
Xavier Deguillard
962fce975d pytreestate: use PyPath instead of PyBytes
Reviewed By: DurhamG

Differential Revision: D19591844

fbshipit-source-id: be2fcdc872f57307e20e24f714b05a6f461e0594
2020-01-28 10:01:51 -08:00
Xavier Deguillard
3e27c70d42 pynodemap: use PyPath instead of PyBytes
Reviewed By: DurhamG

Differential Revision: D19591313

fbshipit-source-id: bd3a8b04b83c226f23ad6d0e1b9509bf65db2f69
2020-01-28 10:01:51 -08:00
Xavier Deguillard
8d20878c1a pydag: use PyPath instead of PyBytes
Reviewed By: quark-zju

Differential Revision: D19591260

fbshipit-source-id: 72ea5be6f79d02cce22c7a5f389cf13a9c69f969
2020-01-28 10:01:50 -08:00
Xavier Deguillard
a0d934ecc2 pyedenapi: use PyPath instead of PyBytes
Reviewed By: quark-zju

Differential Revision: D19591173

fbshipit-source-id: 1a5362c99c37f342b1aeb2d95573e4d09c8440c5
2020-01-28 10:01:50 -08:00
Xavier Deguillard
61aaf894c3 pyrevisionstore: use PyPath instead of PyBytes
Summary:
For Python3 compatibility, let's use PyPath, it hides the logic of encoding for
Python2

Reviewed By: DurhamG

Differential Revision: D19590024

fbshipit-source-id: 7bed134a500b266837f3cab9b10604e1f34cc4a0
2020-01-28 10:01:50 -08:00
Adam Simpkins
de66c7494d py3: delete an unused variable that causing Python 3 compatibility problems
Summary:
In Python 3 this causes problems trying to mix bytes with str.  This does not
appear to be used anywhere, so delete it.

Reviewed By: farnz

Differential Revision: D19591564

fbshipit-source-id: 42b8720e440e8e7c901b325aecb1343d5b79b7cc
2020-01-27 21:23:19 -08:00
Adam Simpkins
c049799e91 py3: convert some dict.iteritems() calls to items()
Summary:
These call sites are hit during initialization.  They all appear to be
small-ish dictionaries so the performance hit of having this iteration be
non-lazy in Python 2 should not matter much.

Reviewed By: farnz

Differential Revision: D19591565

fbshipit-source-id: 324844d64ba63f6e379aab270e453d44a02063e9
2020-01-27 21:23:19 -08:00
Adam Simpkins
3a71160859 py3: convert some encoding.environ uses to os.environ
Summary:
Our plan for switching to Python 3 is to update many types to use Unicode
strings when appropriate.  In particular, we should just go ahead and use
Unicode for environment variables, since this is the Python 3's default
behavior.

This drops the check-code.py check that warns developers against using
`os.environ` directly, and converts several call sites to use `os.environ`
directly instead of using `encoding.environ`

Reviewed By: markbt

Differential Revision: D19591566

fbshipit-source-id: e4ac44dcd5f19cd0bf65ac904fc85edad95a9aed
2020-01-27 21:23:19 -08:00
Jun Wu
a0bd794d55 run-tests: make --debug add debugger settings
Summary: This makes --debug more useful - crashes in tests trigger ipdb sessions.

Reviewed By: mitrandir77

Differential Revision: D19581174

fbshipit-source-id: 770c7b054dfce88082b67a9347ecaeb24d8583bc
2020-01-27 19:37:21 -08:00
Mateusz Kwapich
2f68d4b0d2 remove py3 code transform
Summary:
We won't need it as we're going straight to dual-compatible code instead of
transforming it.

Reviewed By: quark-zju

Differential Revision: D19588132

fbshipit-source-id: 100f19b3e29874de9e8d48692c42cb2d5856bf5a
2020-01-27 19:00:14 -08:00
Jun Wu
373073df47 py3/rust: cpython-ext: optionally show Python error traceback
Summary:
This is optional, but it helps investigating Python errors chained with other
Rust errors.

For example:

  error.RustError: failed fetching from store (, cc38739855a7f356b4a2aaac0a0a858fd646e6bf)
  Caused by:
    TypeError()
    Traceback (most recent call last):
      File "scm3/edenscm/hgext/remotefilelog/contentstore.py", line 53, in get
        chain = self.getdeltachain(name, node)
      File "scm3/edenscm/hgext/remotefilelog/contentstore.py", line 91, in getdeltachain
        chain = self._getpartialchain(name, node)
      File "scm3/edenscm/hgext/remotefilelog/contentstore.py", line 125, in _getpartialchain
        return store.getdeltachain(name, node)
    TypeError

Without this diff there is only "TypeError()" without the traceback.
This can be turned off by unsetting RUST_BACKTRACE.

Reviewed By: markbt

Differential Revision: D19581173

fbshipit-source-id: 74605b78146b6b1c9ddd5ad720dcd19ff73908a8
2020-01-27 18:56:10 -08:00
Chad Austin
b708ee6953 fix flakiness in RawOverlayTest.closed_overlay_stress_test
Summary:
In the case that the overlay was closed before the initial
createOverlayFile, prevent an invalid OverlayFile from being accessed,
causing the test to fail.

Reviewed By: genevievehelsel

Differential Revision: D19591636

fbshipit-source-id: 9d961f044c3b68b5c0b2dcd108ff85db6326276d
2020-01-27 18:17:01 -08:00
Xavier Deguillard
24ae9f9592 cpython-ext: fix python3 compile error
Summary: The format_err is used in shared code too, we need to import it.

Reviewed By: quark-zju

Differential Revision: D19592591

fbshipit-source-id: bd344bf3c295473f4647235a98432d11c9678bf9
2020-01-27 16:58:42 -08:00
Xavier Deguillard
33ea1763ce cpython-ext: add a PyPath type
Summary:
This will be used as an argument to the Rust bindings when using paths. This
type is either a PyBytes in Python2 and uses the various encoding function to
convert into a String, or a PyUnicode in Python3 with no encoding change.

Reviewed By: farnz

Differential Revision: D19587890

fbshipit-source-id: 58903426585693193754691fe3c756b9097b35f6
2020-01-27 16:50:14 -08:00
Xavier Deguillard
e512c370fd py3/rust: cpython-ext: set ob_size on raw PyObject
Summary:
Without this, Rust code using the feature (ex. lz4, used by lz4revlog) will
panic.

Reviewed By: sfilipco

Differential Revision: D19581188

fbshipit-source-id: b499449df4fede27fe66cf8e5af57e8347a0dd48
2020-01-27 16:50:14 -08:00
Xavier Deguillard
f16bb04977 py3/rust: cpython-ext: support memoryview as PySimpleBuf
Summary: Otherwise we got RustPanic when clindex or dagindex reads mmapped changelog.i.

Reviewed By: sfilipco

Differential Revision: D19581189

fbshipit-source-id: 3ee74a1bd000d58272551ae404dcfe7f957bb2c0
2020-01-27 16:50:13 -08:00
Xavier Deguillard
ad58839ca1 py3/rust: use Str type in cliparser and hgcommands
Reviewed By: sfilipco

Differential Revision: D19581176

fbshipit-source-id: e92e5c2538537ec16da25a9819c9a097a24a4d6e
2020-01-27 16:50:13 -08:00
Xavier Deguillard
1c697dbc49 py3/rust: cpython-ext: add Str type
Summary: This converts to bytes on Python 2, but unicode on Python 3.

Reviewed By: markbt

Differential Revision: D19581180

fbshipit-source-id: 0de9056a01ae30810a72352387de5a940b37d7ab
2020-01-27 16:50:13 -08:00
Xavier Deguillard
789d2b5fbb py3/rust: types: add AsRef<str> for RepoPath
Summary:
In a future diff, I have RepoPath in Rust and want to send unicode path to
Python.

Reviewed By: sfilipco

Differential Revision: D19581184

fbshipit-source-id: 73a03707a6bdae4a497a8ee2c14314aa4ffefb6d
2020-01-27 16:50:12 -08:00
Karthikeyan Singaravelan
41cd693b98 Import ABC from collections.abc instead of collections for Python 3.9 compatibility. (#780)
Summary:
Fixes https://github.com/facebook/watchman/issues/778
Pull Request resolved: https://github.com/facebook/watchman/pull/780

Test Plan: sandcastle

Reviewed By: chadaustin

Differential Revision: D19543389

Pulled By: wez

fbshipit-source-id: d9725d2f4325da2152e3c19100a25df0fe6f5dec
2020-01-27 16:35:57 -08:00
Kostia Balytskyi
6bf47a9f5a hgtime: fix corner case of date range parsing
Summary: The docs promise that both `<` and `>` bounds are inclusive, so let's fix that.

Reviewed By: markbt

Differential Revision: D19580840

fbshipit-source-id: 13770a8e9351fe62f58e9a701b526a167752543a
2020-01-27 09:37:00 -08:00
Chad Austin
600da9aef0 bump the assertKeySpaceInvariants to a newer MSVC
Summary: I reproduced the KeySpace.cpp ICE on my Windows 10 VM, so bump the MSVC version check. I have not confirmed it does not ICE on MSVC 2019.

Reviewed By: pkaush

Differential Revision: D19562325

fbshipit-source-id: 03cf84e95c709bb2ebe7c3e0055a4f290eb17f8a
2020-01-24 22:34:24 -08:00
Chad Austin
3199c553fa remove the legacy bind mount code from CheckoutConfig
Summary: This code is no longer used now that `eden redirect` exists.

Reviewed By: wez

Differential Revision: D19565010

fbshipit-source-id: 4703bc39a024c4c491a83ef5a6e85711a8f7e4d8
2020-01-24 18:11:36 -08:00
Stefan Filip
d78982a6e8 dag: move iddag to own file
Summary:
Separate Segment and IdDag in two individual files. This is preparation for
refactoring IdDag to be more flexible in terms of storage. That will probably
involve moving stuff out of IdDag into a new file that deals with the storage
abstractions.

Reviewed By: quark-zju

Differential Revision: D19559127

fbshipit-source-id: b3b9b18e2653157e69148b1f29292a57b30016ec
2020-01-24 15:49:54 -08:00
Mark Thomas
d35692711e tests: convert test-rebase-obsolete to mutation
Summary:
Add `test-rebase-mutation.t`, based on `test-rebase-obsolete.t`, but instead
testing specifically for mutation and visibility behaviour.

Reviewed By: xavierd

Differential Revision: D19549473

fbshipit-source-id: c993686eab47985737eef17ab0f52f31d0929159
2020-01-24 14:47:39 -08:00
Mark Thomas
c0e72db0ad tests: enable mutation and visibility in some rebase tests
Summary:
Enable mutation and visibility for the existing tests that use evolution.

Excludes `test-rebase-obsolete.t` which specifically targets obsmarkers - this
will be converted to a new test in D19549473.

Reviewed By: quark-zju

Differential Revision: D19502220

fbshipit-source-id: 70b2828b60220c6887a6210e91bed01c2809cc51
2020-01-24 14:47:39 -08:00
Mark Thomas
f2d5739705 mutation: allow instabilities if mutation is enabled
Summary:
Disallowing instabilities is a feature of evolution.  For mutation tracking
there is no problem with instabilities.

Reviewed By: quark-zju

Differential Revision: D19502221

fbshipit-source-id: c21a5021bf1a8b29abb6635928da50895e2cec33
2020-01-24 14:47:38 -08:00
Mark Thomas
980829029a mutation: include all commits as predecessors in rebase --collapse
Summary:
Rebase with `--collapse` should behave like `fold` and set all the rebased
commits as predecessors of the new commit.

Reviewed By: quark-zju

Differential Revision: D19502223

fbshipit-source-id: a903ee5647bb4ee947a6cc68d8ca791fe0f1299d
2020-01-24 14:47:38 -08:00
Mark Thomas
398e175762 visibility: ensure visibleheads and store requirements are copied on clone
Summary:
When cloning a repo, ensure the store requirements and visibleheads files are
included in the clone.

Reviewed By: quark-zju

Differential Revision: D19502224

fbshipit-source-id: e3ead586fb1fe10def02d55e33810643c0c78bc8
2020-01-24 14:47:38 -08:00
Mark Thomas
845e35ea5f metalog: ensure legacy files are written out atomically
Summary:
`hg clone` can hardlink store files if the repos are on the same device.  This
means we need to write out the legacy files using an atomic temp file, so that
the hardlink is broken.  This also makes sure anyone watching the file sees the
change atomically.

Reviewed By: quark-zju

Differential Revision: D19502222

fbshipit-source-id: 3e9fb35846111c9d924de3c6b6861ce5e295e035
2020-01-24 14:47:37 -08:00
Mark Thomas
39f8b3109f visibility: add visibleheads to journaled files
Summary:
The visibleheads file should be journaled like the other state files, so that
it is rolled back if a transaction aborts.

Reviewed By: quark-zju

Differential Revision: D19502226

fbshipit-source-id: c2b8d926ce97211411ca01f05e83e62cb77625b9
2020-01-24 14:47:37 -08:00
Mark Thomas
4bc06df032 visibility: allow bundlerepo visibleheads to be updated
Summary:
The `incoming` command works by creating a bundlerepo and then updating the
commits as if a pull occurred.  This means the visibleheads can get updated,
which was previously assumed impossible.

We can just accept the new set of visibleheads as-is.  There may be some
unnecessary heads that are covered by other heads, but since the bundlerepo
is short-lived, this isn't a problem.

Reviewed By: quark-zju

Differential Revision: D19502219

fbshipit-source-id: ce9e98c6a40f96f5305bc7580efb476d5264fc88
2020-01-24 14:47:37 -08:00
Mark Thomas
2859b55829 mutation: the workingctx is not obsolete
Summary: It's possible for the workingctx to be queried for obsolescence.  Make sure it is not considered to be obsolete.

Reviewed By: quark-zju

Differential Revision: D19502225

fbshipit-source-id: a1a37afd39a0fc16f95b4eb917372f193ac3fe82
2020-01-24 14:47:36 -08:00
Chad Austin
83bdc4120b when the import helper fails to start, indicate how to view the log
Summary:
It's not often obvious to people where edenfs.log is located on the
filesystem, so redirect people to `eden debug log` instead.

Reviewed By: simpkins

Differential Revision: D19560709

fbshipit-source-id: 6b031c80bab2c21dfddba4d7ec37d4e70218ea08
2020-01-24 13:22:18 -08:00
Chad Austin
9db11e9454 have eden debug log show the end of the log file by default
Summary: 90% of the time, it's more useful to see the end of the log than the beginning.

Reviewed By: simpkins

Differential Revision: D19560473

fbshipit-source-id: f9b5a922d284053d6afc8ea889586826e3c58582
2020-01-24 13:22:18 -08:00
Jia Chen
c717ef0d7d Update pyre version for eden
Summary: Automatic upgrade to remove `version` override and silence errors.

Differential Revision: D19548623

fbshipit-source-id: 12d5e8ec6450de42107d07b230e4b2d0824f7c9d
2020-01-24 12:32:43 -08:00
Lukas Piatkowski
8c7de41d8c arc/licenselint: add license linting for lua and sql
Summary: The fbcode/linttool has been updated to support lua and sql here D19467627, so updating arc rules to cover the new extensions

Reviewed By: zertosh

Differential Revision: D19534326

fbshipit-source-id: 4e615653acedcdf2da88e7dc4b443ad22af1324f
2020-01-24 06:13:28 -08:00
Jun Wu
52af332c28 renderdag: add tests showing how orders affect rendering
Summary:
I wrote it to understand how renderdag draws the same graph with different
orders. It seems useful for future optimization that tries to reduce the number
of columns. So let's check it in.

Reviewed By: xavierd

Differential Revision: D19440713

fbshipit-source-id: 8bc580799f6b24c87886d5ac306020f50bb694e5
2020-01-23 20:50:56 -08:00
Victor Zverovich
09618791bd Update fmt to version 6
Summary: As title

Reviewed By: yfeldblum

Differential Revision: D17741047

fbshipit-source-id: e921a2fef99a9b6f11e09758e69d99a54815054a
2020-01-23 19:33:14 -08:00
Jun Wu
29c749ef7d dag: add fuzz tests on the octopus DAG
Summary: This gives us some confidence about octopus merge handling.

Reviewed By: DurhamG

Differential Revision: D19540726

fbshipit-source-id: e84de74aecae54429483edd185d39fd1bd858f87
2020-01-23 17:58:51 -08:00
Jun Wu
8ac97da54e bindag: make TestContext more flexible
Summary:
TestContext uses ParentRevs. That limits parents to at most 2.

Use a type parameter so we can opt-in Vec<usize> for octopus merge support,
at the cost of worse cache efficiency.

Reviewed By: DurhamG

Differential Revision: D19540727

fbshipit-source-id: f9e8de151b7b296fd6f0fd89be9de2b8996634c7
2020-01-23 17:58:51 -08:00
Jun Wu
df23791d08 bindag: add some octopus examples
Summary:
Our new algorithms support octopus merges. However there were no tests using
octopus merges. This diff adds a simple one.

Reviewed By: DurhamG

Differential Revision: D19540728

fbshipit-source-id: 8411024f0b7e27c2ebfabbe1935496124c25df7b
2020-01-23 17:58:51 -08:00
Jun Wu
494bdae7cc dag: add a fuzz test about range algorithm
Summary:
The test runs the old and new algorithm and compares their result.  This is more
interesting than using random numbers, since the fuzzing framework will try to
explore new code paths.

Reviewed By: sfilipco

Differential Revision: D19511576

fbshipit-source-id: e9a2066769b54a60bb92643e5715f91a6fccbcb5
2020-01-23 17:58:50 -08:00
Jun Wu
78ea96cb9d bindag: port range algorithm from hg
Summary:
The ported algorithm will work as a comparison to verify dag's range
implementation.

Reviewed By: sfilipco

Differential Revision: D19511574

fbshipit-source-id: 589353d6e6c91b8d6707c977eeb8558ac733b525
2020-01-23 17:58:50 -08:00
Chad Austin
68595a14a1 fix some macOS build warnings
Reviewed By: fanzeyi

Differential Revision: D19548525

fbshipit-source-id: 0f137dab2f5af659842cbbc120e2e12f91d18276
2020-01-23 17:37:39 -08:00
Chad Austin
f35f921d5d allow garbage collection of cached tree objects again
Summary:
Now that loading too many blobs only GCs the blob cache, treat cached
trees as ephemeral again.

Reviewed By: simpkins

Differential Revision: D19358375

fbshipit-source-id: 191427f46276f0f5d024b5bc923a68f7a42044da
2020-01-23 17:33:43 -08:00
Stefan Filip
2dc14cf9a6 manifest: use Rust Tree Manifest by default
Summary:
The Rust Manifest is deployed practically everywhere.
Removing the C++ code from being compiled in Mercurial along with all of the cstore extension.

Reviewed By: quark-zju

Differential Revision: D19543248

fbshipit-source-id: d632e171175e6866563c1aa0808a099b67bd937d
2020-01-23 16:06:51 -08:00
Stefan Filip
f5280b75e9 thrift: update thrift generated files
Summary: Commit updates after having ran `make local`

Reviewed By: xavierd

Differential Revision: D19543278

fbshipit-source-id: 00fdc3ebec32e8a3d706b89402dc91f771984c3c
2020-01-23 16:06:51 -08:00
Xavier Deguillard
8041ff9e82 bisect: remove revision number in output
Summary: Revision numbers are deprecated, let's not print them in the bisect output.

Reviewed By: DurhamG

Differential Revision: D19539678

fbshipit-source-id: 29bca3ee38cb390bdef8dd88cb912aa3dc452065
2020-01-23 15:54:00 -08:00
Xavier Deguillard
31b266d32c graft: remove revision number in output
Summary: Revision number are deprecated, remove it from the user output.

Reviewed By: DurhamG

Differential Revision: D19539679

fbshipit-source-id: 7fc618e832e0e5bb3464d2ed04967b062b6f182a
2020-01-23 15:53:59 -08:00
Puneet Kaushik
32cc7c1eca Enable listMounts to get eden list working
Reviewed By: simpkins

Differential Revision: D19510624

fbshipit-source-id: e097c5b412e62d6856179db971296f8389ec8991
2020-01-23 12:41:39 -08:00
Michael Devine
3f326e911c Repo converter: Add named tuple for commit spec
Summary: The converter has many anonymous tuples being passed around, and it leads to typing error and pain figuring out which field has which data. This diff uses named tuples to implement an important tuple in the code that we use to merge repo history into a single unified history.

Reviewed By: tchebb

Differential Revision: D19377226

fbshipit-source-id: cb7d49227752fd7bc98233f6c0d1b2913cb24b05
2020-01-23 12:36:29 -08:00
Xavier Deguillard
ff09f797f4 demandimport: disable in python3
Summary:
We don't need to it for the initial phases of the conversion. I've seen cases
were exceptions while importing were also hidden and caused weird behavior down
the line.

Reviewed By: DurhamG

Differential Revision: D19539064

fbshipit-source-id: 7415d88ef8fb69c949ed557ab0df7e6f7b39bf86
2020-01-23 11:41:02 -08:00
Wez Furlong
43af241301 eden: remove hypothesis test bits
Summary:
Sadly, we didn't derive as much value from these as we would
have liked, and now they are a source of flakeyness in our tests.

Reviewed By: genevievehelsel

Differential Revision: D19460792

fbshipit-source-id: 48c82cc2d1fdbd81bece4057e93799fbcc4f4725
2020-01-23 11:37:27 -08:00
Xavier Deguillard
fe8855e9fb hgmain: initialize fbinit
Summary:
The memcache code assumes it's initialized, let's actually do it.

The test change is required due to fbinit installing its own SIGTERM handler
that shows a backtrace. Installing our own SIGTERM handler makes it closer to
what Mercurial does already.

Reviewed By: quark-zju

Differential Revision: D19518698

fbshipit-source-id: bc8c2311e65c9c00678756abae3979ccda4453ea
2020-01-23 08:57:22 -08:00
Xavier Deguillard
b6589bde84 revisionstore: prefetch takes &[Key] instead of Vec<Key>
Summary: This can prevent potential moves and clones on the caller of prefetch.

Reviewed By: quark-zju

Differential Revision: D19518697

fbshipit-source-id: 63839fc3f4bb9ca420e290eabaffb481a3584f7b
2020-01-23 08:57:22 -08:00
Jun Wu
fff2cb833f dag: add a fuzz test about gca algorithm
Summary:
The test runs the old and new algorithm and compares their result.  This is more
interesting than using random numbers, since the fuzzing framework will try to
explore new code paths.

This cannot run on stable Rust yet. I added a README for how to run it.

Reviewed By: sfilipco

Differential Revision: D19504096

fbshipit-source-id: 621da02c50a771dee9932f9d7a407cb1f412a543
2020-01-22 19:30:50 -08:00
Jun Wu
af85f4ff3b bindag: add a way to get a subdag of a parsed bindag
Summary: Sometimes the graph is too large. Provide a way to slice it.

Reviewed By: sfilipco

Differential Revision: D19511575

fbshipit-source-id: 504317d6894764043b23ea49dcf09c8cdea96961
2020-01-22 19:30:49 -08:00
Jun Wu
b5482f8976 bindag: add utilities for easier testing
Summary:
As we plan to test the dag crate with some other DAG implementation,
add a convenient structure that setups both DAG implementations.

Reviewed By: sfilipco

Differential Revision: D19503371

fbshipit-source-id: 3e9933ad37301bfac36eb1af6d82b4298af778b6
2020-01-22 19:30:49 -08:00
Jun Wu
7d11508dfa bindag: port GCA algorithm from hg
Summary:
The ported algorithm will work as a comparison to verify dag's gca
implementation.

Reviewed By: sfilipco

Differential Revision: D19503373

fbshipit-source-id: f5253db89fbcdc2fd02f3fdaa0796e24338b1fba
2020-01-22 19:30:49 -08:00
Jun Wu
a98d288938 bindag: apply smallvec optimization
Summary:
This is similar to D17581248. It will make the old linear-scan algorithm (which
will be added later) about 5x faster.

Reviewed By: sfilipco

Differential Revision: D19503372

fbshipit-source-id: c65d7217e7b144603dadd57f54a5e70f513c8e51
2020-01-22 19:30:48 -08:00
Jun Wu
64271f24ba dag: move bindag from benches to a separate crate
Summary: This allows bindag to be used outside benches.

Reviewed By: sfilipco

Differential Revision: D19503374

fbshipit-source-id: 131061f7d1d28125875a86afc330dbb9634249cf
2020-01-22 19:30:48 -08:00
Stanislau Hlebik
f8ba0c99a5 infinitepush: introduce infinitepushbookmark path
Summary:
Previously we had infinitepush path which was preferred over a default path when a
specific revision was pulled or updated to (i.e. "hg pull" would use
paths.default, but "hg pull -r HASH" would use paths.infinitepush if this path
is set in the config).

I'd like to have infinitepushbookmark path which will be preferred for all
cases except for a single commit pulls.

Reviewed By: mitrandir77

Differential Revision: D19513490

fbshipit-source-id: 7f8b809f937aa2e082c2e9cd382709c6db619a4f
2020-01-22 14:05:38 -08:00
Jun Wu
fcaef9ed7e sparse: make sparse profile tracking cheaper
Summary:
Reading all the sparse profiles can be very expensive.
This changes the format we log to the cloud. It seems nobody is depending on the old format.

Reviewed By: wlis

Differential Revision: D19517306

fbshipit-source-id: 256e78acd4c24aceb533aa4d66b90ecf885fcbcd
2020-01-22 13:41:34 -08:00
Xavier Deguillard
d997f31b93 setup: properly install the binaries
Summary:
Recently, the Rust extensions went away (D19340538) but that had the negative
side effect of no longer installing all Rust binaries. Make sure the
{build,install}_rust_ext steps are run properly when Rust binaries are present.

Reviewed By: quark-zju

Differential Revision: D19509526

fbshipit-source-id: 03f62f41595931dd7d5f32544a35d31e20fa3f0c
2020-01-22 09:29:18 -08:00
Harvey Hunt
c5f328be68 hg: Remove purge extension from preimported modules
Summary:
D19506964 moved the purge extension into core, however purge is still
mentioned in dispatch.py's list of external modules to preimport. This causes hg
to crash as such:

```
  Traceback (most recent call last):
    File "/data/users/harveyhunt/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/__init__.py", line 61, in run
      dispatch.runchgserver()
    File "/data/users/harveyhunt/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/mercurial/dispatch.py", line 335, in runchgserver
      _preimportmodules()
    File "/data/users/harveyhunt/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/mercurial/dispatch.py", line 320, in _preimportmodules
      extensions.preimport(extname)
    File "/data/users/harveyhunt/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/mercurial/extensions.py", line 180, in preimport
      mod = getattr(__import__("edenscm.hgext.%s" % name).hgext, name)
  ImportError: No module named purge
```

Remove purge from the extmods list.

Reviewed By: ikostia

Differential Revision: D19514862

fbshipit-source-id: 6e16ff160f0bc842038b1481bf23aab9c608685d
2020-01-22 08:18:09 -08:00
Puneet Kaushik
a8e8d9813e Pass the repository path as string instead of path
Summary: Passing the checkout path was throwing an error on Windows - "error: argument of type 'WindowsPath' is not iterable". Passing it as string to fix it.

Reviewed By: simpkins

Differential Revision: D19507974

fbshipit-source-id: 2a91dc37d2497a30a70c684f5298410956a6acca
2020-01-21 22:08:55 -08:00
Saurabh Singh
17ad617625 purge: move extension to core
Reviewed By: quark-zju

Differential Revision: D19506964

fbshipit-source-id: 173918346e90fadbaa52155786aa30fbd4951c8c
2020-01-21 20:49:08 -08:00
Chad Austin
f7491f7741 add independent cache size limits per object type
Summary:
Instead of clearing every single cached object when the total size
exceeds the ephemeral storage limit, keep a limit per object type and
only clear those that exceed their quota.

Reviewed By: simpkins

Differential Revision: D19358312

fbshipit-source-id: 6918d6f4cc2931aed79a9025d0e0f357ede515e0
2020-01-21 19:41:47 -08:00
Chad Austin
e5e9d8cf69 clear the deprecated blobsize keyspace at startup
Summary:
EdenFS no longer uses the blobsize keyspace. To avoid wasting space,
clear and campact it on startup, along with any keyspaces we
deprecated in the future.

Reviewed By: fanzeyi

Differential Revision: D19354881

fbshipit-source-id: 5285757a0e44ab1080c5f940283e06b17bec811d
2020-01-21 10:33:11 -08:00
Chad Austin
46259647e2 deprecate getManifestEntry
Summary:
Indicate getManifestEntry is deprecated so we remember to delete it
down the line.

Reviewed By: genevievehelsel

Differential Revision: D19459793

fbshipit-source-id: 86dd173cfb12c36cf6ba19eaf4be96dfe624c05f
2020-01-21 10:33:10 -08:00
Chad Austin
36954d229d stop calling getManifestEntry and instead look directly in the manifest
Summary:
getManifestEntry has an inefficient implementation. It can result in
multiple round trips to the Mercurial importer process and under heavy
usage it can clog the Thrift worker thread pool. Moreover, it only
provides data that Mercurial already knows. Instead of calling
getManifestEntry, look in the manifest to see if the entry exists.

Reviewed By: quark-zju

Differential Revision: D19426507

fbshipit-source-id: 67bcd7e0ae62f84b313aa6a7203832fb9a10f315
2020-01-21 10:33:10 -08:00
Chad Austin
d23773e9b0 restructure KeySpace and move it into KeySpace.h
Summary: Simplify the definition and use of KeySpace and move it into its own header.

Reviewed By: simpkins

Differential Revision: D19353441

fbshipit-source-id: ef07677d927a48839b709711388abeb3c1ed9679
2020-01-21 10:33:10 -08:00
Xavier Deguillard
524c85d711 revisionstore: limit delta chain to 1000 entries
Summary:
We've seen a case where a datapack contains a circular delta chain, causing
Mercurial to fall into a infinite loop when trying to read it. Let's fail when
the chain is over 1000 entries.

Reviewed By: quark-zju

Differential Revision: D19458453

fbshipit-source-id: bfa503f7807122eca72cf94418abda161dafa41c
2020-01-21 08:50:59 -08:00
Jun Wu
b863b99cd2 pullcreatemarkers: print how many commits are marked as landed
Summary:
wez silenced the messages in D18796291. I think it's still useful to show why
commits get disappeared after pull. Let's just print a summary by default, and
print less or more things depending on flags:

- `-v`: print what commits are marked as landed, similar to pre-D18796291.
- `-q`: print nothing, hide the summary line.

Reviewed By: DurhamG

Differential Revision: D19437916

fbshipit-source-id: 130c5a24f0978674d363227f44089d307f7aff72
2020-01-21 07:36:34 -08:00
Jun Wu
0d15eb6227 graphql: translate globalrev to hg node in getlandednodes
Summary:
In www, Phabricator used to return both landed global rev and hg commit hashes.
Now it only returns global revs. The "mark landed" logic ignores global revs
and does not mark them as landed.

This diff changes it so the graphql client translates global revs automatically
so the "mark landed" logic can still work.

Reviewed By: DurhamG

Differential Revision: D19437915

fbshipit-source-id: 0e074bda9a0ee53c594750b2b1fd214f31dec843
2020-01-21 07:36:34 -08:00
Jun Wu
3da70a5d46 graphql: support translating GLOBAL_REVs in getmirroredrevmap
Summary: The function assumes binary nodes. Make it aware of GLOBAL_REVs.

Reviewed By: DurhamG

Differential Revision: D19437914

fbshipit-source-id: aa36b81be2f8e4c38c51667b4cf7048eeb7250da
2020-01-21 07:36:34 -08:00
Mark Thomas
4958c4e342 tests: update tests-fb-*.t to use new tinit features
Reviewed By: quark-zju

Differential Revision: D19457679

fbshipit-source-id: d1010f0d2eee69ecdee5c3feba772bd3684d4cec
2020-01-20 02:45:17 -08:00
Mark Thomas
d2730c3ba0 tests: update tests-[r-x]*.t to use new tinit features
Reviewed By: quark-zju

Differential Revision: D19457676

fbshipit-source-id: 5bbab22ff0d98b725b63b906864037fae5fa5171
2020-01-20 02:45:16 -08:00
Mark Thomas
633382de11 tests: update tests-[i-p]*.t to use new tinit features
Reviewed By: quark-zju

Differential Revision: D19457680

fbshipit-source-id: 2be941f22525ed2d9f5a1aa3c92af38fb612eb46
2020-01-20 02:45:16 -08:00
Mark Thomas
b361a8d1bd tests: update tests-[a-h]*.t to use new tinit features
Reviewed By: quark-zju

Differential Revision: D19457677

fbshipit-source-id: 9f121837dfc6bf98ff4525c6535496b292409844
2020-01-20 02:45:15 -08:00
Mark Thomas
b654a9aae5 tests: use disable treemanifest in place of setconfig
Summary: Use `disable treemanifest` in place of `setconfig extensions.treemanifest=!`

Reviewed By: quark-zju

Differential Revision: D19457678

fbshipit-source-id: 47c8b3c4f4ed5cfc97275ca67afab4a86ffe7f0d
2020-01-20 02:45:15 -08:00
Mark Thomas
212cec117b tinit.sh: add configure for setting up standard configurations
Summary:
Add a new `tinit.sh` function: `configure`, which takes over from the
feature-enabling aspect of `enable`.

This provides a few features that are configurable:

  * `dummyssh` sets `ui.ssh` to `dummyssh`.
  * `mutation` and `mutation-norecord` enable mutation and visibility tracking
  * `evolution` enables evolution and disables mutation and visibility.
  * `noevolution` disables evolution, as well as mutation and visibility.

Since `enable` now only refers to extensions, it is joined by a corresponding `disable` to
disable an extension.

Some tests are updated to take advantage of these new functions.

Reviewed By: quark-zju

Differential Revision: D19427595

fbshipit-source-id: 03cc639918b4e667927330d3f3abf16121ebf161
2020-01-20 02:45:14 -08:00
Jun Wu
124e275377 dag: make NameDag use MultiLog for data consistency
Summary: This ensures IdMap and IdDag are guaranteed consistent in the storage layer.

Reviewed By: DurhamG

Differential Revision: D19432658

fbshipit-source-id: 00f1a9b4c747baa1f14d78c31d925682317463b4
2020-01-17 21:49:57 -08:00
Jun Wu
907aadcdd7 indexedlog: add MultiLog
Summary: The MultiLog holds multiple Logs and can atomically sync them.

Reviewed By: DurhamG

Differential Revision: D19432659

fbshipit-source-id: 6ac7dc6f74468f985c6a6b0c419e888722a80037
2020-01-17 21:49:57 -08:00
Jun Wu
5aa872599c indexedlog: make ScopedDirLock remember which directory gets locked
Summary: This makes it possible to do extra sanity checks.

Reviewed By: DurhamG

Differential Revision: D19443783

fbshipit-source-id: 254c2537a6aadd25a67c5e48a768187ce65aa686
2020-01-17 21:49:56 -08:00
Jun Wu
2c9a3b9c61 indexedlog: use a method to create LogMetadata
Summary: This makes the code overall shorter.

Reviewed By: DurhamG

Differential Revision: D19443552

fbshipit-source-id: abd1db227830a88549c7eca1cfd08b67c4914518
2020-01-17 21:49:56 -08:00
Jun Wu
a78597c92d gitlookup: use nodemap index to speed up lookups
Summary:
Scanning through the plain git mapfile is slow. Use the nodemap to speed it up.
To avoid unnecessary risks, this only replace the "lookup" feature of the mapfile,
other features used by the hggit extension or wire-protocols are unaffected.

Reviewed By: singhsrb

Differential Revision: D19458406

fbshipit-source-id: 665184637d3e62590cc5d12ea3aa2563af4351d1
2020-01-17 19:21:40 -08:00
Wez Furlong
8d85b68d6b hg: fsmonitor: fix sockpath propagation from WATCHMAN_SOCK env
Summary:
This unbreaks the watchman/hg integration tests in a more
sustainable and holistic way than the workaround in D19415053.

The issue is that the explicit fsmonitor sockpath configuration option didn't
take into account that the environment needs to take precedence in order for
the appropriate watchman instance to be used by the set of processes in a given
process tree.  It is not feasible to have them all pass `--config` options to
mercurial because that would require teaching several layers about the
fsmonitor extension, and then later to update them as we evolve how mercurial
works.

Reviewed By: quark-zju

Differential Revision: D19415252

fbshipit-source-id: 5872d0462e466bfb5d70f809c3c433d92fb78567
2020-01-17 17:36:03 -08:00
Zeyi (Rice) Fan
1dcd227ccd eden: refresh union datapack store as well
Summary: This diff addresses the unreleased mapped pack files in the old datapack code.

Reviewed By: chadaustin

Differential Revision: D19399737

fbshipit-source-id: 86a6254a2939fd69e1ce2b25b8bfcb36b0deb16b
2020-01-17 15:00:01 -08:00
Zeyi (Rice) Fan
a431e64e4e eden: periodically refresh content store to give it chances to release mapped files
Summary:
As reported by JT, EdenFS may hold the file descriptor of mapped pack files too long even when it is deleted by external processes, thus taking more disk spaces.

This diff fixes this problem by making EdenFS periodically rescan the pack files.

Reviewed By: chadaustin

Differential Revision: D19395439

fbshipit-source-id: 4bfd6a7ac13dceb3099d2704d62b3825433aff4b
2020-01-17 15:00:01 -08:00
Wez Furlong
3eb293e32d hg: fixit quick! pywatchman client construction
Summary:
There was a typo in the name of the parameter; this commit
makes this code consistent with the upstream pywatchman code.

Reviewed By: fanzeyi

Differential Revision: D19453811

fbshipit-source-id: 033cf30c47ff327913a07e177457a4ac23505bbd
2020-01-17 14:11:48 -08:00
Genevieve Helsel
0d5145389d documentation for paths
Summary: this is largely just markdownified comments from `eden/fs/utils/PathFuncs.h`. I've been a bit confused on the differences lately and how they interact, so I read through that file and thought a markdown file would be helpful for future reference, since the file is reaching 2000 lines.

Reviewed By: fanzeyi

Differential Revision: D19434132

fbshipit-source-id: 0ec5551e9da0a3202fcc3efff52f9f6f48f99db4
2020-01-17 12:08:42 -08:00
Jun Wu
8f694e04a1 pull: update remote/master unconditionally
Summary:
This is inteneded to solve multiple problems:

- Mitigate issues where our CI system does not enable remotenames in some
  random code paths, which leads to missing or stale `remote/master`.
  This further mitigates bad cases with narrow-heads' phase calculation
  where a stale `remote/master` can cause many commits to become draft
  incorrectly.
- Mitigate issues in `remotenames` where race condition can happen for
  pushrebase (remotenames does a "listnames" after the push operation).
  See D18601035 for a hacky workaround.

Reviewed By: DurhamG

Differential Revision: D19380940

fbshipit-source-id: 4481ff114d35be37c331c72ac561c2a0894206cb
2020-01-17 10:13:40 -08:00
Jun Wu
755a238a5d osutil: remove usage of ApplicationServices
Summary:
We recently saw a high rate of build failures on MacOS.
They all fail with compiler errors like:

    In file included from edenscm/mercurial/cext/osutil.c:1326:
    In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk//System/Library/Frameworks/ApplicationServices.framework/Headers/ApplicationServices.h:23:
    In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk//System/Library/Frameworks/CoreServices.framework/Headers/CoreServices.h:39:
    In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk//System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Headers/LaunchServices.h:23:
    In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk//System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Headers/IconsCore.h:23:
    In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk//System/Library/Frameworks/CoreServices.framework/Frameworks/OSServices.framework/Headers/OSServices.h:29:
    In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk//System/Library/Frameworks/CoreServices.framework/Frameworks/OSServices.framework/Headers/CSIdentity.h:43:
    In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk//System/Library/Frameworks/Security.framework/Headers/Security.h:38:
    In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk//System/Library/Frameworks/Security.framework/Headers/SecProtocolOptions.h:28:
    /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk//System/Library/Frameworks/Security.framework/Headers/SecProtocolTypes.h:158:19: error: missing ',' between enumerators
        kTLSProtocol12 CF_ENUM_DEPRECATED(10_2, 10_15, 5_0, 13_0) = 8,
                      ^
    /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk//System/Library/Frameworks/Security.framework/Headers/SecProtocolTypes.h:158:20: error: redefinition of enumerator '__AVAILABILITY_INTERNAL__MAC_10_2_DEP__MAC_10_15'
        kTLSProtocol12 CF_ENUM_DEPRECATED(10_2, 10_15, 5_0, 13_0) = 8,
                   ^

It's unclear whether this is a compiler configuration issue, or the Apple SDK
being buggy.

The only user of ApplicationServices is `isgui()` when `$SSH_CONNECTION` is not
set on MacOS.

It seems to me that if we just replace `isgui()` to `True`, very few if nobody
will notice the difference. Therefore let's remove the problematic
ApplicationServices as an attempt to stabilize the build.

FWIW Git does not seem to have any sort of GUI detection. It seems to solely
rely on (static) user configuration.

This reverts part of https://www.mercurial-scm.org/repo/hg/rev/16118b4859a1.

Reviewed By: singhsrb

Differential Revision: D19441351

fbshipit-source-id: f1bfbaf0d015e8d577a83112b5ea0a4be6367156
2020-01-17 07:25:37 -08:00
Pádraig Brady
c8df1542f4 platform009: EdenMount.cpp: fix -Wreturn-std-move with clang-9
Summary:
stderr: eden/fs/inodes/EdenMount.cpp:1122:37:
  error: local variable 'unmountError' will be copied despite being thrown by name [-Werror,-Wreturn-std-move]
  eden/fs/inodes/EdenMount.cpp:1122:37: note: call 'std::move' explicitly to avoid copying

Reviewed By: simpkins

Differential Revision: D19408659

fbshipit-source-id: 1575104414daaeb06892586492cdb74e2df604c5
2020-01-17 04:58:15 -08:00
Jun Wu
4a98b00205 changelog: still migrate non-public commits to zstore
Summary:
This makes it safer to do revlog-fallback -> server-fallback switch in the
future, because the server might not know all the draft commits.

Reviewed By: DurhamG

Differential Revision: D19416287

fbshipit-source-id: 8d69c1c5465b50710110370d84d5fb7c8ba6a6c4
2020-01-17 04:01:05 -08:00
Jun Wu
733961456f indexedlog: fix try_clone external key buffer handling
Summary:
In both cases (clone with or without dirty content), the external key buffers
used by indexes should be re-created, since mem_buf location has changed.

Reviewed By: DurhamG

Differential Revision: D19432657

fbshipit-source-id: fe6f76e7ccfd16ccd2f5c1d89866687a3503603e
2020-01-17 03:58:00 -08:00
Jun Wu
85d7253edd indexedlog: add "shared metadata" abstraction
Summary: This allows us to build the multi-log structure.

Reviewed By: DurhamG

Differential Revision: D19431786

fbshipit-source-id: e0e09b0d5a73d293a80626924b74ddf2ce6d6fa3
2020-01-17 03:57:59 -08:00
Jun Wu
36e70264b6 indexedlog: avoid log::OpenOptions::create_in_memory
Summary: Migrate to the new API. This is more compatible with the next change.

Reviewed By: DurhamG

Differential Revision: D19431788

fbshipit-source-id: dba1a88fd003d17fc1774b0cec9157d32c5acdb0
2020-01-17 03:57:59 -08:00
Jun Wu
de5772cb30 indexedlog: replace Log's dir with a new GenericPath type
Summary:
This allows us to incrementally abstract away parts related to filesystem from
Log. For example, instead of using std::fs to access filesystem directly, use
methods on the GenericPath instead.

Right now the main motivation is to add support for "multi-logs" sharing a single meta
file. To do that, methods reading and writing metas are moved to the
GenericPath type.

This also unifies the open functions. Now OpenOptions::create_in_memory can be
private. Empty in-memory Logs can be opened via `open(())`.

Reviewed By: DurhamG

Differential Revision: D19431784

fbshipit-source-id: dbdf94f60261e09f131c6fdd9fe3b99242a28af5
2020-01-17 03:57:59 -08:00
Jun Wu
bf133b797e indexedlog: expose io::ErrorKind in the wrapped error type
Summary: This makes an upcoming change easier.

Reviewed By: DurhamG

Differential Revision: D19432656

fbshipit-source-id: 65adc883c3c3937aa7022196b83c75715b820721
2020-01-17 03:57:58 -08:00
Jun Wu
24393faaeb indexedlog: move log::tests to a separate file
Summary: The goal is to split log.rs to smaller modules so it's easier to reason about.

Reviewed By: DurhamG

Differential Revision: D19431787

fbshipit-source-id: 23b8346c461da322e7a2240b8224e6194867aaee
2020-01-17 03:57:58 -08:00
Jun Wu
38db4b5cdc indexedlog: move log::OpenOptions to standalone modules
Summary:
The goal is to split log.rs to smaller modules so it's easier to reason about.

OpenOptions has many dependencies. Move the most complicated one (repair) to
another standalone module.

Reviewed By: DurhamG

Differential Revision: D19431783

fbshipit-source-id: 28c3af7cd6e19588bfe9c395e2648244faf3179d
2020-01-17 03:57:58 -08:00
Jun Wu
f6d3fd3d39 indexedlog: move log::LogMetaData to a standalone module
Summary: The goal is to split log.rs to smaller modules so it's easier to reason about.

Reviewed By: DurhamG

Differential Revision: D19431785

fbshipit-source-id: b9a598b3018267ff6e43ef57dc807cb4e880467c
2020-01-17 03:57:57 -08:00
Genevieve Helsel
53fcbd2d64 edge case on gitignore processing with directory replaced with file
Summary: based on comment on D18647089, changes the behavior when a directory is replaced with a file, and then the directory is ignored. The behavior should be that the file is reported as `ADDED`, previously the file would have been reported as `IGNORED`

Reviewed By: simpkins

Differential Revision: D19318796

fbshipit-source-id: 8da1b7c6df182ca591e4aea21f6573ca81790cb4
2020-01-17 00:05:26 -08:00
Zeyi (Rice) Fan
59c2cb7aff eden: make HgImporter safer
Summary:
We want to have a shared pool of `HgImporter` across different threads, but that would require `HgImporter` is safe to be *passed* between threads. (This is not making `HgImporter` entirely thread safe.)

However, `HgImporter` currently holds a pointer to the thread local fb303 counter. This diff pushes down `EdenStats` so we only access the thread local stats variable when we need to add counters. This might be a little slower but it does make it safer.

Reviewed By: chadaustin

Differential Revision: D19053250

fbshipit-source-id: 44a897acc90c6042ae22a0417eece39e099ee13f
2020-01-16 12:45:37 -08:00
Zeyi (Rice) Fan
bd7b3a38c8 eden: decouple LocalStore from HgImporter
Summary: The only reason `HgImporter` still holds a pointer to LocalStore is for looking up proxy hashes. We can directly pass in the Mercurial hash and path instead.

Reviewed By: chadaustin

Differential Revision: D19049039

fbshipit-source-id: 45d1e1f363ed73dca447b22e5891495cf9ad202b
2020-01-16 12:45:36 -08:00
Genevieve Helsel
283c6de0df log mismatched parents during staus
Summary: scuba logging for mismatched parent commits, it would be nice to have a count on how often this is encountered so we can pinpoint spikes of this case. I am not sure how helpful having the parent commits would be in the event, my thought there was that it could help see how far away the commits are and if one of the commtis doesn't exist (like in the case of stripping commtis with a cancelled hg split), but I am open to having an empty event as well if storing these strings is too expensive.

Reviewed By: fanzeyi

Differential Revision: D19432378

fbshipit-source-id: fa3e7cd6aef832c4f918f339b781590b7484cfdc
2020-01-16 12:01:11 -08:00
Thomas Orozco
341e695ac5 hggit: don't crash in updatemeta if a commit does not have extras at all
Summary:
Right now, we check if extras contain `convert_revision` (and tolerate it not
being there), but if we don't have extras at all, we crash. Let's instead treat
"no extras" and "convert_revision not in extras" as the same thing.

Reviewed By: DurhamG, farnz

Differential Revision: D19428698

fbshipit-source-id: 74268dd3bde0f9cdb1b8164d2c2d322837803eaa
2020-01-16 10:13:12 -08:00
Xavier Deguillard
d88b2fed9b remotefilelog: properly support force prefetch
Summary:
There are few cases where we need to prefetch data that is already present in
the local store, but the server holds more up-to-date data. For instance, after
pushing a change to a pushrebase server, the linkrev for the just pushed commit
is no longer valid, and the history data is re-fetched from the server to fix
it.

For now, let's keep the fix for this contained in Python as we will move away
from linkrev altogether in the future, by then, this change will be unecessary
and can be reverted.

I'm overall not a big fan of the hacks needed here, suggestions are welcomed for
a cleaner way to achieve this :)

Reviewed By: DurhamG

Differential Revision: D19412620

fbshipit-source-id: 331f3e4dbb51bcd687149370a62c60c325534f60
2020-01-16 09:41:41 -08:00
Xavier Deguillard
1e809fc681 revisionstore: allow building a {Content,Metadata}Store without local stores
Summary:
In order to avoid pathological linkrevfixup after pushing a change to a
pushrebase repo, we need to be able to fetch history data that is already
present locally from the server. Since the Rust stores will always check
whether the data is present locally before fetching it, we would not be
fetching anything, causing the pathological linkrevfixup to kick in.

By allowing the stores to be built without a local component, the prefetch
function will not find the local history, and will thus be able to fetch it
properly.

Reviewed By: DurhamG

Differential Revision: D19412619

fbshipit-source-id: 421c59c63634ead7f98e6ba89da0532067f7b412
2020-01-16 09:41:40 -08:00
Xavier Deguillard
d89eab8078 pyconfigparser: use String as arguments instead of PyBytes
Summary:
In the cpython bindings, the Rust String can take both PyBytes and PyUnicode
strings, which is perfect for Python3 compatibility as string literals are
PyBytes in Python2 and PyUnicode in Python3. The return values are kept as
Bytes for now as changing this is a much larger change in itself.

Other approaches tried:
 - Using PyUnicode as input/output: an extremely large codemod had to be done,
   with very little benefits
 - Using String as output: since we do have some configs that are unicode, on
   the Python side, the output might either be bytestrings or unicode, leading
   to weird bugs.

Reviewed By: DurhamG

Differential Revision: D18650466

fbshipit-source-id: aebdf30590dcae40b7df2787e5ece88e2ec9395c
2020-01-16 09:31:45 -08:00
Mark Thomas
2815d67f41 mutation: update amend tests to use mutation and visibility
Summary:
Update all amend tests to use mutation and visibility.  For these tests, `mutation.record`
is disabled so that hashes do not change.

Most tests pass as-is.  A few tests have changes which are the results of slight behavioural
changes or bug-fixes compared with obsmarkers.

Reviewed By: quark-zju

Differential Revision: D19413580

fbshipit-source-id: 18696a08db69130b08f829366659783df4c9bcc1
2020-01-16 01:03:23 -08:00
Mark Thomas
05f1ef73b2 obsolete: add option for disabling evolution
Summary:
Allow disabling of evolution and obsolescence markers by setting
`experiemental.evolution=obsolete`.

Reviewed By: quark-zju

Differential Revision: D19411232

fbshipit-source-id: 89601a93cff1f87d04b7230fcb6c1e91cf074e92
2020-01-16 01:03:23 -08:00
Mark Thomas
eef0c650e7 mutation: make the mutationstore the source of truth for mutation entries
Summary:
Previously the mutation commit extras were the source of truth for mutation
information, and the mutation store served as a kind of cache.  This turned out
to be less useful than expected, as oftentimes commits are missing, and the
store is better indexed, so in practice using the store as the source of truth
is better.

This change makes the mutationstore the (sole) source of truth for mutation
data.  The extras are kept, but they are now only useful as human-readable
debug information, and to ensure the commit hash is unique.

Collecting the mutation information during commit creation is now done through
a new `mutinfo` object.  This is a dict with the same keys as the mutation
extras, for simplicity, but it is now passed through the `committablectx` and
used to generate the mutation store entry directly.

The `mutation.enabled` config option is now used to control all aspects of
enabling mutation.

The `mutation.record` config option is now only used to indicate whether the
mutation extras should also added to the commit.  Generally this should be set
to `true`, however the option is retained so that mutation extras can be
stripped by running `hg amend --config mutation.record=false`, which no longer
has the side-effect of not recording mutation information to the store.

The "remote commit" mutation record origin is now obsolete, and won't be
generated anymore.

Pushrebase now relies on the obsmarker information coming back from the server
in order to correctly generate mutation information.  We will need to change
this so that the server returns mutation records before we can fully deprecate
obsmarkers.

Reviewed By: DurhamG

Differential Revision: D19410650

fbshipit-source-id: 8d7094e4bfd8d8e97916898d899a8debd339485f
2020-01-16 01:03:22 -08:00
Josh Rosenbaum
6bf2ce6f9b log common directory path of all files in the changeset
Summary: Begin logging the common directory path for the commit's updated files.

Reviewed By: DurhamG

Differential Revision: D19399717

fbshipit-source-id: d47994f573eac6e7cd1596b0b102cb9577e1eea1
2020-01-15 19:06:17 -08:00
Michael Devine
8e61b4445d Repo converter: Remove "[MERGED]" prefix from commit messages
Summary: When I was first prototyping the converter, I used a prefix to make it clear which commits represented merged commit history. This commit removes that prefix so the commit messages are an exact match.

Reviewed By: tchebb

Differential Revision: D19404075

fbshipit-source-id: 73b4aab39b30ce1dcff917e40d459f9615896fca
2020-01-15 16:01:19 -08:00
Xavier Deguillard
03b6607779 remotefilelog: only prefetch history in linkrevfixup
Summary:
The linknode is part of the history, not the data, no need to force prefetch
the blobs.

Reviewed By: quark-zju

Differential Revision: D19394351

fbshipit-source-id: 2e63d82928ebd5cf8e3a0d1b87d115b45d1428a7
2020-01-15 15:55:11 -08:00
Xavier Deguillard
c92c9d5a03 revisionstore: add some comment explaining the order of local/shared store
Summary:
Mercurial assumes some specific ordering, let's comment about this to make sure
we don't re-order and introduce subtle bugs.

Reviewed By: quark-zju

Differential Revision: D19394352

fbshipit-source-id: 0f9e02d2c6addf040311a54b8161b06bbeaa6be9
2020-01-15 15:55:11 -08:00
Xavier Deguillard
3030ad82b0 remotefilelog: fix remotefilelog.cachepath difference for Rust ContentStore
Summary:
The error message and the exception type are slightly different between the
Rust and Python ContentStore. For now, let's just fix this up manually.

Reviewed By: quark-zju

Differential Revision: D19394350

fbshipit-source-id: e432094a9dfcf605568a1890c0303b733e98d203
2020-01-15 15:55:10 -08:00
Genevieve Helsel
a7d3210d2e normalize hg status error between python and rust
Summary: The rust telemetry wrapper and the python code were not in sync in how they presented errors to the user on failure. This makes them identical in message and error code (however the "abort" from python is red in color).

Reviewed By: quark-zju

Differential Revision: D19313848

fbshipit-source-id: d5b95e8d5130f20fc85c6664933ee18702f7cc5e
2020-01-15 15:30:52 -08:00
Xavier Deguillard
d784f6890c test-ssh.t: make it less flaky
Summary: This eliminates the broken pipes that have been more prevalent recently.

Reviewed By: DurhamG

Differential Revision: D19412153

fbshipit-source-id: 0062eb57ec07dfcbfee3ed9b77cf82cdcda09fb3
2020-01-15 14:05:06 -08:00
Jun Wu
94c383327a dag: make NameDag support buffered changes
Summary:
Previously, NameDag only supports writing directly to disk. That is not easy
for hg to use, since hg knows commits before bookmark names during pull, while
the API requires both commits and bookmarks (to decide which commits belong to
the "master" group) at the same time.

The old API also does not match other storage layers like indexedlog and
metalog, where writes are buffered until explicitly flushed.

This diff adds the "write-in-memory" (named "add_heads"), and "flush" APIs to
make NameDag easier to use. Under the hood, it just copies the "added" portion
of the DAG, and re-use the old API ("build", renamed to "add_heads_and_flush")
to do the job.

Note: The buffered changes pattern cannot be used in IdDag, since Ids might
have to be re-assigned on flush. But NameDag does not expose the raw Ids in its
"normal" interface, so it's fine to reassign Ids on flush.

Reviewed By: markbt

Differential Revision: D19405474

fbshipit-source-id: 75e5e5815c78c3577a0138f48185f6c4b5a80891
2020-01-15 14:02:07 -08:00
Jun Wu
3a74b82a39 dag: rename NamedDag to NameDag
Summary:
This is more consistent with the name "IdDag", because both "Name" and "Id" are
noun.

Reviewed By: singhsrb

Differential Revision: D19405473

fbshipit-source-id: a3b7546dd0ddcae5d9ed562838bd6be10a47063c
2020-01-15 14:02:06 -08:00
Jun Wu
3c5766d59e dag: rename Dag to IdDag
Summary: This is more consistent with the name "NamedDag".

Reviewed By: singhsrb

Differential Revision: D19405472

fbshipit-source-id: f7023307acaf96bf77c9fa9704dcaf6fc59b56f2
2020-01-15 14:02:06 -08:00
Joseph Friesen
1b504b8c93 add TCP support to fsmonitor
Summary:
This is part of proof-of-concept to test Watchman over TCP, a means
to accelerate hg workflows for the ASIC teams within Facebook.

For more context, refer to master task T55191832

Reviewed By: xavierd

Differential Revision: D18482107

fbshipit-source-id: 08ba3641854d81fe2a1cd8739bb4e9abcbc700ed
2020-01-15 13:43:36 -08:00
Joseph Friesen
aebbe4937f Propagate SockPath + TcpSocketTransport changes to hg
Summary: Copied changes from watchman dev area to hg

Reviewed By: wez

Differential Revision: D18574426

fbshipit-source-id: 6c360e42621cef373c5c1d4af34011d80eafc7bf
2020-01-15 13:43:36 -08:00
Wez Furlong
f6a9a6f7e8 eden: du: revise instructions for cleaning up LFS
Summary: `hg gc` can now clean things up there, so suggest that.

Reviewed By: quark-zju, fugalh

Differential Revision: D19413816

fbshipit-source-id: 1c6c08ed9fb4757390883d908531d9b3a7da302d
2020-01-15 12:31:56 -08:00
Jun Wu
85a89c6f96 dag: add git commit graph as test fixture
Summary:
The git repo has a lot of merges. So it's interesting as a testing graph.
The file was generated by:

    git clone https://github.com/git/git --bare
    hg --config extensions.hggit= clone git.git git-hg
    hg --cwd git-hg debugbindag -r 'all()' -o git.bindag

Reviewed By: DurhamG

Differential Revision: D19411825

fbshipit-source-id: 0fec8a16786dc8e6790986bca5c62a76276aa942
2020-01-15 11:03:52 -08:00
Mark Thomas
56cd3eadb5 renderdag: connect vertical lines for non-merge commits
Summary:
This changes the pattern commonly seen in smartlog:

    ╷ o  e7f5f529  ...
    ╭─╯
    │ o  a1b2773d ...
    ╭─╯
    o  ecbb4eaa
    ╷

to:

    ╷ o  e7f5f529  ...
    ├─╯
    │ o  a1b2773da ...
    ├─╯
    o  ecbb4eaa
    ╷

The change only applies to commits with a single parent. Vertical lines from
merge commits stay disconnected intentionally. For example:

    │ │ o  I
    │ │ │
    │ o │      H
    ╭─┼─┬─┬─╮
    │ │ │ │ o  G
    │ │ │ │ │

This makes it more obvious that `H` has 5 parents while `I` only has 1 parent -
`I` does not "borrow" `H`'s parents.  This problem does not exist if `H` only
has 1 parent.

Reviewed By: quark-zju

Differential Revision: D19407687

fbshipit-source-id: 1046c8e2309f50e3f1620ed21f1b10573759a5f8
2020-01-15 06:57:51 -08:00
Jun Wu
dd1b9cc98d pull: use transaction
Summary:
Without this change, the next diff will fail tests with metalog programming
errors (write outside transaction).

Some test changes are caused by the being deprecated `rollback` command.

Reviewed By: DurhamG

Differential Revision: D19380939

fbshipit-source-id: 7c893d3025bb697102835670b8a38f8fb9a624c8
2020-01-14 21:02:27 -08:00
Jun Wu
ba2d83b3f8 pull: remove "new changesets" message
Summary:
Reverts https://www.mercurial-scm.org/repo/hg/rev/eb586ed5d8ce.

The colon syntax (x:y) is deprecated and is unsupported by segmented changelog.

Reviewed By: DurhamG

Differential Revision: D19394101

fbshipit-source-id: 8c66756f1035ab7660180716a2afa052879f384e
2020-01-14 21:02:27 -08:00
Durham Goode
4c5aa36827 gpg: wrap dirstate in transaction
Summary:
As part of requiring transactions for the dirstate, let's wrap its use
in gpg.

Reviewed By: quark-zju

Differential Revision: D18213029

fbshipit-source-id: b05afa6d465e8d2a7b1637fa1c1903e1625a7104
2020-01-14 17:49:05 -08:00
Durham Goode
b52d325ebc histedit: add dirstate transactions
Summary:
As part of making the dirstate require transactions, let's wrap
histedits dirstate changes in transactions.

Reviewed By: quark-zju

Differential Revision: D18213045

fbshipit-source-id: 49baf87e8f8abfe500409f0f79a52eeae09f8ada
2020-01-14 17:49:05 -08:00
Durham Goode
419368da12 eol: wrap dirstate in transaction
Summary:
As part of requiring transactions for the dirstate, let's wrap eol's
manipulation of it.

Reviewed By: quark-zju

Differential Revision: D18213039

fbshipit-source-id: 28b52057e61fe9d88ce9f26be2cbf7482837ba84
2020-01-14 17:49:04 -08:00
Durham Goode
cd3d20e4e3 commitcloud: wrap update in transaction
Summary:
As part of moving the dirstate to require a transaction, let's wrap the
update in commitcloud.

Reviewed By: quark-zju

Differential Revision: D18213046

fbshipit-source-id: e59c880ea5e8d8808499f94c983c26e7466204e2
2020-01-14 17:49:04 -08:00
Durham Goode
b3d7defa23 automv: wrap dirstate in transaction
Summary:
As part of requiring the dirstate to be part of a transaction, let's
wrap its usage in automv.

Reviewed By: quark-zju

Differential Revision: D18213034

fbshipit-source-id: f8557453351e665ac59da9614c84ab6312cff4ef
2020-01-14 17:49:04 -08:00
Durham Goode
61b0ca78e8 fbamend: wrap dirstate in transaction
Summary:
As part of moving the dirstate to always be part of a transaction,
let's wrap its usage in our amend extension.

Reviewed By: quark-zju

Differential Revision: D18213028

fbshipit-source-id: 3562bb589265dceb3d89c997710738f44eb49f33
2020-01-14 17:49:03 -08:00
Durham Goode
b2cc1a40f3 contrib: update perf.py to wrap dirstate in a transaction
Summary:
As part of making the dirstate always participate in the transaction,
let's wrap it's use in perf.py.

Reviewed By: quark-zju

Differential Revision: D18213036

fbshipit-source-id: cd4a7d942556f032d32cbe0630d16414b819ea34
2020-01-14 17:49:03 -08:00
Durham Goode
b2d16b60c1 dirstate: move transaction write code to its own function
Summary:
A future diff will move all dirstate writes to be in a transaction.
Let's start by moving the transaction write logic to it's own function. A future
diff will delete the non-transaction "write" function.

Reviewed By: markbt

Differential Revision: D18213554

fbshipit-source-id: 7fa414314583a64ccada0f8be287d6064024e020
2020-01-14 17:49:03 -08:00
Durham Goode
d663a9dbdf fsmonitor: move warnings into filesystem layer
Summary:
Now that the post dirstate handlers are largely unused, let's move the
fsmonitor state warning into the fsmonitor filesystem. This also allows us to
remove the temporary _newid variable that was used to allow post-transaction
detection of if the transaction happened.

Reviewed By: quark-zju

Differential Revision: D18259479

fbshipit-source-id: da88638d34a80480bbeba8b1531d523785d72c30
2020-01-14 17:49:02 -08:00
Durham Goode
bca361cd3c dirstate: remove most of post dirstate handlers
Summary:
Previously the localrepo object contained a list of handlers to invoke
after status finished executing. This was used for 1) updating the fsmonitor
state with the new clock, ignore hash, and list of notable files, and 2)
updating the list of ignored files in treestate.

As part of moving all fsmonitor subtlities behind the filesystem abstraction,
this diff moves the fsmonitorstate file updating to be in the filesystem layer.
This also moves it inside the same transaction as the lookup file cleanup, which
removes the need for the hacky self._newid logic to detect if the dirstate was
changed by an external process.

(self._newid was previously added so that our second dirstate transaction, which
updates the fsmonitorstate, could determine if the dirstate had either been
changed by this process earlier in the lookup cleanup transaction, or if it had
been changed by an external process).

Reviewed By: quark-zju

Differential Revision: D18076655

fbshipit-source-id: c5175f61432f8e121b6baacd9587fcd288f67098
2020-01-14 17:49:02 -08:00
Durham Goode
af3292ea85 filesystem: refactor lookup marking to be more extensible
Summary:
A future diff is going to move the fsmonitor post-status fixups into
the same transaction as the lookup-marking logic. To prepare for this, let's
refactor the lookup marking logic to separate out the transaciton management
from the actual marking logic.

Reviewed By: quark-zju

Differential Revision: D18076656

fbshipit-source-id: 7711d5b086be7baaff0d9736cf51c6b0a46f237e
2020-01-14 17:49:01 -08:00
Durham Goode
a064e12d93 dirstate: move lookup resolution into the filesystem
Summary:
In an earlier diff we moved the lookup-state resolution from the repo
object to the dirstate object. Now that a filesystem layer exists, let's move
lookup resolution into the filesystem layer so that filesystem.pendingchanges
can now return perfect information.

Reviewed By: quark-zju

Differential Revision: D17749373

fbshipit-source-id: d03177e5e5374bd38c6c9adaa0fe8959052de19e
2020-01-14 17:49:01 -08:00
Xavier Deguillard
b1f3e37e43 tests: the Rust ContentStore always creates .hg/store/packs
Summary:
The Python one only does it when needed, let's fix the 2 tests that were
relying on this.

Reviewed By: quark-zju

Differential Revision: D19393501

fbshipit-source-id: 94e004999045264adce2ffa17b4697583ee0fcbd
2020-01-14 13:46:26 -08:00
Xavier Deguillard
f1c8af3297 remotefilelog: test for phase instead of localcontentstore
Summary:
The phases of a commit is a more precise metric for whether a change is public
or not, let's use this instead of testing for data to be present in the local
contentstore.

Reviewed By: quark-zju

Differential Revision: D19393502

fbshipit-source-id: b1c735c8400e8d5a1a7f7013d9a4a1ad423c5023
2020-01-14 13:46:26 -08:00
Stefan Filip
eca4fbc75d tests: increase timeout for mononoke starting
Summary:
We started to see a lot of tests failing with mononoke timing out at startup.
Increasing the timeout to mitigate the issue.

Reviewed By: krallin

Differential Revision: D19397440

fbshipit-source-id: a8037102b3c8389b98a7c49824fa013fd3e6a080
2020-01-14 12:52:05 -08:00
Stefan Filip
a2ca6bb7dd manifest-tree: move private test function to lib.rs
Summary:
Moving `store_element` and `get_hgid` from testutil.rs to lib.rs in order to
get rid of some buck build warnings. They are only used in lib.rs.

Reviewed By: markbt

Differential Revision: D19350343

fbshipit-source-id: 081783d753425daaae6fadc38589da5e4e8b745d
2020-01-14 11:49:53 -08:00
Stefan Filip
2c9fa6054d manifest: update make_tree to be public
Summary:
It makes sense for manifest-tree to expose a testutil version for building a
TreeManifest. No reason for the testutil function to be limited to the crate.
The only issue is that `make_tree` is to generic so we rename the function to
`make_tree_manifest`.

Reviewed By: markbt

Differential Revision: D19350351

fbshipit-source-id: 06fe04d99bf820d3e81417f5a5514aa4b0d282e6
2020-01-14 11:49:53 -08:00