Commit Graph

3032 Commits

Author SHA1 Message Date
Stefan Filip
d78982a6e8 dag: move iddag to own file
Summary:
Separate Segment and IdDag in two individual files. This is preparation for
refactoring IdDag to be more flexible in terms of storage. That will probably
involve moving stuff out of IdDag into a new file that deals with the storage
abstractions.

Reviewed By: quark-zju

Differential Revision: D19559127

fbshipit-source-id: b3b9b18e2653157e69148b1f29292a57b30016ec
2020-01-24 15:49:54 -08:00
Mark Thomas
d35692711e tests: convert test-rebase-obsolete to mutation
Summary:
Add `test-rebase-mutation.t`, based on `test-rebase-obsolete.t`, but instead
testing specifically for mutation and visibility behaviour.

Reviewed By: xavierd

Differential Revision: D19549473

fbshipit-source-id: c993686eab47985737eef17ab0f52f31d0929159
2020-01-24 14:47:39 -08:00
Mark Thomas
c0e72db0ad tests: enable mutation and visibility in some rebase tests
Summary:
Enable mutation and visibility for the existing tests that use evolution.

Excludes `test-rebase-obsolete.t` which specifically targets obsmarkers - this
will be converted to a new test in D19549473.

Reviewed By: quark-zju

Differential Revision: D19502220

fbshipit-source-id: 70b2828b60220c6887a6210e91bed01c2809cc51
2020-01-24 14:47:39 -08:00
Mark Thomas
f2d5739705 mutation: allow instabilities if mutation is enabled
Summary:
Disallowing instabilities is a feature of evolution.  For mutation tracking
there is no problem with instabilities.

Reviewed By: quark-zju

Differential Revision: D19502221

fbshipit-source-id: c21a5021bf1a8b29abb6635928da50895e2cec33
2020-01-24 14:47:38 -08:00
Mark Thomas
980829029a mutation: include all commits as predecessors in rebase --collapse
Summary:
Rebase with `--collapse` should behave like `fold` and set all the rebased
commits as predecessors of the new commit.

Reviewed By: quark-zju

Differential Revision: D19502223

fbshipit-source-id: a903ee5647bb4ee947a6cc68d8ca791fe0f1299d
2020-01-24 14:47:38 -08:00
Mark Thomas
398e175762 visibility: ensure visibleheads and store requirements are copied on clone
Summary:
When cloning a repo, ensure the store requirements and visibleheads files are
included in the clone.

Reviewed By: quark-zju

Differential Revision: D19502224

fbshipit-source-id: e3ead586fb1fe10def02d55e33810643c0c78bc8
2020-01-24 14:47:38 -08:00
Mark Thomas
845e35ea5f metalog: ensure legacy files are written out atomically
Summary:
`hg clone` can hardlink store files if the repos are on the same device.  This
means we need to write out the legacy files using an atomic temp file, so that
the hardlink is broken.  This also makes sure anyone watching the file sees the
change atomically.

Reviewed By: quark-zju

Differential Revision: D19502222

fbshipit-source-id: 3e9fb35846111c9d924de3c6b6861ce5e295e035
2020-01-24 14:47:37 -08:00
Mark Thomas
39f8b3109f visibility: add visibleheads to journaled files
Summary:
The visibleheads file should be journaled like the other state files, so that
it is rolled back if a transaction aborts.

Reviewed By: quark-zju

Differential Revision: D19502226

fbshipit-source-id: c2b8d926ce97211411ca01f05e83e62cb77625b9
2020-01-24 14:47:37 -08:00
Mark Thomas
4bc06df032 visibility: allow bundlerepo visibleheads to be updated
Summary:
The `incoming` command works by creating a bundlerepo and then updating the
commits as if a pull occurred.  This means the visibleheads can get updated,
which was previously assumed impossible.

We can just accept the new set of visibleheads as-is.  There may be some
unnecessary heads that are covered by other heads, but since the bundlerepo
is short-lived, this isn't a problem.

Reviewed By: quark-zju

Differential Revision: D19502219

fbshipit-source-id: ce9e98c6a40f96f5305bc7580efb476d5264fc88
2020-01-24 14:47:37 -08:00
Mark Thomas
2859b55829 mutation: the workingctx is not obsolete
Summary: It's possible for the workingctx to be queried for obsolescence.  Make sure it is not considered to be obsolete.

Reviewed By: quark-zju

Differential Revision: D19502225

fbshipit-source-id: a1a37afd39a0fc16f95b4eb917372f193ac3fe82
2020-01-24 14:47:36 -08:00
Chad Austin
83bdc4120b when the import helper fails to start, indicate how to view the log
Summary:
It's not often obvious to people where edenfs.log is located on the
filesystem, so redirect people to `eden debug log` instead.

Reviewed By: simpkins

Differential Revision: D19560709

fbshipit-source-id: 6b031c80bab2c21dfddba4d7ec37d4e70218ea08
2020-01-24 13:22:18 -08:00
Chad Austin
9db11e9454 have eden debug log show the end of the log file by default
Summary: 90% of the time, it's more useful to see the end of the log than the beginning.

Reviewed By: simpkins

Differential Revision: D19560473

fbshipit-source-id: f9b5a922d284053d6afc8ea889586826e3c58582
2020-01-24 13:22:18 -08:00
Jia Chen
c717ef0d7d Update pyre version for eden
Summary: Automatic upgrade to remove `version` override and silence errors.

Differential Revision: D19548623

fbshipit-source-id: 12d5e8ec6450de42107d07b230e4b2d0824f7c9d
2020-01-24 12:32:43 -08:00
Lukas Piatkowski
8c7de41d8c arc/licenselint: add license linting for lua and sql
Summary: The fbcode/linttool has been updated to support lua and sql here D19467627, so updating arc rules to cover the new extensions

Reviewed By: zertosh

Differential Revision: D19534326

fbshipit-source-id: 4e615653acedcdf2da88e7dc4b443ad22af1324f
2020-01-24 06:13:28 -08:00
Jun Wu
52af332c28 renderdag: add tests showing how orders affect rendering
Summary:
I wrote it to understand how renderdag draws the same graph with different
orders. It seems useful for future optimization that tries to reduce the number
of columns. So let's check it in.

Reviewed By: xavierd

Differential Revision: D19440713

fbshipit-source-id: 8bc580799f6b24c87886d5ac306020f50bb694e5
2020-01-23 20:50:56 -08:00
Victor Zverovich
09618791bd Update fmt to version 6
Summary: As title

Reviewed By: yfeldblum

Differential Revision: D17741047

fbshipit-source-id: e921a2fef99a9b6f11e09758e69d99a54815054a
2020-01-23 19:33:14 -08:00
Jun Wu
29c749ef7d dag: add fuzz tests on the octopus DAG
Summary: This gives us some confidence about octopus merge handling.

Reviewed By: DurhamG

Differential Revision: D19540726

fbshipit-source-id: e84de74aecae54429483edd185d39fd1bd858f87
2020-01-23 17:58:51 -08:00
Jun Wu
8ac97da54e bindag: make TestContext more flexible
Summary:
TestContext uses ParentRevs. That limits parents to at most 2.

Use a type parameter so we can opt-in Vec<usize> for octopus merge support,
at the cost of worse cache efficiency.

Reviewed By: DurhamG

Differential Revision: D19540727

fbshipit-source-id: f9e8de151b7b296fd6f0fd89be9de2b8996634c7
2020-01-23 17:58:51 -08:00
Jun Wu
df23791d08 bindag: add some octopus examples
Summary:
Our new algorithms support octopus merges. However there were no tests using
octopus merges. This diff adds a simple one.

Reviewed By: DurhamG

Differential Revision: D19540728

fbshipit-source-id: 8411024f0b7e27c2ebfabbe1935496124c25df7b
2020-01-23 17:58:51 -08:00
Jun Wu
494bdae7cc dag: add a fuzz test about range algorithm
Summary:
The test runs the old and new algorithm and compares their result.  This is more
interesting than using random numbers, since the fuzzing framework will try to
explore new code paths.

Reviewed By: sfilipco

Differential Revision: D19511576

fbshipit-source-id: e9a2066769b54a60bb92643e5715f91a6fccbcb5
2020-01-23 17:58:50 -08:00
Jun Wu
78ea96cb9d bindag: port range algorithm from hg
Summary:
The ported algorithm will work as a comparison to verify dag's range
implementation.

Reviewed By: sfilipco

Differential Revision: D19511574

fbshipit-source-id: 589353d6e6c91b8d6707c977eeb8558ac733b525
2020-01-23 17:58:50 -08:00
Chad Austin
68595a14a1 fix some macOS build warnings
Reviewed By: fanzeyi

Differential Revision: D19548525

fbshipit-source-id: 0f137dab2f5af659842cbbc120e2e12f91d18276
2020-01-23 17:37:39 -08:00
Chad Austin
f35f921d5d allow garbage collection of cached tree objects again
Summary:
Now that loading too many blobs only GCs the blob cache, treat cached
trees as ephemeral again.

Reviewed By: simpkins

Differential Revision: D19358375

fbshipit-source-id: 191427f46276f0f5d024b5bc923a68f7a42044da
2020-01-23 17:33:43 -08:00
Stefan Filip
2dc14cf9a6 manifest: use Rust Tree Manifest by default
Summary:
The Rust Manifest is deployed practically everywhere.
Removing the C++ code from being compiled in Mercurial along with all of the cstore extension.

Reviewed By: quark-zju

Differential Revision: D19543248

fbshipit-source-id: d632e171175e6866563c1aa0808a099b67bd937d
2020-01-23 16:06:51 -08:00
Stefan Filip
f5280b75e9 thrift: update thrift generated files
Summary: Commit updates after having ran `make local`

Reviewed By: xavierd

Differential Revision: D19543278

fbshipit-source-id: 00fdc3ebec32e8a3d706b89402dc91f771984c3c
2020-01-23 16:06:51 -08:00
Xavier Deguillard
8041ff9e82 bisect: remove revision number in output
Summary: Revision numbers are deprecated, let's not print them in the bisect output.

Reviewed By: DurhamG

Differential Revision: D19539678

fbshipit-source-id: 29bca3ee38cb390bdef8dd88cb912aa3dc452065
2020-01-23 15:54:00 -08:00
Xavier Deguillard
31b266d32c graft: remove revision number in output
Summary: Revision number are deprecated, remove it from the user output.

Reviewed By: DurhamG

Differential Revision: D19539679

fbshipit-source-id: 7fc618e832e0e5bb3464d2ed04967b062b6f182a
2020-01-23 15:53:59 -08:00
Puneet Kaushik
32cc7c1eca Enable listMounts to get eden list working
Reviewed By: simpkins

Differential Revision: D19510624

fbshipit-source-id: e097c5b412e62d6856179db971296f8389ec8991
2020-01-23 12:41:39 -08:00
Michael Devine
3f326e911c Repo converter: Add named tuple for commit spec
Summary: The converter has many anonymous tuples being passed around, and it leads to typing error and pain figuring out which field has which data. This diff uses named tuples to implement an important tuple in the code that we use to merge repo history into a single unified history.

Reviewed By: tchebb

Differential Revision: D19377226

fbshipit-source-id: cb7d49227752fd7bc98233f6c0d1b2913cb24b05
2020-01-23 12:36:29 -08:00
Xavier Deguillard
ff09f797f4 demandimport: disable in python3
Summary:
We don't need to it for the initial phases of the conversion. I've seen cases
were exceptions while importing were also hidden and caused weird behavior down
the line.

Reviewed By: DurhamG

Differential Revision: D19539064

fbshipit-source-id: 7415d88ef8fb69c949ed557ab0df7e6f7b39bf86
2020-01-23 11:41:02 -08:00
Wez Furlong
43af241301 eden: remove hypothesis test bits
Summary:
Sadly, we didn't derive as much value from these as we would
have liked, and now they are a source of flakeyness in our tests.

Reviewed By: genevievehelsel

Differential Revision: D19460792

fbshipit-source-id: 48c82cc2d1fdbd81bece4057e93799fbcc4f4725
2020-01-23 11:37:27 -08:00
Xavier Deguillard
fe8855e9fb hgmain: initialize fbinit
Summary:
The memcache code assumes it's initialized, let's actually do it.

The test change is required due to fbinit installing its own SIGTERM handler
that shows a backtrace. Installing our own SIGTERM handler makes it closer to
what Mercurial does already.

Reviewed By: quark-zju

Differential Revision: D19518698

fbshipit-source-id: bc8c2311e65c9c00678756abae3979ccda4453ea
2020-01-23 08:57:22 -08:00
Xavier Deguillard
b6589bde84 revisionstore: prefetch takes &[Key] instead of Vec<Key>
Summary: This can prevent potential moves and clones on the caller of prefetch.

Reviewed By: quark-zju

Differential Revision: D19518697

fbshipit-source-id: 63839fc3f4bb9ca420e290eabaffb481a3584f7b
2020-01-23 08:57:22 -08:00
Jun Wu
fff2cb833f dag: add a fuzz test about gca algorithm
Summary:
The test runs the old and new algorithm and compares their result.  This is more
interesting than using random numbers, since the fuzzing framework will try to
explore new code paths.

This cannot run on stable Rust yet. I added a README for how to run it.

Reviewed By: sfilipco

Differential Revision: D19504096

fbshipit-source-id: 621da02c50a771dee9932f9d7a407cb1f412a543
2020-01-22 19:30:50 -08:00
Jun Wu
af85f4ff3b bindag: add a way to get a subdag of a parsed bindag
Summary: Sometimes the graph is too large. Provide a way to slice it.

Reviewed By: sfilipco

Differential Revision: D19511575

fbshipit-source-id: 504317d6894764043b23ea49dcf09c8cdea96961
2020-01-22 19:30:49 -08:00
Jun Wu
b5482f8976 bindag: add utilities for easier testing
Summary:
As we plan to test the dag crate with some other DAG implementation,
add a convenient structure that setups both DAG implementations.

Reviewed By: sfilipco

Differential Revision: D19503371

fbshipit-source-id: 3e9933ad37301bfac36eb1af6d82b4298af778b6
2020-01-22 19:30:49 -08:00
Jun Wu
7d11508dfa bindag: port GCA algorithm from hg
Summary:
The ported algorithm will work as a comparison to verify dag's gca
implementation.

Reviewed By: sfilipco

Differential Revision: D19503373

fbshipit-source-id: f5253db89fbcdc2fd02f3fdaa0796e24338b1fba
2020-01-22 19:30:49 -08:00
Jun Wu
a98d288938 bindag: apply smallvec optimization
Summary:
This is similar to D17581248. It will make the old linear-scan algorithm (which
will be added later) about 5x faster.

Reviewed By: sfilipco

Differential Revision: D19503372

fbshipit-source-id: c65d7217e7b144603dadd57f54a5e70f513c8e51
2020-01-22 19:30:48 -08:00
Jun Wu
64271f24ba dag: move bindag from benches to a separate crate
Summary: This allows bindag to be used outside benches.

Reviewed By: sfilipco

Differential Revision: D19503374

fbshipit-source-id: 131061f7d1d28125875a86afc330dbb9634249cf
2020-01-22 19:30:48 -08:00
Stanislau Hlebik
f8ba0c99a5 infinitepush: introduce infinitepushbookmark path
Summary:
Previously we had infinitepush path which was preferred over a default path when a
specific revision was pulled or updated to (i.e. "hg pull" would use
paths.default, but "hg pull -r HASH" would use paths.infinitepush if this path
is set in the config).

I'd like to have infinitepushbookmark path which will be preferred for all
cases except for a single commit pulls.

Reviewed By: mitrandir77

Differential Revision: D19513490

fbshipit-source-id: 7f8b809f937aa2e082c2e9cd382709c6db619a4f
2020-01-22 14:05:38 -08:00
Jun Wu
fcaef9ed7e sparse: make sparse profile tracking cheaper
Summary:
Reading all the sparse profiles can be very expensive.
This changes the format we log to the cloud. It seems nobody is depending on the old format.

Reviewed By: wlis

Differential Revision: D19517306

fbshipit-source-id: 256e78acd4c24aceb533aa4d66b90ecf885fcbcd
2020-01-22 13:41:34 -08:00
Xavier Deguillard
d997f31b93 setup: properly install the binaries
Summary:
Recently, the Rust extensions went away (D19340538) but that had the negative
side effect of no longer installing all Rust binaries. Make sure the
{build,install}_rust_ext steps are run properly when Rust binaries are present.

Reviewed By: quark-zju

Differential Revision: D19509526

fbshipit-source-id: 03f62f41595931dd7d5f32544a35d31e20fa3f0c
2020-01-22 09:29:18 -08:00
Harvey Hunt
c5f328be68 hg: Remove purge extension from preimported modules
Summary:
D19506964 moved the purge extension into core, however purge is still
mentioned in dispatch.py's list of external modules to preimport. This causes hg
to crash as such:

```
  Traceback (most recent call last):
    File "/data/users/harveyhunt/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/__init__.py", line 61, in run
      dispatch.runchgserver()
    File "/data/users/harveyhunt/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/mercurial/dispatch.py", line 335, in runchgserver
      _preimportmodules()
    File "/data/users/harveyhunt/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/mercurial/dispatch.py", line 320, in _preimportmodules
      extensions.preimport(extname)
    File "/data/users/harveyhunt/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/mercurial/extensions.py", line 180, in preimport
      mod = getattr(__import__("edenscm.hgext.%s" % name).hgext, name)
  ImportError: No module named purge
```

Remove purge from the extmods list.

Reviewed By: ikostia

Differential Revision: D19514862

fbshipit-source-id: 6e16ff160f0bc842038b1481bf23aab9c608685d
2020-01-22 08:18:09 -08:00
Puneet Kaushik
a8e8d9813e Pass the repository path as string instead of path
Summary: Passing the checkout path was throwing an error on Windows - "error: argument of type 'WindowsPath' is not iterable". Passing it as string to fix it.

Reviewed By: simpkins

Differential Revision: D19507974

fbshipit-source-id: 2a91dc37d2497a30a70c684f5298410956a6acca
2020-01-21 22:08:55 -08:00
Saurabh Singh
17ad617625 purge: move extension to core
Reviewed By: quark-zju

Differential Revision: D19506964

fbshipit-source-id: 173918346e90fadbaa52155786aa30fbd4951c8c
2020-01-21 20:49:08 -08:00
Chad Austin
f7491f7741 add independent cache size limits per object type
Summary:
Instead of clearing every single cached object when the total size
exceeds the ephemeral storage limit, keep a limit per object type and
only clear those that exceed their quota.

Reviewed By: simpkins

Differential Revision: D19358312

fbshipit-source-id: 6918d6f4cc2931aed79a9025d0e0f357ede515e0
2020-01-21 19:41:47 -08:00
Chad Austin
e5e9d8cf69 clear the deprecated blobsize keyspace at startup
Summary:
EdenFS no longer uses the blobsize keyspace. To avoid wasting space,
clear and campact it on startup, along with any keyspaces we
deprecated in the future.

Reviewed By: fanzeyi

Differential Revision: D19354881

fbshipit-source-id: 5285757a0e44ab1080c5f940283e06b17bec811d
2020-01-21 10:33:11 -08:00
Chad Austin
46259647e2 deprecate getManifestEntry
Summary:
Indicate getManifestEntry is deprecated so we remember to delete it
down the line.

Reviewed By: genevievehelsel

Differential Revision: D19459793

fbshipit-source-id: 86dd173cfb12c36cf6ba19eaf4be96dfe624c05f
2020-01-21 10:33:10 -08:00
Chad Austin
36954d229d stop calling getManifestEntry and instead look directly in the manifest
Summary:
getManifestEntry has an inefficient implementation. It can result in
multiple round trips to the Mercurial importer process and under heavy
usage it can clog the Thrift worker thread pool. Moreover, it only
provides data that Mercurial already knows. Instead of calling
getManifestEntry, look in the manifest to see if the entry exists.

Reviewed By: quark-zju

Differential Revision: D19426507

fbshipit-source-id: 67bcd7e0ae62f84b313aa6a7203832fb9a10f315
2020-01-21 10:33:10 -08:00
Chad Austin
d23773e9b0 restructure KeySpace and move it into KeySpace.h
Summary: Simplify the definition and use of KeySpace and move it into its own header.

Reviewed By: simpkins

Differential Revision: D19353441

fbshipit-source-id: ef07677d927a48839b709711388abeb3c1ed9679
2020-01-21 10:33:10 -08:00