Commit Graph

2866 Commits

Author SHA1 Message Date
Durham Goode
1a3fe83b62 edenfs: remove automigrate on debugedenimporthelper
Summary:
This is causing a deadlock when an hg process takes the lock, accesses
an edenfs file, then eden launches debugedenimporthelper which also tries to
take the lock. Let's back it out for now.

Reviewed By: krallin

Differential Revision: D19331038

fbshipit-source-id: b8480c2e07079c3babeae8de48c02affa43188fc
2020-01-09 09:44:29 -08:00
Jun Wu
459ae48cef indexedlog: stabilize log::tests::test_repair_and_delete_content
Summary:
Previously, between 100 to 500 runs, the test is likely to fail:

  ---- log::tests::test_repair_and_delete_content stdout ----
  thread 'log::tests::test_repair_and_delete_content' panicked at 'assertion failed: `(left == right)`
    left: `"Verified first 141 entries, 999 of 1400012 bytes in log\nReset log size to 999\nIndex \"c\" is incompatible with (truncated) log\nRebuilt index \"c\""`,
   right: `"Rebuilt metadata\nVerified first 141 entries, 999 of 1400012 bytes in log\nReset log size to 999\nRebuilt index \"c\""`', src/log.rs:3414:9

The "Rebuilt metadata" part is missing.

This is because after D17742003, the end of `meta` is the `epoch`, which is a
random number. The test is rewriting the end of `meta` to corrupt it. That
has a chance to not corrupt the file. Change the test to corrupt the fixed
header of meta so it can always corrupt the file and therefore stabilize
the test.

Reviewed By: xavierd

Differential Revision: D19325087

fbshipit-source-id: 53d46c8a6cb9771582f8f37e27f185d303fde0ce
2020-01-09 08:48:47 -08:00
Xavier Deguillard
4322befcb0 remotefilelog: deduplicate prefetched filenodes
Summary:
The Rust Contentstore code assumes that the prefetch request doesn't contain duplicated
filenodes so let's do it on the Python side.

Reviewed By: quark-zju

Differential Revision: D19318847

fbshipit-source-id: 73375de60f4fb81021eba16506037984eebdf048
2020-01-09 07:47:00 -08:00
Xavier Deguillard
38139a9785 tests: make file fetching messages optional
Summary:
The Rust ContentStore will not print these, and in order to better understand
what tests are broken with it enabled, let's make them optional.

Reviewed By: quark-zju

Differential Revision: D19318848

fbshipit-source-id: 87a1da510b51adc40a0d08157925a990456d3e9c
2020-01-09 07:46:59 -08:00
Jun Wu
064068169b dag: rename slice to VertexName
Summary:
Per discussion, we decided to use "VertexName" as the struct name for things
like commit hashes, or the string names in tests (or the Mozilla DAG in tests).
Therefore, introduce a dedicated VertexName type and repalce all callsites to
use it.

`bytes::Bytes` is used so copying the `VertexName` is somewhat considered cheap.

This adds some overhead copying slices (and `Bytes` has some overhead). It
regresses the "building segments" benchmark from 673ms to 773ms, which seems
okay given the cleaner interface.

Reviewed By: markbt

Differential Revision: D19154905

fbshipit-source-id: 4c6d4eca67c11c10ed5f21999ccdc3f1b01695e8
2020-01-08 21:35:28 -08:00
Jun Wu
24f403368b patch: fix applying hunks to renamed files
Summary: Previously the patch logic cannot apply hunks to renamed files. Fix it.

Reviewed By: DurhamG

Differential Revision: D19271024

fbshipit-source-id: 8ba82587dd2f4b84a1d5c7e51c4c86a5e11f0ca0
2020-01-08 19:26:23 -08:00
Jun Wu
6ec68995a3 test-record: add test case for selecting chunks in a renamed file
Reviewed By: DurhamG

Differential Revision: D19271025

fbshipit-source-id: c7d39dc6ecb3bafc304e983366b2b2965e26c472
2020-01-08 19:26:23 -08:00
Jun Wu
94c7fc65d8 run-tests: share chg server for all tests
Summary:
Previously, each test uses a separate chg server, because there are different
states in different tests. So chg servers cannot be reused across tests.

Nowdays chg servers are more stateless, and can be shared for all tests.
Do that to reduce chg server startup overhead.

With this change, `./run-tests.py --time $(rg -l chg-compatible)` went down
from 4:57 to 4:04 on my devserver.

A configuration `--chg-sock-path` is added for advanced use-cases.

Reviewed By: DurhamG

Differential Revision: D18924911

fbshipit-source-id: c2f7d84ee4cbfb5d599cfdc0ab47f922fae4a67e
2020-01-08 19:24:05 -08:00
Jun Wu
232d5047cc dag: fix incorrect gca calculation
Summary: `gca(X) = gca(roots(X))`, not `gca(heads(X))`.

Reviewed By: sfilipco

Differential Revision: D19154907

fbshipit-source-id: 8d54992fbf5b03ce7ab2fbcc689c79351bf75947
2020-01-08 18:46:08 -08:00
Jun Wu
c6c4baa07e dag: redefine Id::{MIN, MAX} using Group
Summary:
Redefine Id::{MIN, MAX} using Group so Id::MAX.group() is a valid group.
Previously Id::MAX has an invalid group.

Reviewed By: sfilipco

Differential Revision: D19154908

fbshipit-source-id: 6108f02da786eb228a7d1b772ca5134c1ebb6f6b
2020-01-08 18:46:07 -08:00
Jun Wu
c4cd7df3b2 dag: extract logic calculating universal commits to a method
Summary:
Address review feedback on D18670160.

This also fixes the definition of "universal" a bit - the new code no longer
assumes there are only one head in the master group.

Reviewed By: sfilipco

Differential Revision: D19154906

fbshipit-source-id: 4fbf080354318300ce09dc7fba2dcc5f942b6308
2020-01-08 18:46:07 -08:00
Jun Wu
bcaff9062c dag: rebuild non-master id and segments if needed
Summary:
Add utilities to rebuild non-master ids and segments if necessary.

The `NamedDag` structure ensures indexes have 1:1 mapping.

Reviewed By: sfilipco

Differential Revision: D18838995

fbshipit-source-id: 4a48b183c182bd5e6336a2ca4973c36091fbbfd8
2020-01-08 18:46:07 -08:00
Jun Wu
28380e0272 dag: fix logic about avoiding unnecessary high level segments
Summary:
The check of "is this high level segment necessary or not" should be done
before dropping the last segment. This fixes an issue discovered in D18838992.

Reviewed By: sfilipco

Differential Revision: D19154904

fbshipit-source-id: fb4c83c39d66215bae168ad98e5cf78de91cc5a3
2020-01-08 18:46:06 -08:00
Jun Wu
d4eddd6b11 dag: use NamedDag in tests
Summary:
NamedDag is the high-level interface that is more interesting to test.

High-level segments are changed subtly because NamedDag syncs them to disk
first.

Reviewed By: sfilipco

Differential Revision: D18838992

fbshipit-source-id: c6a557e0a44a1d24320ea4a9e4283262f6f30f67
2020-01-08 18:46:06 -08:00
Jun Wu
2b4095e3fb indexedlog: fix warnings on Windows
Summary: Those warnings are not seen on POSIX build.

Reviewed By: sfilipco

Differential Revision: D19320441

fbshipit-source-id: feddf728eb9627834559b87d83e20f0afd9080c8
2020-01-08 18:11:38 -08:00
Jun Wu
dd79207c32 indexedlog: fix benchmarks
Summary: The `rand` APIs had changed. Update the benchmark code accordingly.

Reviewed By: sfilipco

Differential Revision: D19320122

fbshipit-source-id: 44c376b17afea52ecc548d79b44cbb50f7e9e567
2020-01-08 18:11:37 -08:00
Jun Wu
cfe135737b indexedlog: fix clippy issues
Summary: Fix issues reported by clippy.

Reviewed By: sfilipco

Differential Revision: D19320121

fbshipit-source-id: 58f6402eb2a4f52914a760299bdfa72e5a63b2e6
2020-01-08 18:11:37 -08:00
Durham Goode
2ad1e0ad7e eden: automigrate during debugedenimporthelper
Summary:
We had eden-backing repos that were using formats from years ago
because they never ran pull to trigger the automigrate. Let's automigrate as
part of debugedenimporthelper to keep these repos up-to-date.

Reviewed By: quark-zju

Differential Revision: D19320510

fbshipit-source-id: 475a6acf8748677de3183dfb5d4c6df7caca606d
2020-01-08 17:31:59 -08:00
Durham Goode
42e6ae2b55 tests: update case collision test for tag deprecation
Summary: The case collision test broke when I disabled tags.

Reviewed By: singhsrb

Differential Revision: D19321272

fbshipit-source-id: d59a0ad38b80f35931bbb4fb52e400d714ebbf37
2020-01-08 17:00:42 -08:00
Chad Austin
bf4441ee86 use unicode micro symbol when logging thrift call durations
Summary: µ

Reviewed By: wez

Differential Revision: D19191188

fbshipit-source-id: 287d208808d782179ce9e70c323f8dd45e5168c3
2020-01-08 14:43:25 -08:00
Chad Austin
91eda6e3e3 fix cmake build
Summary:
D19135344 broke EdenFS's cmake builds, and our macOS releases haven't
been going out. Rather than pass the same streaming options to the
Thrift compiler in the CMake build, remove deprecated_server_streams
from both builds.

Reviewed By: simpkins

Differential Revision: D19204397

fbshipit-source-id: c22e634ded82a683daff5c92182c8a1eb1ed4d21
2020-01-08 14:08:34 -08:00
Durham Goode
27e9ca5317 telemetry: truncate fullcommand to 256 bytes
Summary:
If users or automation invoke a bunch of really long commands, it can
occupy a lot of space in our logs. The full command for really long commands is
unlikely to be useful, so let's truncate them to 256 bytes.

Reviewed By: ikostia

Differential Revision: D19317932

fbshipit-source-id: 428479a18dd7e5a7ae4d8f862cf26c02c15b1fcb
2020-01-08 12:58:44 -08:00
Xavier Deguillard
83d46c7c3b pyrevisionstore: remove to_pyerr
Summary:
Python errors wrapped in a Rust Error would be incorrectly changed into a
RuntimeError instead of the original one. The map_pyerr method does the right
thing, and is also shorter, let's use it.

Reviewed By: quark-zju

Differential Revision: D19313883

fbshipit-source-id: 1ecb41a8eef9d41618905d6d00f199252d373f96
2020-01-08 12:52:43 -08:00
Durham Goode
d8663cb534 treestate: backout making it a hard requirement
Summary:
It turns out some old repos are still using the legacy treedirstate
format, which broke when I made this a hard requirement. Let's roll this back
for now.

Reviewed By: singhsrb

Differential Revision: D19313773

fbshipit-source-id: ded78be033cb04323a2e1d237059947dee509df9
2020-01-08 11:33:28 -08:00
Mark Thomas
43550ca5bb commitcloud: fix ordering and workspace handling for cloud sl history
Summary:
Interactive smartlog history relies on the order from the server, which has
changed, and so is now incorrect.  Sort the versions from the server so
we can be sure they're always in the right order.

Interactive smartlog history also ignores the workspace parameters to specify
the workspace.  Fix that.

Reviewed By: farnz

Differential Revision: D19309253

fbshipit-source-id: 9fc1b443e72557331e22d0987c73286f5c64e16d
2020-01-08 05:30:17 -08:00
Durham Goode
9778811573 tags: remove tags feature
Summary:
We no longer user tags in any of our repositories, so let's delete all
the code and remove some tech debt.

Previously drawdag was our last remaining use of tags, but in a previous diff I
switched that to use bookmarks.

Reviewed By: quark-zju

Differential Revision: D19042226

fbshipit-source-id: d7871c71c1026599aa70fdd806418347ea214a8a
2020-01-07 12:31:10 -08:00
Durham Goode
898149fa37 treestate: make treestate a hard requirement
Summary:
We want to delete all the non-treestate dirstate implementations. Let's
start throwing an exception if treestate is not enabled. We temporarily have a
bypass in case we break an important usecase in the process.

This also sets the standard new repo to be created in treestate mode, but adding
treestate to newreporequirements.

Reviewed By: quark-zju

Differential Revision: D19204621

fbshipit-source-id: 61c0c69ce6e559615ef570d2343bea78c1b5d5ee
2020-01-06 21:37:05 -08:00
Durham Goode
7286630863 rebase: prevent fast-foward rebases with conflicting changes
Summary:
Tweakdefaults adds the ability for `hg rebase -d` to simply fastforward
the user if the destination is a descendant of the current working copy parent.
If the working copy is dirty, it merged the files, potentially leaving the
working copy with merge conflicts that are hard to recover from.

A normal hg update disallows merges that would result in potential conflicts.
Let's update the rebase code to use the same merge logic, so fast forward
rebases simply abort if there are potential conflicts.

Reviewed By: quark-zju

Differential Revision: D19204198

fbshipit-source-id: 1b0c2c0fdab909eebe74d1a211dbfb2d780058cd
2020-01-06 15:06:50 -08:00
Mark Thomas
2158e4c531 commitcloud: add Rust rendering of cloud smartlog
Summary:
Allow rendering of cloud smartlogs using renderdag.  Use the ability of renderdag
to reserve columns to provide the smartlog-like indentation of draft stacks.

Reviewed By: quark-zju

Differential Revision: D19272577

fbshipit-source-id: 3311d83511b0d5347b45f33c282ce26b8133d047
2020-01-06 15:00:08 -08:00
Mark Thomas
8e28741a8b smartlog: add Rust rendering of graphs
Summary: Allow rendering of smartlog graphs using renderdag.

Reviewed By: quark-zju

Differential Revision: D19272578

fbshipit-source-id: 1cd19753781c58e49038d72f12c8f5ebf3bad6f9
2020-01-06 15:00:08 -08:00
Mark Thomas
da3d898cd4 cmdutil: add Rust rendering of graphs
Summary:
Allow rendering of graphs using renderdag by setting `experimental.graph.renderer` to `ascii`, `ascii-large` or `lines`.

The minimum height of each row can be set using `experimental.graph.min-row-height`.

Reviewed By: quark-zju

Differential Revision: D19272580

fbshipit-source-id: ac5eff8446bd3c09d07624a193195b4224055ffc
2020-01-06 15:00:07 -08:00
Mark Thomas
0822231d79 pyrenderdag: add python bindings for renderdag
Reviewed By: quark-zju

Differential Revision: D19272576

fbshipit-source-id: f79019d3506efe03675f49b9ad728582b22bd205
2020-01-06 15:00:06 -08:00
Mark Thomas
e91c6ffa7c renderdag: make test fixtures common
Summary:
Move the test fixtures into a common module, so that they don't need to be
repeated in each test.  Since each fixtures is now a struct, this also makes it
clearer what each of the items are.

Reviewed By: quark-zju

Differential Revision: D19288290

fbshipit-source-id: 394805c652592177f11ccb096b8e5e95361456e4
2020-01-06 15:00:06 -08:00
Mark Thomas
cdd0f39805 renderdag: add test case for long messages
Reviewed By: quark-zju

Differential Revision: D19288289

fbshipit-source-id: d7d7023900d4f7c8d1ed14d15217548875a349c6
2020-01-06 15:00:05 -08:00
Mark Thomas
693ca3ff6a renderdag: Use builder pattern for output renderers
Summary:
Generalize construction of output renderers (renderers that render to `String`)
to avoid duplication of options.  At the moment there is only one, but later we
may add new options.

This also allows us to construct output renderers from any renderer that
renders to `GraphRow`, which means we can create adapters that modify or
re-order the rows of the graph.

Reviewed By: quark-zju

Differential Revision: D19286350

fbshipit-source-id: a5649ca2f48e263ee24584339179655fb612d3d1
2020-01-06 15:00:05 -08:00
Mark Thomas
213b3f086c renderdag: add a DAG rendering crate
Summary:
A new implementation for rendering DAGs.

This new crate implements a generic DAG renderer, that can convert a
topologically sorted sequence of DAG nodes into a sequence of strings suitable
for rendering to a terminal.

The new renderer differs from the old renderer in a few important ways:

* It prioritizes keeping commits linear, and will allow gaps to form if
  that will allow the history of the commits to be kept in a straight
  line.  This makes it easier to track long parallel histories.

* It supports octopus merges (nodes with more than two parents).  Even
  though Mercurial doesn't support octopus merge commits, summary DAGs
  with omitted nodes can still end up with logical octopus merges.

* It supports reservation of columns for specific nodes.  This can be
  used to support smartlog-style indentation of draft stacks without
  needing to hack around it by creating fake nodes.

* It separates out forming the graph from generating the lines.  This
  allows multiple back-ends for generating different styles of graph.

There are three back-ends implemented:

`AsciiRenderer` renders similar to the old graph renderer, using ASCII
characters.  For example:

```
  o  F
  |
  | o  E
  | |
  | | o  D
.-----'
o | |  C
+---'
o |  B
:/
o  A
```

`AsciiLargeRenderer` uses larger ASCII blocks to give a clearer picture of
complex graphs.  For example:

```
   o   F
   |
   |
   |  o   E
   |  |
   |  |
   |  |  o   D
  ______/
 / |  |
o  |  |   C
| ___/
|/ |
o  |      B
: /
:/
o   A
```

`BoxDrawingRenderer` uses Unicode box drawing characters to give a more
continuous rendering of the graph, however requires support for these
characters in the terminal font.  For example:

```
  o  F
  │
  │ o  E
  │ │
  │ │ o  D
╭─────╯
o │ │  C
├───╯
o │  B
├─╯
o  A
│
~
```

Reviewed By: quark-zju

Differential Revision: D19272579

fbshipit-source-id: bb6fa4685c965544cc3b6b9261df3a3ec161b41f
2020-01-06 15:00:04 -08:00
Jun Wu
8e3a606106 graphql: do not error out if signal_summary is None
Summary:
This can happen for "Reserved" diffs.

Reported By: hyr

Differential Revision: D19270860

fbshipit-source-id: 24e9b8c08fc16e3089e0dc3f8064a5e9671d9ace
2020-01-06 14:27:43 -08:00
Genevieve Helsel
a03dac0a83 reactivate test_abort_split_with_pending_add
Reviewed By: chadaustin

Differential Revision: D19167578

fbshipit-source-id: fa2c848a4e157bcf9c466b4b65132168201e4d97
2020-01-06 08:52:27 -08:00
Genevieve Helsel
26f4427200 add eden debug parents
Summary: adds a cli debug command to inspect the working copy parent. by default just returns eden's snapshot contents, but adds optional --hg flag to print Mercurial's dirstate information

Reviewed By: chadaustin

Differential Revision: D19167518

fbshipit-source-id: b65e112df6abe4e0e7a8a528a90b2e3d17297e66
2020-01-06 08:52:27 -08:00
Jon Maltiel Swenson
cc1a7ddcde Kill rsocket server stack
Summary: This stack is deprecated in favor of the rocket stack.

Reviewed By: rhodo

Differential Revision: D18123600

fbshipit-source-id: db6e3bf4add9c49cf8f1d2f12f97b207e87c7302
2020-01-03 15:53:10 -08:00
Jun Wu
a97842c8c2 fbscmquery: fix globalrev graphql query
Summary: `"globalrev"` is invalid. Use `GLOBAL_REV` instead.

Reviewed By: singhsrb

Differential Revision: D19271059

fbshipit-source-id: c26347b92b1b92a5f958f2f71dbdfdf3635a0876
2020-01-03 12:40:04 -08:00
Jun Wu
3db941beb4 phabstatus: handle JSON decode error
Summary:
Sometimes GraphQL does not return JSON result. Do not crash on the client-side.

Context:
https://fb.workplace.com/groups/scm/permalink/2565497036833234/

Reviewed By: farnz

Differential Revision: D19254348

fbshipit-source-id: 75ca93b1dc8648d5c59915ecf450b9366df13c01
2019-12-30 16:45:11 -08:00
Simon Farnsworth
deacaed311 Don't delete bookmarks in drawdag if there are none
Summary: In Mononoke tests, we have failures because `hg book -fd` aborts if it's not given any bookmark names. Fix up this function to not run `hg book -fd` if there are no bookmarks

Reviewed By: quark-zju

Differential Revision: D19251930

fbshipit-source-id: 2f1696d9207ad64a94c28941e72806d9b0e686fe
2019-12-30 15:41:14 -08:00
Adam Simpkins
e45e2c6f9a unbreak dirstate.status() in EdenFS repositories
Summary:
D19190259 broke `dirstate.status()` in EdenFS checkouts.  Fix the code to
properly check if the dirstatemap supports `nonnormalsetfiltered`

Reviewed By: quark-zju

Differential Revision: D19221147

fbshipit-source-id: 17f69f5b461467bfb67858fb21dc0230b928a740
2019-12-23 16:44:12 -08:00
Jun Wu
c214a2de9e test-profile: fix the test
Summary: Use `debugpython` to replave `python`

Reviewed By: akushner

Differential Revision: D19221125

fbshipit-source-id: 68f5d83f9abe38174492fdaaf7496875696c4302
2019-12-23 16:37:10 -08:00
Jun Wu
3170aa7cbc test-parse-index2: remove the test
Summary:
The test is broken by D19190496 because the test uses some weird test cases
about legacy Pythons.

The "parseindex2" test is revlog-related. The revlog is kind of stable, tested
indirectly in countless tests, and is also going away. We don't run the code
on legacy versions of Pythons.The "parseindex2" test does not provide much
value. Therefore just remove it.

Reviewed By: akushner

Differential Revision: D19221126

fbshipit-source-id: 5b904d97caa47f92098435ee85c1bb8b725c9f84
2019-12-23 16:37:10 -08:00
Jun Wu
9d36855838 bindings: add hgtime
Summary:
This makes it possible to use the hgtime parsing utilities in Python.
Namely, patterns like `since 6 days ago to now` can now be parsed.

Reviewed By: sfilipco

Differential Revision: D18946335

fbshipit-source-id: c12ea3437301f20a8c457525c0798f471b8f174a
2019-12-22 11:27:33 -08:00
Jun Wu
624e47ac1c hgtime: add a way to change "now" for testing
Summary:
The existing tests rely on this feature.
This requires replacing all `Local::` usage with `HgTime::now()`.

Reviewed By: sfilipco

Differential Revision: D18946336

fbshipit-source-id: 1a92183cc96f2fcb013b68a32e78f09f28a49452
2019-12-22 11:27:33 -08:00
Jun Wu
8812c8c97d hgtime: fix bounds of "a to b" range
Summary:
For example, `2018 to 2019` should be parsed as `2018-1-1 to 2019-12-31 23:59:59`,
not `2018-1-1 to 2019-1-1`.

Fix it to match the desired behavior.

Reviewed By: sfilipco

Differential Revision: D18946331

fbshipit-source-id: 640bbe49352d1f01855183ea9927bb0009646243
2019-12-22 11:27:33 -08:00
Jun Wu
c624847c84 hgtime: limit year in 1900 to 9999
Summary:
The Python `datatime` stdlib has limitations:

- `datetime.datetime` can only express year 1 to 9999.
- `strptime` requires year >= 1900.

Limit the `HgTime` to 1900 ..= 9999 range to be consistent.

Note: Mercurial has an `i32` range limit, which seems to problematic
since `i32::MAX` is Jan 19, 2038. So I kept using `i64`.

This addressed some XXX comments about not returning errors.

The code change is mostly because added error handling.

Reviewed By: sfilipco

Differential Revision: D18946333

fbshipit-source-id: 0e4756457b0f13451dc5008ef19d4670a7aaa7fb
2019-12-22 11:27:32 -08:00