Commit Graph

56318 Commits

Author SHA1 Message Date
Jun Wu
e5ef3a81b2 changelog: do not treat draft remove heads as unconditionally visible
Summary: This makes it possible to hide draft branches using `hg hide`.

Reviewed By: markbt

Differential Revision: D20403505

fbshipit-source-id: d316e02c24ce636fdc6593f95a5d974b1ba0fc85
2020-03-12 13:02:44 -07:00
Mark Thomas
d37a1fe4ff commitcloud: write cloud sync state within the transaction
Summary:
During cloud sync, the `_applycloudchanges` step will pull all the new commits
from commit cloud, and then record that it has updated to the new workspace
version in the cloud sync state.

If the transaction subsequently fails and gets rolled back, we will roll back
the pull (so the new commits vanish) but we will not roll back the recording of
having updated to the new cloud workspace version.

On the *next* cloud sync, commit cloud will think the user has deliberately
hidden all of those missing commits, and remove them from the cloud workspace.

Write the cloud sync state within the transaction, so if the transaction is
rolled back, we also roll back the recording of the new workspace version.

Reviewed By: quark-zju

Differential Revision: D20420240

fbshipit-source-id: f3d201f299e491cbff47fc6b703a4e1af4015b18
2020-03-12 12:58:17 -07:00
Mark Thomas
d4cfd6f918 commitcloud: store cloud sync status in the store vfs
Summary:
Commit cloud sync works at the store granularity, so the status should be
stored in the store vfs.  Move it there.

Improve the output of `hg cloud status` by logging more information about the workspace.

Reviewed By: quark-zju

Differential Revision: D20419223

fbshipit-source-id: 4f0d6b9aab55d2bbeaf89b489606f0bc25400de5
2020-03-12 12:58:17 -07:00
Arun Kulshreshtha
7a22f95631 mononoke_api: use fbinit::compat_test
Summary:
Avoid boilerplate in unit tests by consistently using `fbinit::compat_test`.

This is a very mechanical change; most of the modified lines are due to indentation changes.

Reviewed By: krallin

Differential Revision: D20414564

fbshipit-source-id: 11c6bd702de28c8a21978f0345d2b0ff8a574e72
2020-03-12 12:34:07 -07:00
Jun Wu
6908bc874c pytracing: support basic filtering in treespans
Summary:
Expose the "TreeSpans" type from tracing-collector so filtering can be done on
them.

Reviewed By: DurhamG

Differential Revision: D19797707

fbshipit-source-id: 56b63c8fb79865666ce923612dbd5a9cc512dce6
2020-03-12 12:19:39 -07:00
Jun Wu
ad8190713b cpython-ext: serialize Rust str into Python str type
Summary:
Previously Rust str was serialized into bytes. To be Python 3 friendly, let's
serialize it into `str`.

Reviewed By: markbt

Differential Revision: D19797706

fbshipit-source-id: 388eb044dc7e25cdc438f0c3d6fa5a5740f22e3d
2020-03-12 12:19:38 -07:00
Jun Wu
3376363721 tracing-collector: add is_event to TreeSpan
Summary: Expose the is_event property via public APIs.

Reviewed By: DurhamG

Differential Revision: D19797705

fbshipit-source-id: f441825e98208964f7b3d6815a177b464430cbb7
2020-03-12 12:19:38 -07:00
svcscm
1e3574da1f Updating submodules
Summary:
GitHub commits:

e5ccead7a0
c15e85bdcb

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: a51cfd356820779a64d05b21468383ff6fce9f1a
2020-03-12 12:15:13 -07:00
Stanislau Hlebik
e3bf91e944 mononoke: add generate_placeholder_diff option
Summary:
Let's use functionality added in D20389226 so that we can generate
diffs even for large files.

I've contemplated between two approaches: either "silently" generate
placeholder diff for files that are over the limit or add a new option
where client can request these placeholders. I choose the latter for a few
reasons:

1) To me option #1 might cause surprises e.g. diffing a single large file
doesn't fail, but diffing two files whose size is (LIMIT / 2 + 1) will fail.
Option #2 let's the client be very explicit about what it needs - and it also
won't return a placeholder when the actual content is expected!
2) Option #2 makes the client think about what it wants to be returned,
and it seems in line with source control thrift API design in general i.e.
commit_file_diffs is not trivial to use by design to because force client to
think about edge cases. And it seems that forcing client to think about additional
important edge case (i.e. large file) makes sense.
3) The code change is simpler with option #2

I also thought about adding generate_placeholder_diff parameter to CommitFileDiffsParams,
but that makes integration with scs_helper.py harder.

Reviewed By: markbt

Differential Revision: D20417787

fbshipit-source-id: ab9b32fd7a4768043414ed7d8bf39e3c6f4eb53e
2020-03-12 12:00:26 -07:00
Thomas Orozco
62a027768f mononoke: add a test for BFS fetching over SSH
Summary:
This adds a test demonstrating that we can perform BFS fetching over SSH. The
test should demonstrate that the fetches are:

- Done in a BFS fashion (we don't fetch the entire trees before comparing them,
  instead we do a fetch at each layer in the tree). In particular, note that
  the cc tree, which is unchanged, doesn't get explored at all.
- Done using the right paths / filenodeids, and therefore the right linknodes
  are located.

Reviewed By: farnz

Differential Revision: D20387124

fbshipit-source-id: b014812b0e6e85a5cdf6abefe3fe4f47b004461e
2020-03-12 11:16:07 -07:00
Thomas Orozco
999333424e mononoke/server: add support for logging Scuba to a file
Summary:
This is convenient because it makes it possible to tell what is going on within
Mononoke from the outside (e.g. introspect perf counters).

I'll land this after D20387115.

Reviewed By: farnz

Differential Revision: D20387125

fbshipit-source-id: ee070c4d658a0ec8f232152fe8b34bd0b56e6888
2020-03-12 11:16:07 -07:00
svcscm
b77cec0d44 Updating submodules
Summary:
GitHub commits:

b2a8577080
543fef6260
82f46e6745
02f23c3b6f
07a3f7f008

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: a64cd72832d871b071c69de37e2e0acd7a55a583
2020-03-12 10:40:01 -07:00
Jun Wu
32d3077147 windows: use getfstype from bindings.fs
Summary: A left over of D20313385.

Reviewed By: xavierd

Differential Revision: D20423035

fbshipit-source-id: 0dc5aae3660725ada8e1b311bb82e6429b001289
2020-03-12 10:16:15 -07:00
Arun Kulshreshtha
5c023bb37d treemanifest: add BFS fetching over SSH
Summary:
This diff adds client-side support for the new `designatednodes` mode of the `gettreepack` wireproto command (implemented in D20307442), which allows fetching an exact set of specified tree nodes. This enables the performance benefits of BFS tree fetching over SSH (whereas this sort of fetching was previously only possible over HTTPS via EdenAPI).

To allow us to smoothly control the rollout of this feature, it is gated behind the `treemanifest.ondemandfetch` config option. In theory, this should not be needed since the client will check that the server supports `designatednodes` before attempting to do on-demand fetching, but if users report issues it's nice to have a killswitch.

Reviewed By: quark-zju

Differential Revision: D20382825

fbshipit-source-id: f9aef3fdae08ae84a9720deead3549fe2fba0b9b
2020-03-12 09:59:47 -07:00
Thomas Orozco
86ad431e16 mononoke/fastreplay: add an replay_outcome_permille counter
Summary:
Similarly to D20418610, let's keep track of request failure rates in ODS
counters so they can be health-cheked by SF.

Reviewed By: farnz

Differential Revision: D20418705

fbshipit-source-id: 1d281e06c59414a0610836106370592596b47e39
2020-03-12 08:18:11 -07:00
Thomas Orozco
b60606cf9a mononoke/request_handler: log success permille to its own counter
Summary:
Earlier, I added counters for successes and failures, but unfortunately
canaries do not allow for tracking formulas, so I also need the actual request
success rate logged.

More broadly speaking, it probably does make sense to have a very high level
"success rate" counter to query anyway. This adds it.

Reviewed By: farnz

Differential Revision: D20418610

fbshipit-source-id: 94f115f8201aef4f6498f4da98e170194186f0f8
2020-03-12 08:13:09 -07:00
svcscm
d0f2684946 Updating submodules
Summary:
GitHub commits:

5fa6ff45d5

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: f1fce765904026bf07861a0ec744734fc23432cb
2020-03-12 06:40:41 -07:00
Thomas Orozco
472362fd6c treemanifest: comma-terminate directories list in gettreepack
Summary:
We cannot differentiate an empty list from a list containing the empty path only. This fixes that.

See D20309700 earlier in this stack for the server-side bits.

Reviewed By: quark-zju

Differential Revision: D20309699

fbshipit-source-id: 02efc747c3c5e2774f949ba0a6370169b55f3cf3
2020-03-12 06:35:34 -07:00
Thomas Orozco
06d11938cf mononoke/hgproto: expect comma-terminated list of directories
Summary:
We need to differentiate the empty directory from no directories. Adding a
trailing comma after each directory instead of separating them achieves that.

Reviewed By: StanislavGlebik

Differential Revision: D20309700

fbshipit-source-id: 387ec477560968392de0a9631d67ccb591bd3cab
2020-03-12 06:35:34 -07:00
Thomas Orozco
87491d14a7 mononoke/repo_client: add designated manifest fetching in gettreepack
Summary:
This will allow the hg client to do tree fetching like we do in the API Server,
but through the SSH protocol — i.e. by passing a series a manifest ids and
their paths, without recursion on the server side through gettreepack.

Reviewed By: StanislavGlebik

Differential Revision: D20307442

fbshipit-source-id: a6dca03622becdebf41b264381fdd5837a7d4292
2020-03-12 06:35:34 -07:00
Stanislau Hlebik
ba871d3bdc xdiff: allow rendering diff for large files
Summary:
The goal of the stack is to support "rendering" diffs for large files in scs
server. Note that rendering is in quotes - we are fine with just showing a
placeholder like "Binary file ... differs". This is still better than the
current behaviour which just return an error.

In order to do that I suggest to tweak xdiff library to accept FileContentType
which can be either Normal(...) meaning that we have file content available, or
Omitted, which usually means the file is large and we don't even want to fetch it, and we
just want xdiff to generate a placeholder.

Reviewed By: markbt, krallin

Differential Revision: D20389226

fbshipit-source-id: 0b776d4f143e2ac657d664aa9911f6de8ccfea37
2020-03-12 04:27:23 -07:00
Kostia Balytskyi
30e39de418 edenscm: add a separate path type for infinitepush writes
Summary:
In order to support gradual rollout of infinitepush for other backends (e.g. Mononoke), we need the ability to route read vs write requests separately. To achieve this, we need a separate path type: `infinitepush-write` (kind of like `default-push`, the naming inconsistency is a little unfortunate, as I don't want to use `infinitepush-push`).

The desired behavior of the new path type is as follows:
- takes precedence over `infinitepush` path when the user does `hg push -r . --to scratch/bla`
- replaces `infinitepush` path when the user does `hg push infinitepush -r . --to scratch/bla`
- absence of this path means draft pushes will go to `infinitepush` path
- draft pulls always go to `infinitepush` path, and *there's no fallback to `infinitepush-write`*
- commit cloud always talks to `infinitepush-write`, if it is present (meaning that commit cloud pulls do go to `infinitepush-write` path
  - this is done, as commitcloud uses infinitepush paths to also check whether something is backed up
  - and also, commitcloud may need to pull very soon after something has been pushed

Reviewed By: quark-zju

Differential Revision: D20368158

fbshipit-source-id: 59db174cebbf2b48765dff37bc93aad176c2d7c1
2020-03-12 03:22:12 -07:00
svcscm
7fcd49242c Updating submodules
Summary:
GitHub commits:

31c8155517
47651305b1

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: e42381f1706e8c4ba665fc4f509465ab939c8faa
2020-03-12 01:33:25 -07:00
Yedidya Feldblum
a55282e5fd Require callers to pass NetworkSocket to TAsyncSocket
Summary: [Thrift] Require callers to pass `NetworkSocket` to `TAsyncSocket` instead of relying on `TAsyncSocket` doing the conversion.

Reviewed By: vitaut

Differential Revision: D19800150

fbshipit-source-id: f2b8e76fb16e187810844d8d98654e975cf8dd13
2020-03-11 23:45:16 -07:00
Ratnadeep Joshi
8e4b04107c Add an option to mount a checkout read-only
Summary: Added option to mount command

Reviewed By: chadaustin

Differential Revision: D20256352

fbshipit-source-id: e52fbd10f318221777b7e2d21ed9ba0a17ce1687
2020-03-11 21:38:48 -07:00
svcscm
1beb826aae Updating submodules
Summary:
GitHub commits:

069e1dfd48
7ec76e3870
723f2001e8
51fef66e70
2aaaa8b743
858e8b11ba
e56b53951d
a2c8a21b8f

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: d4bf034a0244c2c8d26e1b947a72c6ad387db15e
2020-03-11 21:21:12 -07:00
Jun Wu
5fd51a48d2 changelog: extract some logic to methods
Summary: Those functions are reused in in a future diff.

Reviewed By: sfilipco

Differential Revision: D20367838

fbshipit-source-id: 944babf8c02f0560f8ac8ca5d7c4263432032715
2020-03-11 20:37:30 -07:00
Jun Wu
ffacd3a857 pydag: add new APIs
Summary: Expose the newly added APIs to Python.

Reviewed By: sfilipco

Differential Revision: D20367839

fbshipit-source-id: 856365513e0ff2010a8776c64ce6198d02953dc8
2020-03-11 20:37:30 -07:00
Jun Wu
194b38385a nameset: add a way to convert between NameSet and SpanSet
Summary:
This will be used in the Python world for legacy reasons. It shouldn't be used
in new Rust node.

To use it, the name `LegacyCodeNeedIdAccess` has to be used so we can do a code
search to find all users of it.

Reviewed By: sfilipco

Differential Revision: D20367834

fbshipit-source-id: 9b93a29f1461ce24bba6f31a2bbb1f327e216c6d
2020-03-11 20:37:30 -07:00
Jun Wu
eef56d9c5b namedag: add a sort API
Summary: This will be useful to actually sort commits.

Reviewed By: sfilipco

Differential Revision: D20367835

fbshipit-source-id: 43bc7835277af3a14ef323ce34247e0c03878dc8
2020-03-11 20:37:29 -07:00
Jun Wu
2ecc0bb757 namedag: move "all" concept to DagSet
Summary:
The old "AllSet" implementation is not very practical - it does not support
iteration. Practically, the "all()" set comes from the DAG. Change the "all"
concept to a hint similar to "is_topo_sorted", and update the fast path
(intersection) accordingly.

Reviewed By: sfilipco

Differential Revision: D20367837

fbshipit-source-id: fdbf370897c93058bfcab0571c1f6fa4b99b0f6b
2020-03-11 20:37:29 -07:00
Jun Wu
ef1696b4db namedag: rename arc_map to snapshot_map
Summary: The word "snapshot" more accurately describes its purpose.

Reviewed By: sfilipco

Differential Revision: D20367836

fbshipit-source-id: c91a0bd402fa1718b5d805beedc0e062824c53d3
2020-03-11 20:37:29 -07:00
Wez Furlong
36d2d45ea7 eden: fix LFS cache size reported by eden du
Summary: Looks like an honest typo/copypasta

Reviewed By: xavierd

Differential Revision: D20401180

fbshipit-source-id: 544d96afc5b3e6365430641f73d1d2acc115b8d5
2020-03-11 20:06:31 -07:00
svcscm
a33fb3f0fe Updating submodules
Summary:
GitHub commits:

79a81c9baa
8c1bf36804
67a4344606
fbb0c170af
2eb0be9f68
2d9efc9ab2
a4fad3a468
0d6464b601
a5f69715d1
a7f1cb2e11

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: ecfac74344c501b83bb5cc51df87fc9183e7a369
2020-03-11 19:58:48 -07:00
Yixian Jiang
a72245ef36 clean sigar build dependency
Summary:
Clean the sigar build dependency completely under:
- fbcode/fbzmq/public_tld/build/...
- fbcode/openr/public_tld/build/...

Delete files:
- fbcode/opensource/fbcode_builder/manifests/sigar
- fbcode/opensource/fbcode_builder/specs/sigar.py

Reviewed By: steven1327

Differential Revision: D20376067

fbshipit-source-id: 3321a14df2551525acee605028ef06a04cda7c6a
2020-03-11 19:13:20 -07:00
Adam Simpkins
03873d9870 suppress exceptions that occur in sparse telemetry tracking
Summary:
The `sparse` extension's dirstate tracking code attempts to read dirstate data
after the command has executed.  For commands that didn't require accessing
the dirstate, this can end up reading it for the first time, and can this fail
in some circumstances.  Ignore all errors that occur when trying to compute
this telemetry, so they don't break the main command functionality.

In particular, we see reproducible scenarios where `hg` crashes due to this
error when invoked by watchman to compute the list of files changed between
two commits.  Watchman currently always invokes Mercurial with `HG_PENDING`
enabled, and this can cause dirstate loading to fail when there is pending
data.

Reviewed By: quark-zju

Differential Revision: D20402221

fbshipit-source-id: 782d9b6eff26d50ef20f080c0cbcbc852e425146
2020-03-11 18:34:09 -07:00
svcscm
d8a9b8067c Updating submodules
Summary:
GitHub commits:

23db137688
f4c6d8e6f6
331e6199df
372f2f41d0

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: e44bad3c3388883c0992e633d4458cb6ebeadc23
2020-03-11 17:47:05 -07:00
Jun Wu
a7dedfdfd6 fsinfo: remove old getfstype implementation
Summary: Remove the old Python, C implementation of getfstype.

Reviewed By: xavierd

Differential Revision: D20313385

fbshipit-source-id: 475c73343aae2fa2f5ad898c7b0879bfa2c80e93
2020-03-11 17:35:41 -07:00
Jun Wu
c5c75c9f59 fsinfo: autocorrect "" to "."
Summary:
Without this:

  In [3]: util.getfstype('')
  IOError: [Errno 2] No such file or directory (os error 2)

And there is a code path hitting this:

  File "edenscm/mercurial/util.py", line 1483, in checknlink
    fstype = getfstype(os.path.dirname(testfile))
		# testfile = '.'
	  # os.path.dirname(".") = ""

The old implementation works fine for an empty path:

	In [2]: m.util.getfstype('')
  Out[2]: 'eden'

So let's make the new Rust implementation consistent.

Reviewed By: xavierd

Differential Revision: D20313387

fbshipit-source-id: 258c424a3e8a796d983e20b0d4656e8e3f413706
2020-03-11 17:35:40 -07:00
Jun Wu
ea59a506f1 fsinfo: expose fstype via bindings.fs.fstype
Summary: Expose the Rust fstype implementation so we can remove C and Python code.

Reviewed By: xavierd

Differential Revision: D20313390

fbshipit-source-id: 931b4f6bc8c4b81f30bfea06d055fae3c677eedd
2020-03-11 17:35:40 -07:00
Jun Wu
61bebcaacc fsinfo: try harder to get fuse fs type
Summary: Similar to D13982877. Try to get names like "fuse.ntfs".

Reviewed By: farnz

Differential Revision: D20313392

fbshipit-source-id: 8363d3d92843e6afb53a0003950be083034bd841
2020-03-11 17:35:39 -07:00
Jun Wu
13374f9d74 fsinfo: drop most type parameters
Summary:
Only keep type parameters at the top-level function.
This reduces the binary size and speeds up rustc.

Reviewed By: xavierd

Differential Revision: D20313388

fbshipit-source-id: 29d77731ff462fee1f1bb9f234601e3430198ae7
2020-03-11 17:35:39 -07:00
Jun Wu
c83006002c fsinfo: return unknown on unsupported platforms
Summary: This makes the code a bit more portable.

Reviewed By: xavierd

Differential Revision: D20313389

fbshipit-source-id: 080538939fa4d2d72e5905f23ad9be987d952748
2020-03-11 17:35:38 -07:00
Jun Wu
9cdc818915 fsinfo: drop "repo" from method names
Summary:
Rename the main method to "fstype". The API has no relation with repo.
So let's rename it.

Reviewed By: xavierd

Differential Revision: D20313386

fbshipit-source-id: 80dd1231ccccfe945150b117b151bce773f0dfeb
2020-03-11 17:35:38 -07:00
Jun Wu
951c8ab082 fsinfo: backport from telemetry
Summary: The fsinfo crate provides the "filesystem type" information.

Reviewed By: xavierd

Differential Revision: D20313391

fbshipit-source-id: f717f5edb32957d59d03090117cfdb8123f03933
2020-03-11 17:35:37 -07:00
Xavier Deguillard
dafc8a15f6 pyworker: reduce flakiness of the quickcheck tests
Summary:
Thes test proved to be a bit flaky due to some invalid input from quickeck,
let's filter these cases and ask quickcheck for different input.

Reviewed By: quark-zju

Differential Revision: D20397604

fbshipit-source-id: cc97bef0e286009f1394c7aa2c0429d1e1b1c8c2
2020-03-11 17:17:21 -07:00
Adam Simpkins
55a2324219 skip automigration if the repository has an abandoned transaction
Summary:
A common cause for automigration failure is if the repository has an abandoned
transaction.  Just ignore the error and skip migration if this happens.
Otherwise important read-only commands (like `hg debugedenimporthelper`)
refuse to even start in this situation.

Reviewed By: farnz

Differential Revision: D20394546

fbshipit-source-id: abb75fe455e9ee815032431705df7f38ee50283a
2020-03-11 16:25:50 -07:00
Adam Simpkins
8e78254e55 fix a ProgrammingError bug in visibility migration
Summary:
The code to disable visibility was writing to files without a transaction,
which causes a `ProgrammingError` to be thrown when using metalog.

Reviewed By: farnz

Differential Revision: D20394547

fbshipit-source-id: c529de84ed7b127df18e7f60572cece2169dc520
2020-03-11 16:25:50 -07:00
Adam Simpkins
983a26875d update debugedenimporthelper's debug utilities to use ui.write()
Summary:
Print output using `repo.ui.write()` rather than directly using `print()` or
`sys.stdout`.  This is needed to make the output visible during the
Mercurial `.t` tests.

Reviewed By: farnz

Differential Revision: D20394549

fbshipit-source-id: 7ee50fe26e98c1c34b74ce054d2187050c42b2f3
2020-03-11 16:25:50 -07:00
Adam Simpkins
1c6360fe73 add more annotations for edenscmnative.parsers
Summary:
Add type annotations for the `edenscmnative.parsers` methods that handle
serializing and deserializing the dirstate data.

Reviewed By: quark-zju

Differential Revision: D19958218

fbshipit-source-id: 6e20efbc1b0a6ba15b297e47a1e6eec8ed47ee52
2020-03-11 16:21:30 -07:00