Commit Graph

63825 Commits

Author SHA1 Message Date
Jun Wu
f10466c9f1 dirsync: do not wrap remote repo
Summary:
Typical mistake of `reposetup` - the `repo` object might be remote and lack of
local repo methods.

Reviewed By: DurhamG

Differential Revision: D26822960

fbshipit-source-id: 54fb95c51b09ef5021b0a10a93b4509e7aa4a115
2021-03-04 11:38:28 -08:00
generatedunixname89002005307016
0124627d4b suppress errors in fbcode/eden - batch 1
Differential Revision: D26753306

fbshipit-source-id: 335f45b534c6544eaf06e9d42fb16d758f8b0a5e
2021-03-04 11:03:06 -08:00
svcscm
eeed69e109 Updating submodules
Summary:
GitHub commits:

c95d4bc2c6
7821f3a216
30bdbd64ae
030a8b9be7

Reviewed By: wittgenst

fbshipit-source-id: 91c86895fe508a9384ed38a1f6322524af313406
2021-03-04 10:46:28 -08:00
Thomas Orozco
aae28ddb6c mononoke: add perf counters + better errors to filenodes
Summary:
We're missing some metrics to measure how much queries we are making here, and
our error reporting isn't very comprehensive. Let's improve this.

Reviewed By: StanislavGlebik

Differential Revision: D26819845

fbshipit-source-id: c78b8dee02d58dc09ea5f5bd9712e5dfa0bb3f29
2021-03-04 10:32:02 -08:00
Mark Juggurnauth-Thomas
7eb0817dbd test-scs-modify-bookmarks: wait for warm bookmark cache to update
Summary:
The recent change to make `list-bookmarks` read from the warm bookmark cache
has made this test flaky, as we must wait for the warm bookmark cache to update
before the new values are visible.

Update `wait_for_bookmark_update` to use `list-bookmarks` which always uses
the warm bookmark cache.  A similar function is also needed for bookmark deletion.

Reviewed By: StanislavGlebik

Differential Revision: D26816627

fbshipit-source-id: e99b83454070d565646b069177e18bd6a01946ae
2021-03-04 10:19:51 -08:00
Xavier Deguillard
9a7c51ef9c nfs: add RPC types for REMOVE procedure
Summary: This is merely adding the types needed to implement the REMOVE procedure.

Reviewed By: kmancini

Differential Revision: D26694441

fbshipit-source-id: 736d06ed7bfae40f827fbec6017e069fe11d7464
2021-03-04 09:59:48 -08:00
Xavier Deguillard
6daea7e4e0 nfs: implement the LINK RPC
Summary:
Since EdenFS doesn't support hardlinks, this is merely collecting the attribute
of the file before always failing.

Reviewed By: kmancini

Differential Revision: D26682014

fbshipit-source-id: 8c39170147023a03e87ed7a0fae66f9184b4f690
2021-03-04 09:59:47 -08:00
Xavier Deguillard
466b3144b8 nfs: add RPC types for LINK
Summary: During clone, Mercurial calls this, despite EdenFS not advertising hardlinks.

Reviewed By: kmancini

Differential Revision: D26682013

fbshipit-source-id: b43affeb7577354991308165113d5d5070bf8605
2021-03-04 09:59:47 -08:00
Xavier Deguillard
b5ad70b619 nfs: implement the FSSTAT RPC
Summary:
The RPC simply queries various filesystem attributes, we merely forward what
statfs on the overlayfs gives us.

Reviewed By: kmancini

Differential Revision: D26681613

fbshipit-source-id: 5b94d05cafff8d77390fe60a3b5cf1dc3e022f42
2021-03-04 09:59:47 -08:00
Xavier Deguillard
f7050ae2ea nfs: add RPC types for FSSTAT
Summary: This merely adds the types for the RPC.

Reviewed By: kmancini

Differential Revision: D26681615

fbshipit-source-id: d092cf0b6b5bb7435702d125b5c6ea7ee68356dc
2021-03-04 09:59:47 -08:00
Xavier Deguillard
e87ea9c085 nfs: implement the WRITE RPC
Summary:
This simply writes the data passed in to the inode. Note that the current
implementation has a protocol violation since it doesn't sync the data written
to disk but advertise to the client that it did. This is completely wrong from
a data consistency guarantee, but is probably fine for now. Once the code
becomes closer to being production ready, this will be changed to honor the
client asks.

Reviewed By: kmancini

Differential Revision: D26681614

fbshipit-source-id: 82ad7a141be3bbe365363b1f6692ae62f253423f
2021-03-04 09:59:46 -08:00
Xavier Deguillard
3d377e2df3 nfs: replace folly::io::Appender with folly::io::QueueAppender
Summary:
The Appender API doesn't allow us to simply append an IOBuf chain to it forcing
the entire chain to be copied to it. For the most part, this isn't an issue,
but to reduce the overhead of the READ NFS procedure, this will be a nice
benefit.

For the most part, this diff is a codemod, with a couple of places that needed
to be manually fixed. These were in the rpc/Server.{cpp,h} and
rpc/StreamClient.{cpp,h}.

Reviewed By: genevievehelsel

Differential Revision: D26675988

fbshipit-source-id: 04feef8623fcddd02ff7aea0b68a17598ab1d0f8
2021-03-04 09:59:46 -08:00
Xavier Deguillard
8125e2e115 nfs: de-duplicate some common test functions
Summary:
The ser, de and roundtrip are duplicated 3 times in the various tests, let's
move them into a common library.

Reviewed By: chadaustin

Differential Revision: D26675989

fbshipit-source-id: 1da0bc33429795a889b72c76fa18e5fa3cb7df6f
2021-03-04 09:59:46 -08:00
Xavier Deguillard
3bea128aeb nfs: add types for WRITE RPC
Summary: This merely adds the various types for the WRITE RPC.

Reviewed By: chadaustin

Differential Revision: D26671895

fbshipit-source-id: 8409c8a1f90e97478aed7c9f19b881c46234b539
2021-03-04 09:59:46 -08:00
Xavier Deguillard
0841d553fd nfs: allow serializing/deserializing IOBuf
Summary:
For the READ/WRITE RPC calls, copying data in and out of an IOBuf chain can be
fairly expensive, to avoid this overhead, we can simply clone the data out of
the IOBuf chain directly, saving on the cost of copy.

Since the code uses a folly::io::Appender that doesn't support adding an IOBuf
to it, we still pay the cost of copying data to it, switching to
folly::io::QueueAppender may solve this.

Reviewed By: chadaustin

Differential Revision: D26671896

fbshipit-source-id: 0161f04cb820bf27ef66fdef6b4a1ce4eb778b96
2021-03-04 09:59:45 -08:00
Xavier Deguillard
b46a7b2a11 nfs: implement the CREATE RPC
Summary:
The CREATE RPC is intended to create regular files on the server. It supports 3
ways of creating them: unchecked, guarded and exclusive. The exclusive mode
allows for a client to being able to re-transmit the creation and for it to not
fail when that happens. The way this is implemented requires the server to
store a cookie passed in by the client and on retransmission to compare it in
order to not fail the file creation. This complexity may not be warranted and
needed in the case of EdenFS, and the spec allows for a server to not support
this operation, thus EdenFS claims to not support it.

The 2 other modes are more traditional, guarded will simply fail if the file
already exist, while unchecked succeed in that same situation. It's not
entirely clear in the RFC what the behavior is supposed to be in the unchecked
case, and whether the attributes needs to be applied. For now, the error is
just ignored, a warning message is logged and a success is returned in this
case. In the future, setting the file attribute (in particular the mode and
size) may be necessary.

Reviewed By: kmancini

Differential Revision: D26654766

fbshipit-source-id: c2e8142b8a4ff756a868e5859fdda4b07e53bddf
2021-03-04 09:59:45 -08:00
svcscm
2901ea4c1d Updating submodules
Summary:
GitHub commits:

8177630b0e
618db6a4a9
82ae49dae2
89d3179ab8
a1ee37deeb
8edbf6cd82

Reviewed By: wittgenst

fbshipit-source-id: 4591f44cc7de20057f0974d3054c4650fc5fff0c
2021-03-04 09:37:16 -08:00
Stefan Filip
07750d0a3a notes: add 2019-10 Manifests, Past and Future
Summary:
I moved the source of the sphinx project to newdoc for newdoc/dev.
I updated the sphinx config for markdown to something that works for recent
versions.
PNG images rendered better than SVG for me.
I moved the TARGETS to newdoc.

Reviewed By: quark-zju

Differential Revision: D26801426

fbshipit-source-id: 3ae51e886d27f848f0f6a48c96056da607d8da45
2021-03-04 08:52:27 -08:00
Stefan Filip
1162e2ba0e gitignore: track Makefile under scm
Summary:
Mercurial uses Makefile here to build stuff. We want to track Makefile under
scm.

Reviewed By: quark-zju

Differential Revision: D26802165

fbshipit-source-id: 1fe8db13d50c07a6a0681180959eba22eaf8d486
2021-03-04 08:52:27 -08:00
svcscm
755edaa73a Updating submodules
Summary:
GitHub commits:

81f7ac23d6
343adaa2d1
2af74e369e

Reviewed By: wittgenst

fbshipit-source-id: 0e919f433c39ed3352aa56ed3f5d54c8d7e55ab9
2021-03-04 08:52:26 -08:00
svcscm
1a207249fd Updating submodules
Summary:
GitHub commits:

e116482e96
92ea84dcac
a6a4852fc7
495420551c
c7fd45cb15

Reviewed By: wittgenst

fbshipit-source-id: fe46f7bf111de54c81142b1305c732c63eb5e546
2021-03-04 07:37:21 -08:00
Thomas Orozco
598dbd76b6 third-party: import a Tokio 1.x compatible version of Gotham
Summary:
This won't be very long lived, as I'll remove the 0.2 version once Mononoke is
updated (we are the only users), and stop using a fork as soon as Gotham
releases. However, for now, I need both versions in order to avoid to do it all
in 1 diff.

A couple things worth noting here:

- See the previous diff for why I am using 2 Git repos for this.
- I can't easily use 2 forks for this because there are some extra crates they
  vendor, notably `borrow_bag`. I removed it from both forks and pulled it from
  crates.io instead.

Reviewed By: mitrandir77

Differential Revision: D26781291

fbshipit-source-id: 0b9824c07b880dbd5b94cd6d62d2cb2a68e348e9
2021-03-04 06:42:56 -08:00
Thomas Orozco
5871cd0a90 third-party/rust: patch Hyper 0.14 sending Content-Length on 101 Continue responses
Summary:
There is a bug in Hyper with 101 Continue responses: it sends a Content-Length.
This makes Proxygen unhappy with websocket upgrades. We used to have this
patched in hyper-02, but since Mononoke is about to update to Tokio 1.x, we
also need in the matching Hyper.

One thing that's a bit awkward is you might notice I changed the fork where the
patch comes from. This is because `cargo vendor` cannot deal with 2 overriding
sources coming from the same upstream. I posted about my adventures in this
here: https://fb.workplace.com/groups/rust.language/permalink/5278514098863829/

Reviewed By: HarveyHunt

Differential Revision: D26780899

fbshipit-source-id: e775b7151427898d63d8767acaa53f5f68229db6
2021-03-04 06:42:56 -08:00
Thomas Orozco
2a803fc10d third-party/rust: update futures
Summary:
Those newer versions of Futures have compatibility improvements with Tokio,
notably:

- https://github.com/rust-lang/futures-rs/pull/2333
- https://github.com/rust-lang/futures-rs/pull/2358

Reviewed By: farnz

Differential Revision: D26778794

fbshipit-source-id: 5a9dc002083e5edfa5c614d8d2242e586a93fcf6
2021-03-04 06:42:55 -08:00
Thomas Orozco
734ea4d855 mononoke/git: make git-pool use tokio_shim
Summary: This is used in Metagit and I'd like to decouple those 2 Tokio 1.x migrations.

Reviewed By: HarveyHunt

Differential Revision: D26813352

fbshipit-source-id: 7bc34e1cad00c83bf66edce559b07104d44a7357
2021-03-04 06:33:10 -08:00
svcscm
ce016a1749 Updating submodules
Summary:
GitHub commits:

980693e7e5

Reviewed By: wittgenst

fbshipit-source-id: cdfade5683c041a42c077870650c7d1431dcd04e
2021-03-04 02:19:49 -08:00
Alex Hornby
8d74e200e8 rust: async shed/sql Transaction apis
Summary: Now the queries macros are asynced, lets do the same with the Transaction api exposed from them.

Reviewed By: krallin

Differential Revision: D26730195

fbshipit-source-id: 278753a5d0401f602ce50519138164bb5e49d550
2021-03-04 01:52:42 -08:00
Alex Hornby
090cdc9389 mononoke: remove sql01::queries from sync_commit_mapping
Summary: Migrate to the std futures version of sql::queries!

Reviewed By: krallin

Differential Revision: D26700359

fbshipit-source-id: 39c75d7896a5975e53dd3af53860ce486683b4ed
2021-03-04 01:52:42 -08:00
Alex Hornby
5ef48e3b4b mononoke: remove sql01::queries from dbbookmarks
Summary: Migrate to the std futures version of sql::queries!

Reviewed By: krallin

Differential Revision: D26700357

fbshipit-source-id: ea9382028b2e5abfa1946e1c5de344e32ac60d04
2021-03-04 01:52:41 -08:00
Alex Hornby
98802f349d mononoke: remove sql01::queries from segmented_changelog
Summary: Migrate to the std futures version of sql::queries!

Reviewed By: krallin

Differential Revision: D26700360

fbshipit-source-id: 9ed2664d522bde8d0e923142357ca876a7de2613
2021-03-04 01:52:41 -08:00
Alex Hornby
a189cf9905 mononoke: remove sql01::queries from mutable counters
Summary: Migrate to the std futures version of sql::queries!

Reviewed By: krallin

Differential Revision: D26700358

fbshipit-source-id: 4a100705c43d77d67fb784afbb6b44b57904cba0
2021-03-04 01:52:41 -08:00
Alex Hornby
2ff9ad0fea rust: async sql queries macros
Summary:
Async the query macros.  This change also migrates most callsites, with a few more complicated ones handle as separate diffs, which temporarily use sql01::queries in this diff.

With this change the query string is computed lazily (async fn/blocks being lazy) so we're not holding the extra memory of query string as well as query params for quite as long.  This is of most interest for queries doing writes where the query string can be large when large values passed (e.g. Mononoke sqlblob blobstore )

Reviewed By: krallin

Differential Revision: D26586715

fbshipit-source-id: e299932457682b0678734f44bb4bfb0b966edeec
2021-03-04 01:52:41 -08:00
svcscm
cfd978798c Updating submodules
Summary:
GitHub commits:

240b7fe850
cb25bc1128

Reviewed By: wittgenst

fbshipit-source-id: b9d8c5c7ceff708bd2b59d72e5ce1e470fc25131
2021-03-04 01:52:40 -08:00
svcscm
58ccacab51 Updating submodules
Summary:
GitHub commits:

63c11c4e50

Reviewed By: wittgenst

fbshipit-source-id: 2f20ea77802f75eea4d995b19a0f3de747f95130
2021-03-03 22:39:16 -08:00
Jun Wu
e70d89dd39 py3: fix absorb -i
Summary:
The previous fix D23357655 (d60e80796a) actually only fixes py2 absorb -i. On Python 3,
`b"-"[0]` is `45`, not `b"-"` like Python 2. Fix it again using `b"-"[0:1]`.

Reviewed By: singhsrb

Differential Revision: D26805315

fbshipit-source-id: 07ca850373a6bc49b561466ead478024631ce051
2021-03-03 20:50:31 -08:00
svcscm
2d28a8708a Updating submodules
Summary:
GitHub commits:

e2a49e7507
7dd554796a

Reviewed By: wittgenst

fbshipit-source-id: b82fd1eaaf5d7f90dc25ceba11d029f2e977e9c0
2021-03-03 20:44:16 -08:00
Xavier Deguillard
d9207cbe4e backingstore: do not initialize memcache for debug builds
Summary:
Memcache is dogslow to initialize, taking >30s on debug build. As a
consequence, this slows down every single test by that amount time, with the
guarantee that no blobs will be found in memcache, ie: a total waste of time.

On release builds, Memcache is significantly faster to initialize, so let's
only disable initializing Memcache for debug builds only.

Reviewed By: fanzeyi

Differential Revision: D26800265

fbshipit-source-id: 8b393c603414de68268fdadb385de177e214a328
2021-03-03 20:14:46 -08:00
Durham Goode
f22e274e94 py3: add py3 windows package to hgbuild
Summary:
Adds a new hg py3 windows package to hgbuild for publishing. Currently
the tests don't run. I'll do that next.

Reviewed By: quark-zju

Differential Revision: D26768336

fbshipit-source-id: bd4533abbbc1e1c861aa9995b7a3868a7f6a1a22
2021-03-03 19:52:54 -08:00
svcscm
bbd40309ec Updating submodules
Summary:
GitHub commits:

626e5cafa6
d9b650538e
e11e7bf1aa

Reviewed By: wittgenst

fbshipit-source-id: f84ffd80ac2762c043736a76acf641c43c1acbfe
2021-03-03 19:52:54 -08:00
svcscm
24445cf5ae Updating submodules
Summary:
GitHub commits:

5916a3a072
1259373958
a46be0cadb
62f48d83d3

Reviewed By: wittgenst

fbshipit-source-id: afb083438483e60712ddd2835766aa326b27d566
2021-03-03 18:42:52 -08:00
Jun Wu
4afd737525 hide: cleanup non-essential remote bookmarks automatically
Summary: Trigger the cleanup logic automatically if there are too many remote bookmarks.

Reviewed By: sfilipco

Differential Revision: D26802251

fbshipit-source-id: 1ab40c7be98c507865ea17001dc9775b6edf4446
2021-03-03 18:30:08 -08:00
Jun Wu
01035df3a4 hide: make --cleanup remove non-essential remote bookmarks
Summary: This is handy to make the `sl` output cleaner.

Reviewed By: sfilipco

Differential Revision: D26802250

fbshipit-source-id: 1b74f3d9a42ab6b0780f07bec59b03a5cd0ea6a9
2021-03-03 18:30:08 -08:00
Jun Wu
e33255b67d remotenames: add a command to clean up remote bookmarks
Summary: Add API and command to remove non-essential remote bookmarks

Reviewed By: sfilipco

Differential Revision: D26802252

fbshipit-source-id: bf715905f146f31aac19dccb90022bea31392323
2021-03-03 18:30:07 -08:00
Jun Wu
4abfeb66e8 remotenames: raise if key bookmarks point to unknown commits
Summary:
Previously remotenames pointing to unknown commits are just ignored.
If key remotename like master is ignored, it can cause very slow operations
in pull, etc. Let's just raise an exception in this case.

Reviewed By: DurhamG

Differential Revision: D26800236

fbshipit-source-id: 13be4af5499da1b9098b4ff1a6ef41c54092824a
2021-03-03 18:20:05 -08:00
Jun Wu
7a759b6075 visibility: remove public heads
Summary:
Remove public heads when using Rust changelog backends. This should address
some issues seen in commit cloud sync.

This is done at the metalog commit time so we get the latest "remotenames" data for
accurate "public()" set calculation.

Reviewed By: singhsrb

Differential Revision: D26792731

fbshipit-source-id: 00b894fee9804740d664dad0ac47be564820da33
2021-03-03 18:06:08 -08:00
svcscm
c7cede16b1 Updating submodules
Summary:
GitHub commits:

64d8f3fdcd

Reviewed By: wittgenst

fbshipit-source-id: 390b87e232ac9012054d165effc290c6c81d7b6f
2021-03-03 18:06:07 -08:00
svcscm
0c65db2639 Updating submodules
Summary:
GitHub commits:

c0fd94f1eb
d8bd4d9ddd

Reviewed By: wittgenst

fbshipit-source-id: 510cce2995938023d9d6e2cd1ffc10a6e05c1e04
2021-03-03 17:20:34 -08:00
Jun Wu
8142c5e5f1 windows: set output encoding to UTF-8
Summary:
The output encoding is used to render the graph log edges. With D26612487 (62ba7447f6), we
switched to Rust IO. The Rust IO requires UTF-8 data. So let's set
outputencoding to UTF-8.

Reviewed By: sfilipco

Differential Revision: D26799551

fbshipit-source-id: aa3e6420067d7c75bef47448e12e48f4cef56a84
2021-03-03 16:45:16 -08:00
svcscm
f18783d43d Updating submodules
Summary:
GitHub commits:

1beb55a5f3
8a6dbbf582
a85231436d
c2792dd135
114fd03755
382c1cdcc6

Reviewed By: wittgenst

fbshipit-source-id: cf49166c149dcbaf3f75b64abd824b34bed0c9b2
2021-03-03 16:45:16 -08:00
svcscm
36e0669579 Updating submodules
Summary:
GitHub commits:

12660505e0
bc57378516
4126bdc0e1
12f097bf23

Reviewed By: wittgenst

fbshipit-source-id: e2062ed369a3b3ef961a627d67df640adf035efa
2021-03-03 15:10:08 -08:00