Commit Graph

61611 Commits

Author SHA1 Message Date
Simon Farnsworth
00871310a7 Ensure we have only one ConfigStore for the application
Summary: It's designed as a singleton store for normal use - rather than have lots of ways to get different config stores, let's use one global store

Reviewed By: krallin

Differential Revision: D24447404

fbshipit-source-id: 6ecc46351b14794471f654cec98527a11a93d3ef
2020-10-24 06:23:49 -07:00
svcscm
9d4f8f31d1 Updating submodules
Summary:
GitHub commits:

af8630489d

Reviewed By: yns88

fbshipit-source-id: 5850f916bb81621b1a1fd2111bc46b4953b4b9e5
2020-10-24 06:23:49 -07:00
svcscm
c1d9cd7049 Updating submodules
Summary:
GitHub commits:

aa5e4bf208
b59acfa385
5ff4569549
05786142e6
7e3088ccb0
0fa6f2f529
43846731fd
a2f7326aa4
779ed09eb1
068f8c943d
9f44bf4178
afa0bd5de6

Reviewed By: yns88

fbshipit-source-id: e599ca091ac235891efcfcbf7b351772ebd52671
2020-10-24 02:33:02 -07:00
Dave Rigby
0e98381071 FindGlog: Add support for 'glogd' Debug library (#1479)
Summary:
Glog v0.4.0 when configured with Debug build type adds a 'd' suffix to
the library file. This results in FindGlog.cmake failing to locate it.

Update FindGlog.cmake to use check for 'glogd', and use
select_library_configurations() to set GLOG_LIBRARY to the correct
found filename.

(Note: this has no effect if a non-Debug type is used.)

Pull Request resolved: https://github.com/facebook/folly/pull/1479

Reviewed By: yfeldblum

Differential Revision: D24503510

Pulled By: Orvid

fbshipit-source-id: 705df05a4a3d7df2df8af3bb66c319fb044adbce
2020-10-24 01:18:21 -07:00
Chad Austin
496c69e182 allow masking trace events
Summary:
Sometimes you only want to trace writes into the EdenFS checkout, so
add the ability to run `eden strace` with `--reads`, `--writes`, or
both in order to see only those events.

Reviewed By: wez

Differential Revision: D24468539

fbshipit-source-id: a1b3c730987cf86ce3d39952c6a5e9c5edaddfc2
2020-10-23 19:53:00 -07:00
svcscm
9b2fd0e94d Updating submodules
Summary:
GitHub commits:

348007db56
1e77d94bb3
7f894bbec9
07c7c1dd91
9e00bb0de8

Reviewed By: yns88

fbshipit-source-id: e67dc3dcea2a1445ce1e46bdf463f79fc0922d41
2020-10-23 19:53:00 -07:00
svcscm
3617c5a23a Updating submodules
Summary:
GitHub commits:

81db842b42
fdd751a1ff
6d1a4c8d94
505179b125
df1b6cd2ca
de8ec06a4b
c10b2349d3

Reviewed By: yns88

fbshipit-source-id: 5aee84cb8c60af18b5b757955173603c6c2a2100
2020-10-23 13:21:37 -07:00
Mateusz Kwapich
1c153d47bb make replaybookmarks py3-compatible
Reviewed By: DurhamG

Differential Revision: D24508222

fbshipit-source-id: 091ff81cae49ff7e4882edd59ace6f024e26c426
2020-10-23 12:37:01 -07:00
Chad Austin
81d18bde42 allow querying Overlay file size outside of FileInode
Summary: Allow calling into OverlayFileAccess::getFileSize from outside of FileInode.

Reviewed By: xavierd

Differential Revision: D24300035

fbshipit-source-id: 2a5fc730122a79220fd38c928b74342433c4e9f2
2020-10-23 11:16:10 -07:00
Chad Austin
cc8c4c4ceb refactor debugInodeStatus
Summary:
Before replacing the existing debugInodeStatus changes, make some
small refactorings to help me understand it.

Reviewed By: xavierd

Differential Revision: D24299970

fbshipit-source-id: 3b60aa8365bb9a45e12357374282ba837ccc36eb
2020-10-23 11:16:10 -07:00
Chad Austin
ba39580899 add observed inode traversal functions
Summary:
Add a function that makes it easy to traverse EdenFS's view of the
traversed-so-far filesystem.

Reviewed By: xavierd

Differential Revision: D24299962

fbshipit-source-id: 275c849846bf45a2e80780411d60266b961d825b
2020-10-23 11:16:10 -07:00
Jan Mazur
7476abd217 log commits with changeset id to scribe
Summary:
Adding number of changed files to mononoke_commit and mononoke_commit_draft scribe categories.

I need to create scuba table for mononoke_commit scribe category.

Reviewed By: krallin

Differential Revision: D24390738

fbshipit-source-id: 80b9917751a42377b49190cbb2f650a21b086e74
2020-10-23 07:46:15 -07:00
Pavel Aslanov
23fc168668 convert ManifestOps to new style futures
Summary:
- convert ManifestOps to new style futures
- at this point `//eden/manifest:manifest` crate is completely free from old style futures

Reviewed By: krallin

Differential Revision: D24502214

fbshipit-source-id: f1cdb11bd8234f22af5c905243f71e1e9fca11f1
2020-10-23 06:42:35 -07:00
Pavel Aslanov
af0933e044 convert manifest tests to new style futures
Summary: Convert manifest tests to new style futures

Reviewed By: farnz

Differential Revision: D24464328

fbshipit-source-id: 0b95d526b43b13384ec6ec5a4791b06f154c9b23
2020-10-23 06:42:35 -07:00
Pavel Aslanov
129c89972b convert manifest derivation to new futures
Summary: `derive_manifest_*` functions are now based on new futures and use new bounded_traversal

Reviewed By: krallin

Differential Revision: D24448824

fbshipit-source-id: 5af06851f9c48a73e53daafe1f4972c186a5ca80
2020-10-23 06:42:35 -07:00
Kostia Balytskyi
0f104b8b00 commit_rewriting: make CommitSyncConfigVersion serializable into mysql
Summary: Somewhat more convenient to work with, no need to look inside.

Reviewed By: StanislavGlebik

Differential Revision: D24474898

fbshipit-source-id: 7ee0920e7d0d5a2102c68695a5cc0d9e237d958d
2020-10-23 04:59:49 -07:00
Simon Farnsworth
2649095959 hg: check reponame when looking diffs via GraphQL
Summary:
Users get confused and try to pull from the wrong repo - e.g. `hg checkout D23665606` in `fbsource` - and find the resulting output about missing hashes scary.

Give them a nice clear error instead.

Reviewed By: wez

Differential Revision: D24485097

fbshipit-source-id: e9aeb7ed6aaf494f828d736ca5203100807fb321
2020-10-23 04:21:01 -07:00
svcscm
b7203b4cc5 Updating submodules
Summary:
GitHub commits:

a09f3efbd3
a947fd995c
1592db4a5b
b9967031db

Reviewed By: yns88

fbshipit-source-id: 2ac90d17b7cb8860cf5cb4ce2fb2d60a8bcf72fa
2020-10-23 01:52:58 -07:00
Stanislau Hlebik
f10d1bb5d4 mononoke: add an option exclude files from rsync
Reviewed By: ikostia

Differential Revision: D24475159

fbshipit-source-id: e955dd37a8e222dfc56cf0311d8f57a76e73e0c4
2020-10-23 01:23:39 -07:00
svcscm
a43808bbec Updating submodules
Summary:
GitHub commits:

a7c4b5ba08

Reviewed By: yns88

fbshipit-source-id: 8c043b79df6c7e3ab06b8836cca2095e867594cc
2020-10-23 01:23:39 -07:00
svcscm
f6e3f6161e Updating submodules
Summary:
GitHub commits:

ab3a3c54a1
99f8dcdd82
832b2b623e
b1cdb8cc86
c20c0b4960
340e4ee157
10138eeb07

Reviewed By: yns88

fbshipit-source-id: ccae287dbf430549a8d96b9bdb30ebfb78641897
2020-10-22 18:35:02 -07:00
Durham Goode
3e747835e2 revisionstore: fix indexedlog max size limit
Summary:
indexedlogdatastore is supposed to use remotefilelog.cachelimit to set
the max size, but instead it was setting the max per-log size, which means the
max size was N times bigger. Let's fix that.

Reviewed By: xavierd

Differential Revision: D24483181

fbshipit-source-id: f33cedbfdbb318e9d5eb9fda497645050b93e9fe
2020-10-22 18:21:11 -07:00
Thomas Orozco
24560ded2e thrift: call hooks from Rust codegen
Summary:
This updates Thrift codegen for Rust to call hooks on a context stack in a
similar fashion as the C++ codegen.

There is still a bit of room for improvement here. Notably, we don't know how
much data we actually read or sent to the client, so for now those are
hard-coded to zero in the codegen. That's better than not calling those hooks
at all (which is what happens right now), but it could stand to be improved.

Reviewed By: jsgf

Differential Revision: D24445298

fbshipit-source-id: 470daf03057424dc300b6a193668be835ae28452
2020-10-22 17:01:47 -07:00
Thomas Orozco
fdeafe05d7 don't generate server-side stubs in checked-in codegen
Summary:
In the next diff in this stack, I'm changing the Rust thrift library as well as
the codegen, and this is causing quite a bit of pain with regard to this
codegen that is checked in to the hg build:

- Regenerating the codegen isn't super obvious.
- The Sandcastle build fails because it uses the current codegen with the old
  library.

According to lukaspiatkowski, this has also been a problem in the past when Eden got
migrated to Tokio 0.2, so I'd like to save myself and others some pain by just
not generating the server side codegen and not checking it in, since it's
unused. This reduces the surface of stuff that might go out of sync.

#forcetdhashing

Reviewed By: markbt

Differential Revision: D23649604

fbshipit-source-id: d684bec427431a366de42c88e53072caa98d5b2f
2020-10-22 17:01:47 -07:00
svcscm
be9020c968 Updating submodules
Summary:
GitHub commits:

223a443b2d
d5d78479de
dab4025802
1c78e4b235
970d978607
5f4ada09e7
1e9d1c7718
28e1666726
e9d7ff5d73
6969948bcc

Reviewed By: yns88

fbshipit-source-id: 63e5d23b7257a6bc00436c149c3af3c9d0acb57a
2020-10-22 16:45:14 -07:00
Xavier Deguillard
c5d631fd09 service: unify startup on Windows/Linux/macOS
Summary:
Now that Linux/macOS startup no longer uses fork it becomes trivial to share
the same code with Windows. This improves Windows startup in several different ways:
 - `edenfsctl start` now displays the status of the start process in the console
 - `edenfsctl start` no longer has an arbitrary timeout after which it reports
   a timeout even though EdenFS is still starting up.

This also kills a bunch of Windows specific code that is no longer needed.

Reviewed By: fanzeyi

Differential Revision: D24393690

fbshipit-source-id: 28100aec96da81c92d5b592353edceed332e2364
2020-10-22 16:24:17 -07:00
Xavier Deguillard
a75af7a63d PathFuncs: allow paths on Windows to be '\' separated
Summary:
Previously, when that code was ported on Windows, paths separator were
converted from '\' to '/' when a wide string was provided, all the other paths
were treated as is.

The main issue with this strategy is that not all paths can be converted, the
non-stored ones for instance are immutable, which leads to some subtle bugs
down the line. For instance, the paths: "Z:/foo/bar/baz" and "Z:\foo/bar\baz"
would not be equal as the path separator isn't the same, but both of these are
actually the same path underneath.

To solve this, this diff first introduce a Windows path separator, and then
modifies the path comparison functions to ignore the path separator and only
compare the components.

I'm definitively not a fan of the pattern I use for searching for both / and \
in paths, suggestions are welcome for how to improve that.

Reviewed By: chadaustin

Differential Revision: D24376980

fbshipit-source-id: 0702bf775c7c3937b2138abd5a63d339ac80aaed
2020-10-22 16:24:17 -07:00
Xavier Deguillard
fafb9177c8 path: verify that composed paths are comprised of valid PathComponent
Summary:
The path iterator functions are skipping sanity checking of paths as they
assume that the path has already been validated. Unfortunately, that isn't the
case as the only sanity checking we are doing is on the beginning and end of
the string that is passed in. Notably, `RelativePathPiece{"./foo"}` would not
fail the sanity checking even though it it's not made of valid components.

Reviewed By: chadaustin

Differential Revision: D24439804

fbshipit-source-id: 852b3a1180b185cb0bfb96bf5bcdc98b231f32c5
2020-10-22 16:24:17 -07:00
Xavier Deguillard
566ca81dc0 utils: truncate nul bytes after obtaining a temp file name
Summary:
As the string is created with MAX_PATH wide characters, a lot of them are going
to stay nul. A future diff adds a sanity check to the PathComponent that
validates that it doesn't contain any nul bytes, which chokes on these paths.

Reviewed By: chadaustin

Differential Revision: D24479715

fbshipit-source-id: e51c4b1c53a3f375664c0c0a1e325ebe302b0cd3
2020-10-22 16:24:17 -07:00
Durham Goode
1cb5071e8b treemanifest: remove error about prefetching multiple keys
Summary:
When the new rust store is enabled, it can throw exceptions if the rust
code tries to fetch multiple trees when the server doesn't support bfs fetching.
The python code path ignores these requests, letting the eventual single tree
lookup path (i.e. not the prefetch path) perform the download. Let's let the
rust code do the same for rust stores.

Eventually every repo will support bfs fetching and we can delete this code path
entirely.

Reviewed By: quark-zju

Differential Revision: D24459106

fbshipit-source-id: d9a931364bd3235113b0dbcf09cba35a16e16e8e
2020-10-22 14:29:10 -07:00
Xavier Deguillard
5917e7306e inodes: enable tests on @mode/win
Summary: Comment out the tests that do not compile on Windows.

Reviewed By: fanzeyi

Differential Revision: D24458438

fbshipit-source-id: 14bd4803ccd68ea6e02f95f868b8a801161518ea
2020-10-22 14:11:10 -07:00
Stanislau Hlebik
356bc0d69f mononoke: add an option to limit the number of changed files in rsync
Summary: let's make sure we don't create too big commits by setting the limit

Reviewed By: ikostia

Differential Revision: D24451569

fbshipit-source-id: 7b4f653509fce47d921e55dddad2de49ac5672a3
2020-10-22 13:01:25 -07:00
Stanislau Hlebik
87f26ebe8d mononoke: admin rsync command
Summary:
This is a command that creates a commit that copies one directory into another.
Later you can take a commit and push it.

At the moment the command is very simple and also not very useful. At the very
least we need a way to limit the size of the commit, and this option is coming
in the next diff.

Note that the command doesn't overwrite the files if they exist in the target
directory - we may add an option to do so in future. Also if a file exists in
target directory but doesn't exist in source directory then it's left
untouched.

Reviewed By: ikostia

Differential Revision: D24451568

fbshipit-source-id: 2c7b945aadc99a1ca960349992c3684ef02ba35e
2020-10-22 13:01:25 -07:00
Xavier Deguillard
acb4d6e672 utils: compile the tests with @mode/win
Summary: This merely filters out the files that don't compile on mode/win.

Reviewed By: chadaustin

Differential Revision: D24456917

fbshipit-source-id: d314ca2936e064133f2fc6a6199b38d17f2d9ef7
2020-10-22 12:52:13 -07:00
Chad Austin
7ec5741e06 indicate access() is a read FUSE call
Summary:
FUSE_ACCESS was incorrectly categorized as an 'other' FUSE request
type. Categorize it as 'read'.

Reviewed By: wez

Differential Revision: D24437458

fbshipit-source-id: 5ea0a7d5a0fd57b9230f37674fd2568a4917bab5
2020-10-22 11:57:11 -07:00
Chad Austin
37f030ba72 optimize ACL lookups by pretending to support FUSE_POSIX_ACL
Summary:
I noticed while playing with `eden strace` that, on my Linux machine,
running `ls -lA` would cause a pile of FUSE_GETXATTR requests for
`system.posix_acl_access` and `system.posix_acl_default`.

FUSE does not cache getxattr yet, but the kernel can cache ACLs if we
pretend to support them. Thus, advertise FUSE_POSIX_ACL support, but
continue returning ENOSYS for all setxattr calls.

Reviewed By: wez

Differential Revision: D24039130

fbshipit-source-id: a2e69c43b728fb51fb1d1b41ca9d31eba8efaa19
2020-10-22 11:57:11 -07:00
Chad Austin
c2e9a4acd7 eden strace
Summary:
Add an `eden strace` command that subscribes to all FUSE events for
the specified mount (or current directory), and streams them to stdout
with response latency measurements and the requesting process.

If there are any pending requests at the time `eden strace` is run,
they are printed first.

Reviewed By: wez

Differential Revision: D24038978

fbshipit-source-id: 59a2112dbdb1708571d12b04bdccaf9eca725cf7
2020-10-22 11:39:49 -07:00
svcscm
0e368d7b37 Updating submodules
Summary:
GitHub commits:

eaee7b4bae
e6f158c56d
f8aa1bd98c
6a782bda39

Reviewed By: yns88

fbshipit-source-id: 08dd6c610a21e939a43eee3f2a56af839997154e
2020-10-22 11:09:24 -07:00
Kostia Balytskyi
1710662e9e hook_tailer: be able to run on push-redirected commits
Summary:
This is to be able to run the hook tailer on fbsource commits, which had been
push-redirected from ovrsource.

Reviewed By: StanislavGlebik

Differential Revision: D24447814

fbshipit-source-id: bfb164ec45dd2d4d5c3c83f133b48000a71a50a3
2020-10-22 02:48:18 -07:00
Kostia Balytskyi
65f806fa2a cross_repo_sync: don't expect Option<CommitSyncConfigVersion>
Summary: We know it's there.

Reviewed By: StanislavGlebik

Differential Revision: D24449015

fbshipit-source-id: 3ce402b21ce1b8bd4f28980d42f86651bf77c68f
2020-10-22 02:44:09 -07:00
Mark Thomas
253d8253b9 scribe_commit_queue: log all identities of pushing user
Summary: As well as the unixname of the pusher, log all identities provided by the pusher.

Reviewed By: krallin

Differential Revision: D24446453

fbshipit-source-id: 0854b3a10221da16d87b53fd849f033f7bb2a27e
2020-10-22 01:11:11 -07:00
Mark Thomas
f31d6918a5 scribe_commit_queue: fix deps lint
Reviewed By: StanislavGlebik

Differential Revision: D24447560

fbshipit-source-id: 426af3393b6418ad29f43ef2152f3995aace0cc1
2020-10-22 01:11:10 -07:00
svcscm
bc8b8a4241 Updating submodules
Summary:
GitHub commits:

db71ef8cf9

Reviewed By: yns88

fbshipit-source-id: 12241b7ef903173015fa8d9ea6ae896066ea1e3e
2020-10-22 01:11:10 -07:00
svcscm
6fcdba8845 Updating submodules
Summary:
GitHub commits:

298b697f65

Reviewed By: yns88

fbshipit-source-id: 11825dabd8b6b4edcbaceab91bd7cd18b2061302
2020-10-21 18:43:46 -07:00
Jun Wu
a5136d1f18 phabstatus: prefetch diff status using new smartset.prefetch APIs
Summary:
Use the new API `smartset.prefetch` for the prefetching logic for `log`
commands.

Prefetching is only enabled if the template contains related symbols.

Reviewed By: DurhamG

Differential Revision: D24289052

fbshipit-source-id: 810da3b249d45a7516aaf39a0b74f6965b87f96f
2020-10-21 17:49:15 -07:00
Jun Wu
ea03d62698 log: prefetch commit data by template
Summary: This makes `log` command more efficient using the `hybrid` backend.

Reviewed By: DurhamG

Differential Revision: D24293498

fbshipit-source-id: db3bac28a0b04ba6af449357bc08b285cd62780d
2020-10-21 17:49:15 -07:00
Jun Wu
afa9da8ff8 smartset: add prefetch APIs
Summary:
Add prefetch APIs on smartset to make it easier to specify what need to be
prefetched.

This is intended to be used by `{phabstatus}`, log with lazy commit text backend,
and revsets like `author(x)`, etc.

Reviewed By: sfilipco

Differential Revision: D24321292

fbshipit-source-id: 7e645ccbe39feeae1e2da7bfb115623945bf9cef
2020-10-21 17:49:15 -07:00
Jun Wu
9f0d3e4745 smartset: add clone API
Summary:
This makes it possible to "fork" sets and set different prefetch properties on
them.

Reviewed By: sfilipco

Differential Revision: D24365329

fbshipit-source-id: c4bdd6ab21e6a891a02d450f9e6edab3a3781eca
2020-10-21 17:49:15 -07:00
Jun Wu
9f664a8b30 smartset: use Rust reentrant generator for generatorset
Summary:
The generatorset has a pure Python implementation for rewindable generator.
However it is not thread-safe, and we want thread-safety since the set
iteration will be driven by async Rust in multiple threads.

One of the issues is, during thread 1 `next(gen)`, thread 2 might call
`next(gen)`, and that's not allowed by Python.

Fix them by switching to the Rust RGenerator.

Reviewed By: DurhamG

Differential Revision: D24365328

fbshipit-source-id: 2785e80c7c460a7f754ed23e3af99f4a5c9fbcdf
2020-10-21 17:17:08 -07:00
Katie Mancini
2721a17907 Handle permission errors in eden du
Summary:
Buck-out can contain some files that we wont have the ability to read on mac os
(like .Spotlight-V100 and .Trashes). This means `eden du` will error out on
reading them.

We handle FileNotFound but not permission errors, lets treat them the same.

Reviewed By: genevievehelsel

Differential Revision: D24396252

fbshipit-source-id: 31e2ecfeff1783ae9c3479f59335d95a0d9b6de7
2020-10-21 17:00:10 -07:00