Commit Graph

14713 Commits

Author SHA1 Message Date
Jeremy Braun
93f731cf79 Limit watchman+eden's fetching of inodes on common hg updates
Summary: Eden's `getEntryInformation` API currently loads Inodes for all paths queried, force-materializing the path component's Inodes. This often isn't required, and eden could be fetching non-loaded data from the object store instead.

Reviewed By: chadaustin

Differential Revision: D35372760

fbshipit-source-id: e31a450a20b09249f03339dcd1aeca2eb363046e
2022-04-19 09:33:32 -07:00
Jeremy Braun
ab3918ad4a Add EdenMount::getInodeOrTreeOrTreeEntry, use it in thrift calls for SHA1 and BlobMetadata
Summary: There's no need to materialized iNodes to request the content Sha1 and/or BlobMetadata, as that can be fetched from the objectStore.

Reviewed By: chadaustin, xavierd

Differential Revision: D35467564

fbshipit-source-id: 2848f4d21725a9f5d40251fde2e0eb29ea81302e
2022-04-19 09:33:32 -07:00
Muir Manders
679a25d42b pyerror: don't strip off anyhow context
Summary: Instead of stripping anyhow context off all errors, only strip within the concrete error handler "specific_error_handler". This way, the fallback error handler (that produces the beloved RustError) gets the top level error with all the anyhow context.

Reviewed By: DurhamG

Differential Revision: D35602091

fbshipit-source-id: 547ae7e1a2352af74817f90ab08cadb0fce65efc
2022-04-19 07:00:24 -07:00
Stiopa Koltsov
369873ff18 Regenerate
Summary:
Following diff D35738033 adds another crate and reindeer generates unrelated changes.

Run

```
fbcode/common/rust/tools/reindeer/vendor
```

without any changes.

Reviewed By: dtolnay

Differential Revision: D35738034

fbshipit-source-id: 0539b8abbd694479dbe89939cb8c5ddc6272bd71
2022-04-19 00:29:05 -07:00
Katie Mancini
ad04eb9bc1 add unit test for access
Summary: This code has enough risk of a copy paste error that it deserves a unit test.

Reviewed By: chadaustin

Differential Revision: D35161787

fbshipit-source-id: 5691d13a74a0f059dfd6a93ea2852dca8399a165
2022-04-18 21:55:15 -07:00
CodemodService Bot
712c09c627 Daily common/rust/cargo_from_buck/bin/autocargo
Reviewed By: zertosh

Differential Revision: D35670295

fbshipit-source-id: f03ed9d4247923ee4dad6605b261a3ef69b35f67
2022-04-18 19:06:16 -07:00
Jun Wu
4ae35dabeb testing: add hghave utilties to runtime
Summary: Enable .t runtime to access hghave checks.

Reviewed By: DurhamG

Differential Revision: D34725128

fbshipit-source-id: c821fe34482f160951810a26f3ed9b6b466dd1f0
2022-04-18 16:13:35 -07:00
Jun Wu
0307ee609a testing: memorize hghave checks
Summary:
Most hghave checks are "immutable" results. Memorize them to reduce cost.
If some checks depend on actual tests, we can special case them later to
not be memorized.

Reviewed By: DurhamG

Differential Revision: D34725137

fbshipit-source-id: f010bddb856df1950854b839aa8c311c6a40bdbd
2022-04-18 16:13:35 -07:00
Jun Wu
7cae09de0b testing: vendor hghave.py
Summary:
This allows using the hghave library without shelling out to a `hghave` binary
in tests.

Reviewed By: DurhamG

Differential Revision: D34725136

fbshipit-source-id: 1b8115a791414a68ebd2e7ec077f77bca2a0833a
2022-04-18 16:13:35 -07:00
Jun Wu
ab08faf821 testing: compare multiline text with glob patterns
Summary: This is ported from testutil's eqglob (D16209212 (b87a92215f)).

Reviewed By: DurhamG

Differential Revision: D34725120

fbshipit-source-id: 2a94a9065f82e6d3b49770a7df2f398e5131f164
2022-04-18 16:13:35 -07:00
Jun Wu
d0b6e10966 testing: add 'sed' to shell builtin
Summary: `sed 's/a/b'` (and other sed commands) are used frequently in tests. Implement it.

Reviewed By: DurhamG

Differential Revision: D34790212

fbshipit-source-id: 76bfac795e6b14a8d83cbee9fd5c74341112786e
2022-04-18 16:13:35 -07:00
Jun Wu
66b5be97ee testing: add 'wc -l' to shell builtin
Summary: `wc -l` is also commonly used. Implement it.

Reviewed By: LynBusch

Differential Revision: D34725129

fbshipit-source-id: b20259026424d2bdd7ab8c53b7683c120cda96e3
2022-04-18 16:13:35 -07:00
Jun Wu
93fd23fbe2 testing: add grep to shell builtin
Summary:
`grep` is commonly used. Implement it.

This diff might also serve as an example about how to extend the stdlib.

Reviewed By: LynBusch

Differential Revision: D34725127

fbshipit-source-id: 32ba5cdfc521f538923f05c09c2fa9afb6e88f05
2022-04-18 16:13:35 -07:00
Jun Wu
bf75662088 testing: initial transformer from .t to Python
Summary:
To be able to execute .t test without bash, and to be future
proof so tests can use Python more easily instead of bash.
Define a way that supports both classic .t and pure Python
and a mix of them.

See the docstring for details.

Not all `.t` features are handled in this diff.
Namely `#testcases` support is missing.

Reviewed By: LynBusch

Differential Revision: D34725133

fbshipit-source-id: f68c88a2e5c0fa4f3ce3de0b39ff17e11350b1be
2022-04-18 16:13:35 -07:00
Jun Wu
43d53709b2 testing: fs-related shell builtin commands
Summary: Add commands to operate on the filesystem.

Reviewed By: LynBusch

Differential Revision: D34725131

fbshipit-source-id: 7769e0d10764e3c39ea3e5bd11c9fda07af80895
2022-04-18 16:13:35 -07:00
Jun Wu
2b99d03e96 testing: initial shell interpreter
Summary:
See the added module docstring for details.

Unlike testutil/dott (D16172901 (27908b883c)), this implementation behaves
much more closely to a real shell thanks to conch-parser.

For reviewers: This is a large change but still relatively
simple compared to other shell implementations. It might be
easier just looking at the tests.

Differential Revision: D34725134

fbshipit-source-id: 1ec54ae2c0146ba2533c16f3fbc27f4275ca80f4
2022-04-18 16:13:35 -07:00
Jun Wu
a4a0436d23 bindings: expose conch-parser
Summary: Expose the conch-parser features to Python.

Differential Revision: D34725124

fbshipit-source-id: 9993a67fda8f92f14fb6897a2b50747742735c3a
2022-04-18 11:56:11 -07:00
Jun Wu
86dbc0f256 conch-parser: serde::Serialize for AST types
Summary:
This makes it easier to export the AST in other formats (ex. Python objects).

Conflicted names (ex. Parameter::Star vs SimpleWord::Star) are attributed with serde(rename) so they can be distinguished more easily.

Reviewed By: LynBusch

Differential Revision: D34725130

fbshipit-source-id: d91a80cec5b858669f5d16c95aa63046470730e1
2022-04-18 11:56:11 -07:00
Jun Wu
fde88fdd4e conch-parser: add TARGETS
Summary: Add TARGETS so it can be used in buck builds.

Reviewed By: yancouto

Differential Revision: D34725122

fbshipit-source-id: 00b958738b05a30a4b9a1470978645b5fad0ad8d
2022-04-18 11:56:11 -07:00
Jun Wu
7762e74ca2 conch-parser: resolve rustc warnings
Summary: This makes the code compile with future versions of rustc.

Reviewed By: LynBusch

Differential Revision: D34725123

fbshipit-source-id: ec0d4b0e5fee338487dbb817ac64333a6f7a1040
2022-04-18 11:56:11 -07:00
Jun Wu
f8932b955d third-party: vendor conch-parser
Summary:
Import https://github.com/ipetkov/conch-parser commit
e1d3ef269684f46c9696bf3f83e81c37e711570d under MIT license.

The conch-parser seems to be a decent shell parser that
solves the shell syntax parsing headache. For example,
it handle heredocs with substitution (`<< EOS`, `$V`
should be substituted) or no-substitution (`<< 'EOS'`,
`$V` or a slash does not have special meaning) properly.
Heredoc is frequently used in our tests. Most other
shell parsing libraries I checked lack proper heredoc
handling, and/or have an incompatible license. Although
it seems some corner cases can be improved, for example,
`A=1 for i in $A; do ...; done` treats `for` as a
normal command instead of a shell loop keyword. But
that does affect the main test use-cases.

In a later diff the library is modified to support serialization.
Given that the public crates.io is too old and the modification
needed, I imported it into edenscm's own vendor directory
for local customization.

Differential Revision: D34725119

fbshipit-source-id: 3e7d3d3d4f92380a0fd104275e75863332b429db
2022-04-18 11:56:11 -07:00
Yipu Miao
246a6be36a Add a benchmark of setPathObjectId
Summary: A script to run the benchmark for ```setPathObjectId```

Reviewed By: chadaustin

Differential Revision: D35271074

fbshipit-source-id: a0f6d9ab1fe6ace929f615a620e2cd4c36fdcc0e
2022-04-18 10:41:25 -07:00
Pyre Bot Jr
24a256db61 suppress errors in fbcode/eden - batch 1
Differential Revision: D35696165

fbshipit-source-id: e69071d77122fd9e6463915cfa1d930d20ed4c17
2022-04-15 22:13:58 -07:00
Pyre Bot Jr
64d2708de8 suppress errors in eden
Differential Revision: D35674585

fbshipit-source-id: 891fea7bc4db979d2d087ff5b1ee93404de2e75e
2022-04-15 09:21:53 -07:00
David Tolnay
560ff38618 Remove even more capacity assertions that aren't true on 1.60's Vec implementation
Summary:
Part of T117105786 -- a set of test failures that bisect to {D35558601 (238a176019)}.

D35558601 (238a176019) already made these same changes to eden/hg-server/lib/indexedlog/src/base16.rs and eden/scm/lib/indexedlog/src/base16.rs. I didn't imagine that there would be not 2 but **4** identical copies of this data structure and tests. :O

Reviewed By: zertosh

Differential Revision: D35660712

fbshipit-source-id: b2e624da04eafd1b2f3b13a0219d84645d96fcdd
2022-04-14 16:32:34 -07:00
Pranjal Raihan
56e2440c1d Make *AsyncClient types aliases instead of subclasses of apache::thrift::Client
Reviewed By: iahs

Differential Revision: D35332438

fbshipit-source-id: c119097d841be5f356327c5d9a347bf279be9c5f
2022-04-14 10:57:15 -07:00
Jan Mazur
53e5d327df make users of repo_import aware that it can be dangerous in some cases
Summary: Merging a repo that has multiple refs / heads / unmerged branches (relatively to master) can be dangerous. Let's make users aware of that.

Reviewed By: farnz

Differential Revision: D35608687

fbshipit-source-id: a103862ebd89c034cfe23b27606aa558cddb069c
2022-04-14 07:46:16 -07:00
CodemodService Bot
4f4598dd18 Daily common/rust/cargo_from_buck/bin/autocargo
Reviewed By: krallin

Differential Revision: D35639860

fbshipit-source-id: b37e1013a9c10db729589fe817e787ff64f30420
2022-04-14 06:37:34 -07:00
Mateusz Kwapich
9be245e677 test for forced reloading
Summary: This part of segmented changelog was never tested and we depend on this behaviour for emergencies. We're going to depend on it even more now that we won't reload the SC every hour.

Reviewed By: yancouto

Differential Revision: D35614793

fbshipit-source-id: cd10deceab7fb1be5a77f26ef904c09ea7562996
2022-04-14 03:29:44 -07:00
Mateusz Kwapich
11c966228b avoid reloading segmented changelog if tailer did not make any progress
Summary:
We should not reload and reset the server-side segmented changelog if the
tailer has not written a new version since we last loaded - instead, we should
keep our existing one in-memory, because it will be more up to date than the
one we load.

In situations when the tailer is not working properly this might mitigate the issue for the customers instead of exposing them to it.

Reviewed By: farnz

Differential Revision: D35408139

fbshipit-source-id: 10d30b6d2d055fc938f9d0f22f3c633f7d7495c2
2022-04-14 02:16:25 -07:00
Mateusz Kwapich
98d1c489ff use the SegmentedChangelogManager in the test
Summary:
We use completely different codepath for testing, let's go through the same as
we go in prod.

Reviewed By: farnz

Differential Revision: D35408141

fbshipit-source-id: 05f3f026c0c15b7b460c554828089247f8680eaf
2022-04-14 02:16:25 -07:00
Stanislau Hlebik
a2d7a49953 debugsparsematch: allow to separate with \0
Summary: It's useful for parsing

Reviewed By: quark-zju

Differential Revision: D35609276

fbshipit-source-id: 16890ac4557636fbc68f89251584261e153e3e8a
2022-04-13 23:15:50 -07:00
Durham Goode
5d3d0f567d buck: enable buck build on Windows
Summary:
This makes 'buck build mode/win //eden/scm:hg' work on Windows. The
resulting binary doesn't actually run though, since buck run depends on hg.sh
which doesn't work on Windows. That will need to be converted later.

Reviewed By: quark-zju

Differential Revision: D35133291

fbshipit-source-id: 15070930612b56a381a8c8e311f6eaac3f1d9856
2022-04-13 16:38:34 -07:00
David Tolnay
238a176019 Update fbcode to Rust 1.60.0
Summary: Release notes: https://blog.rust-lang.org/2022/04/07/Rust-1.60.0.html

Reviewed By: zertosh

Differential Revision: D35558601

fbshipit-source-id: 02fd798c00ef02a3ebd2580ff921d0807a29e092
2022-04-13 16:19:27 -07:00
Andrew Hariri
542cd38136 update reqwest to 0.11.10
Summary: Maintainer still unresponsive on https://github.com/seanmonstar/reqwest/pull/1412

Reviewed By: wqfish

Differential Revision: D35600272

fbshipit-source-id: e01a7ae459ea215180158f4186d33fb1fbed18ff
2022-04-13 14:44:40 -07:00
Mateusz Kwapich
0c09a0053c make manager return the loaded version
Summary:
This diff makes the manager return the loaded segmented changelog version
which is going to allow us

Reviewed By: farnz

Differential Revision: D35408143

fbshipit-source-id: 748901c992e2e67e561970f836892a13c6d88e27
2022-04-13 12:47:03 -07:00
Mateusz Kwapich
a866d52cbc add a function for checking latest version to manager
Summary:
To avoid unnecessary reloads we need a facility to check the latest version (so
we can compare it with one currently loaded)

Reviewed By: farnz

Differential Revision: D35408138

fbshipit-source-id: 14021edda52105fad71ce430feee42bc42659546
2022-04-13 12:47:03 -07:00
Mateusz Kwapich
5ef2f55dea change periodic reload to a named struct
Summary: This will allow me to augment it with other stuff.

Reviewed By: yancouto, farnz

Differential Revision: D35408140

fbshipit-source-id: 1096255eabd84c995ba0bb71209eafd3974c0610
2022-04-13 12:47:03 -07:00
Michael Bolin
056ae53196 rename github to github-cli because it broke autocargo
Reviewed By: zertosh

Differential Revision:
D35615001
Ninja: need to fix autocargo and oss-eden-darwin-getdeps is taking forever

fbshipit-source-id: 1dfc74532615823be3e4cb39ff82e1f7cc0ea1a6
2022-04-13 12:20:02 -07:00
Chad Austin
00b71929a3 reduce unit test runtime by decoupling EdenServer from the full set of BackingStore implementations
Summary:
By lifting construction of the BackingStores into main, the core
"service" target no longer needs to depend on their
implementations. This shaves almost 60% off the unit test runtime on
my devserver. The tests could be made even faster by decoupling all of
our core logic from HgBackingStore and friends.

Reviewed By: xavierd

Differential Revision: D34774356

fbshipit-source-id: 87c2d5f44cfd84d6e01bea44dd4105f6415ce162
2022-04-13 12:01:59 -07:00
Chad Austin
036bfbf745 allow BackingStores to advertise whether checkout can assume blob IDs and contents are 1:1
Summary:
Some backing stores have a 1:1 relationship between blobs and their
IDs. Others don't. If they do, checkout can compare blob contents more
quickly.

In addition, add a config knob that we can use to opt hg into the fast
path, at the risk of correctness. It might be useful for getting out
of sticky situations where any update triggers a ton of fetches.

Reviewed By: xavierd

Differential Revision: D34761024

fbshipit-source-id: 36d513d37c38fc29a708e583ebc2614f07f11bb4
2022-04-13 12:01:59 -07:00
Yan Soares Couto
959764d6da snapshot: Fix max-untracked-size with symlinks and set default to 1GB
Summary:
We currently have no default. Untracked files can be very large (like binaries and such), and easy to miss. We already had an option for that, but were not using it by default. Should the default be even smaller?

It also didn't work with invalid symlinks, as `Path.stat` tried to follow symlinks, so I fixed that.

If a client really wants to upload a large file, they can specify the command line argument by themselves.

Reviewed By: farnz

Differential Revision: D35498620

fbshipit-source-id: 7c21c779a680fb209b82943ec59aa5523cd1608c
2022-04-13 11:26:50 -07:00
CodemodService FBSourceRustfmtLinterBot
2c8f120b81 Daily arc lint --take RUSTFMT
Differential Revision: D35606540

fbshipit-source-id: 1d51e1170b13c0c8512e935ea83b03f96fc2d4e4
2022-04-13 11:07:31 -07:00
Michael Bolin
87d6ff062c add Rust method to fetch pull request data
Reviewed By: quark-zju

Differential Revision: D35573631

fbshipit-source-id: 316485394963bd0c2e06d536d80611a8a0336bc4
2022-04-13 02:47:11 -07:00
Michael Bolin
12a75b87e5 generalize logic to make requests to GitHub GraphQL API
Reviewed By: quark-zju

Differential Revision: D35567823

fbshipit-source-id: 99b14c80a08ccaf6e69caa442d716e8f17f25c11
2022-04-13 02:47:11 -07:00
Michael Bolin
684daeb665 introduce Rust crate to talk to GitHub via its GraphQL API
Reviewed By: quark-zju

Differential Revision: D35491509

fbshipit-source-id: 489a491d1e81e93608f93ef25c3fbbb75e0907c4
2022-04-13 02:47:11 -07:00
Michael Bolin
2d7c25b779 import GitHub's schema.docs.graphql
Summary:
This diff imports the most recent version of `schema.docs.graphql`
available on https://docs.github.com/. Details explained in the `README.md`
included in this commit.

Reviewed By: quark-zju

Differential Revision: D35600010

fbshipit-source-id: 52050db604ca884ce7465da58c3ed2aa3fb85dbc
2022-04-13 02:47:11 -07:00
Ilia Medianikov
36297cb8ff mononoke: create usable segmented changelog in TestRepoFactory
Summary:
Currently unit tests that need a segmented changelog create it separately from the repo because in test repo there is only a `DisabledSegmentedChangelog`. I am making it possible to have an `OnDemandUpdateSegmentedChangelog` that is both useful and can be updated to include needed commits.

Look on new `test_is_ancestor` test to see what is made possible.

Reviewed By: markbt

Differential Revision: D35053513

fbshipit-source-id: 221600b50c8a0140b08f633dc3758489f2524f60
2022-04-13 02:45:23 -07:00
Ilia Medianikov
e387b2619c mononoke: add FacebookInit to TestRepoFactory
Summary: Segmented Changelog requires `CoreContext` for construction, so we can create a test mock if `TestRepoFactory` has a `FacebookInit`.

Reviewed By: markbt, mitrandir77

Differential Revision: D35251021

fbshipit-source-id: 57051e227804669794f19fc2ca130f52cb938b0c
2022-04-13 02:45:23 -07:00
Huapeng Zhou
7a8725f586 rust: update crossterm to 0.23.0 and arc-swap to 1.5.0
Summary:
Needed to import [cloud-hypervisor](https://github.com/cloud-hypervisor/cloud-hypervisor).

(Note: this ignores all push blocking failures!)

Reviewed By: dtolnay

Differential Revision: D34434866

fbshipit-source-id: 69741b4ac767e0ab5ecc8307dd40267d9dcbf4ae
2022-04-12 23:50:06 -07:00