Summary:
When the `send_async` method is used to dispatch multiple concurrent requests, the method needs to return an `AsyncResponse` for each request. Since `AsyncResponse`'s constructor is itself `async` (it waits for all of the headers to be received), internally the method ends up with a collection of `AsyncResponse` futures.
Previously, in an attempt to simplify the API, the method would insert all of these futures into a `FuturesUnordered`, thereby conceptually returning a `Stream` of `AsyncResponses`. Unfortunately, this API ends up making it harder to consume the resulting `AsyncResponses` concurrently, as one might want to do when streaming lots of data over several concurrent requests.
This diff changes the API to just insert the `AsyncResponse` futures into a `Vec` to allow the caller to use them as desired. To maintain compatibility with the old behavior for the sake of this diff, the one current callsite has been updated to just dump the returned `Vec` into a `FuturesUnordered`. This will be changed later in the stack.
Reviewed By: quark-zju
Differential Revision: D29204195
fbshipit-source-id: ecee8cff430badd8213c2efef62fc68fbd91fde9
Summary: Nothing was using this metadata, and removing it simplifies the subsequent diffs in this stack.
Reviewed By: quark-zju
Differential Revision: D29147228
fbshipit-source-id: aa4828b710c3ef719f4d66adec5f66cd5b7d05d1
Summary:
I'm seeing significant Zippy load when I do a check scrub of our big repo to make sure that it's all in SQL Blobstore as well as our main blob stores.
Teach scrub to not bother talking to the main blobstores unless the write-mostly blobstore is either missing the data or unable to retrieve it.
Reviewed By: ahornby
Differential Revision: D29233349
fbshipit-source-id: 1127129ff283477558cddb03686c3c13aee47fb5
Summary:
Now that the InodeMap is pre-populated with the Overlay, we no longer need the
`folly::kIsWindows` test during checkout as inodes will enter this condition
when unloaded (IsInodeRemembered).
Reviewed By: fanzeyi
Differential Revision: D25106153
fbshipit-source-id: 511d795ae947651e1eaf3c54b8f1ab83c77f5cc4
Summary:
This dep got updated in D29165283 (b82c5672fc) across a major version but the code depending
on it wasn't so now it's broken.
Reviewed By: mitrandir77
Differential Revision: D29229087
fbshipit-source-id: 5f2a14dd9f0447dd4578e8321991dfb3df32dcc2
Summary:
This assumes the Eden binary is next to edenfsctl. Like in the previous diffs,
it isn't.
Reviewed By: xavierd
Differential Revision: D29200149
fbshipit-source-id: a7d335c5f3748c795b20480e26b24b9287708c50
Summary:
Like it says in the title. This fails because it assumes it's next to the
EdenFS binary but that's not true in Buck v2.
Reviewed By: xavierd
Differential Revision: D29200150
fbshipit-source-id: 1be5b6c647047a50dd553031691e9bbec5f9caaf
Summary: Pretty much the same as D29061439 (8f263485d9), but in another place that needs it.
Reviewed By: xavierd
Differential Revision: D29197979
fbshipit-source-id: eafccefee54a64131eb7547ad19ab1da708852e6
Summary:
On a non-EdenFS mount, a case change of a file or directory isn't reported as
the file can still be accessed with the same path, and it has the same content
as before. EdenFS had a different behavior whereas renaming a file to a
different case would report a missing file and an untracked one, with different
casing.
This can be surprising to users as the behavior is different from Mercurial,
and it's also hard to actually fix as 2 renames would need to happen, a single
one would not work due to the case insensitivity nature of the filesystem.
While I do believe that reporting the case change might be more desirable to
allow users to actually commit changes the case of files, Mercurial might be
broken in subtle ways today if we allow this, thus it's best to make EdenFS
behave similarly to Mercurial.
Reviewed By: genevievehelsel
Differential Revision: D29150552
fbshipit-source-id: 6cceaa4c9fa61c03f35fcd91a9c01554da252222
Summary:
On case insensitive mounts, updating between commits that have a different
casing for a file/directory would lead to the update failing due to EdenFS
believing that an untracked file is present in the mount. That conflict is
however bogus and EdenFS simply gets confused in
TreeInode::processCheckoutEntry about the entry with the different casing.
To fix this, we should avoid comparing paths in a case sensitive manner and
instead compare then in a case insensitive fashion. This allows the rest of the
checkout code to update the directory/file in place.
On Windows, there is one more subtlety: we can't change the casing of a
file/directory that is already a placeholder and thus we need to force the
entire hierarchy to be removed, this will also make the checkout fail in case
of untracked files in the hierarchy, which is also the behavior on case
sensitive systems.
Reviewed By: genevievehelsel
Differential Revision: D29121741
fbshipit-source-id: 3d2cdacf296a3d061fc828cd6d04d249542cb63f
Summary: Update versions for several of the crates we depend on.
Reviewed By: danobi
Differential Revision: D29165283
fbshipit-source-id: baaa9fa106b7dad000f93d2eefa95867ac46e5a1
Summary:
## This diff
* We are migrating `edenfsctl strace` to cpp due to segfaulting from Thrift-py3 (similar to {D25515377 (a152fa4585)}).
* This diff implements new subcmd `edenfsctl trace strace` for this cpp migration.
Reviewed By: xavierd
Differential Revision: D29164534
fbshipit-source-id: 4d8ed23393004f394159c36f71e0c78c077c7c73
Summary: It was rolled out weeks ago and there seem to be no complaints so far.
Reviewed By: DurhamG
Differential Revision: D29172396
fbshipit-source-id: 4f3310597ef1bb1abd92ee7207700a3d0039c598
Summary:
add an option to pass some metadata in the token
This will be used for content tokens, for example. We would like to guarantee that the specific content has been uploaded and it had the specific length. This will be used for hg filenodes upload.
Reviewed By: markbt
Differential Revision: D29136295
fbshipit-source-id: 2fbd3917ee0a55f43216351fdbc1a6686eb80176
Summary:
use `commitknown` edenapi api for checking the existing commits
it uses the same `lookup_commits` under the hood but a bit shorter to use
we won't need the tokens for existing changesets, so can use a simpler api
also, make `lookupfilenodes` function a bit shorter
Reviewed By: markbt
Differential Revision: D29134677
fbshipit-source-id: 257624d64480102c34761560b2bd768049cbfa83
Summary:
upload file content into blobstore
the existing Mononoke API already validates the provided hashes and calculates the missing one
we would probably need to write to all multiplexed blobstores, but multiplexing will be addressed separately
Reviewed By: markbt
Differential Revision: D29103111
fbshipit-source-id: 0cac837efc238f618a35420523279fb7aa91668a
Summary: Allow puts to sqlblob with mysql backing to use the InlineBase64 hash type.
Reviewed By: farnz
Differential Revision: D28829452
fbshipit-source-id: 265cf45e55284d34d3002a9db205e14eaee4fa39
Summary: Obsolescence markers have been deprecated in favour of mutation and visibility for some time. Remove syncing of obsmarkers via the commit cloud service.
Reviewed By: liubov-dmitrieva
Differential Revision: D29159443
fbshipit-source-id: 33e1d526a9df5c02059931c880529d695036c616
Summary:
It's useful to have it configurable.
While here, also use slog instead of println to attach timestamp as well
Reviewed By: Croohand
Differential Revision: D29165693
fbshipit-source-id: d844926560b15042445d5861a281870ac102d12e
Summary:
Non-buck build cannot succeed if Python.h is missing.
Let's check it explicitly so we don't pick a bad Python.
Reviewed By: DurhamG
Differential Revision: D29179295
fbshipit-source-id: 421c824053d066914a6611f05815527768f257ee
Summary:
It turns out that `python3` might be a symlink to a fbcode Python that cannot
perform the build in some (not all) environment. Let's try more names like
`python3.8`, `python3.7`, etc.
Reviewed By: DurhamG
Differential Revision: D29178933
fbshipit-source-id: da6cae351f25a90ab8a9da85282d09f79505c5e7
Summary:
IPython 7 has compatibility issues when bundling into zip:
- The dependency parso does not work when embedded in zip. It expects
`python/grammar38.txt` to exist on the filesystem (not in a zip).
- It stacktraces after exiting the IPython shell.
- TAB completion does not really work.
Downgrading to IPython 5 and avoid the bundling the parso dep to resolve the
issues.
This should fix `test-autofix.t` and `test-argspans.py` failures.
Reviewed By: DurhamG
Differential Revision: D29170653
fbshipit-source-id: 14a3d16deaca72fbfb7b3acc0a4246a771c4d0aa
Summary: They will be reused in import_pull_data
Reviewed By: quark-zju
Differential Revision: D29147950
fbshipit-source-id: 192bf33c30067f43c4fcaaf3054741b39efb4e25
Summary: This is an interface for importing pull data into dag
Reviewed By: quark-zju
Differential Revision: D29142979
fbshipit-source-id: b40b94403a044c0b74d1574528aa374ec309a0cf
Summary: This will be used to import pull data into segmented changelog
Reviewed By: quark-zju
Differential Revision: D29142981
fbshipit-source-id: 2d19a035ee0b6cefef8fc0547a5dfb79f284a1de
Summary:
`cl.userust()` is always True after D29020191 (3765f8bd76) so `cl.userust() is False` code
is now dead.
Reviewed By: andll
Differential Revision: D29142464
fbshipit-source-id: f9a7e5c56641218758f12bad3de43d1cd1a71716
Summary:
Like it says in the title. Let's allow specifying an oncall here since that
oncall will be tasked with retroactive review of the commit.
Reviewed By: StanislavGlebik
Differential Revision: D29162534
fbshipit-source-id: 9ed3ac43c38a1120bb16a2f5b5218fdbf80e0d47
Summary:
Similar to earlier diffs in this stack, let's not use
I think we need a solution for this in Buck 2 itself instead of within the Eden tests since this stack is getting a bit out of control, but this is probably worth doing anyway since source repo access isn't desirable (nor available) on e.g. RE.
Reviewed By: xavierd
Differential Revision: D29063216
fbshipit-source-id: 278f718b861adbf3aab9e019baaec8d84bac1433
Summary:
When running tests, we need a way for Eden to find its privhelper. Normally, it
finds it by looking next to the EdenFS binary itself, but while that works in
Buck 1, it does not work in Buck 2, where the binaries are in separate
directories
Compared to the previous diffs, this requires some extra care because on a real
system, EdenFS is running as a setuid binary, and we don't want to let people
just run whatever binary they want the EdenFS privhelper!
To that end, we reject this argument entirely if the binary is setuid.
Reviewed By: xavierd
Differential Revision: D29061439
fbshipit-source-id: bf9427211d4209cf0bea805b9ea3a53270ec455b
Summary: I'd like to use this to cache a `Optional[str]` as well.
Reviewed By: xavierd
Differential Revision: D29132026
fbshipit-source-id: 5989ae636feca061441b2c986343c8ef9759fd69
Summary:
Similar to D29098920 (9a813fb14b), I'm regenerating filenodes now. Because of that I need to
bump cache key
Reviewed By: mitrandir77
Differential Revision: D29134968
fbshipit-source-id: 2f2b5b41bedcb0a037be7eade74e7b45a8990880
Summary: Pack files are no longer supported, yet we still have many tests which exercise them. In preparation for landing `scmstore` as a drop-in replacement for ContentStore, I'm removing our tests which only exist to test datapack-specific functionality.
Reviewed By: DurhamG
Differential Revision: D29099012
fbshipit-source-id: 635a913ee0d93ed8d536e71f8fa6a600b823b343
Summary:
The only real change here is: https://github.com/BurntSushi/ripgrep/pull/1756
This is a patch release but fixes a very glaring bug that others have
depended on. This diff fixes the uses to match the old behavior.
Although it's billed as a "fix", it's actually a huge perf improvement
for Linttool, which uses predominantly recursive suffix globs. The fact
that we don't have to compile ~5,000 regexps at Linttool startup anymore
makes such a huge difference that I am going to do write up soon.
Reviewed By: ndmitchell
Differential Revision: D29085977
fbshipit-source-id: 304470e5fa8cb986738aa0d9dd941641684a9194
Summary:
Python 2 is no longer built anywhere. Let's make the various py3
options the default, like renaming 'make local3' to 'make local' and let's get
rid of the dead setup.py and rpm spec.
Reviewed By: quark-zju
Differential Revision: D29077093
fbshipit-source-id: 0c50c2296fe10ff1db9ac8f9b0df2a4836c0ea5b
Summary:
For a while we had two methods in async runtime: block_on_future and block_on_exclusive, due to historic reasons
Recently those methods were calling same code, and now it is time to replace both of them and rename to block_on
Reviewed By: quark-zju
Differential Revision: D29121107
fbshipit-source-id: 5faa76ae181e491b55d799c23c9de1b4e80298f3
Summary:
If this opens a transaction and fails, then we start a never-ending cycle of
starting hg cloud backup and failing.
This command normally doesn't open transactions but it might via other
extensions, here is one example: P423007962.
This ends up killing people's machines:
https://fb.workplace.com/groups/scm/permalink/3957513017631622/
This fixes that by not kicking off background backups in this case.
Reviewed By: liubov-dmitrieva
Differential Revision: D29136077
fbshipit-source-id: 1c4e6de6147571dd6d728f761324506b1804f303
Summary:
The added command gives access to read (execute) dynamicconfig without
using an on-disk repo.
It can be used by the clone script to stage rollout lazy changelog, or just to
verify dynamic config changes without using a repo.
Reviewed By: DurhamG
Differential Revision: D29123072
fbshipit-source-id: e8856d816a636fa860bfcc9694306a4a37552523
