Summary:
Like it says in the title, this updates us to use Daemonize 0.5, though from
Github and not Crates.io, because it hasn't been released to the latter yet.
The main motivation here is to pull in
https://github.com/knsd/daemonize/pull/39 to avoid leaking PID files to
children of the daemon.
This required some changes in `hphp/hack/src/facebook/hh_decl` and `xplat/rust/mobium` since the way to
run code after daemonization has changed (and became more flexible).
Reviewed By: ndmitchell
Differential Revision: D30694946
fbshipit-source-id: d99768febe449d7a079feec78ab8826d0e29f1ef
Summary:
Manual component version update
Bump Schedule: https://www.internalfb.com/intern/msdk/bump/?schedule_fbid=342556550408072
Package: https://www.internalfb.com/intern/msdk/package/181247287328949/
Oncall Team: rust_foundation
NOTE: This build is expected to expire at 2022/09/01 09:14AM PDT
---------
New project source changes since last bump based on D30663071 (08e362a355e0a64a503f5073f57f927394696b8c at 2021/08/31 03:47AM -05):
| 2021/08/31 04:41AM -05 | generatedunixname89002005294178 | D30665384 | [MSDK] Update autocargo component on FBS:master |
| 2021/08/31 07:14PM PDT | kavoor | D30681642 | [autocargo] Make cxx-build match version of cxx |
| 2021/09/01 04:05PM BST | krallin | D30698095 | autocargo: include generated comment in OSS manifests |
---------
build-break (bot commits are not reviewed by a human)
Reviewed By: farnz
Differential Revision: D30717040
fbshipit-source-id: 2c1d09f0d51b6ff2e2636496cf22bcf781f22889
Summary:
The mockall crate's `automock` attribute previously created nondeterministic output, which leads to frequent random "Found possibly newer version of crate" failures in Buck builds that involve cache.
The affected trait in Conveyor is:
https://www.internalfb.com/code/fbsource/[4753807291f7275a061d67cead04ea12e7b38ae2]/fbcode/conveyor/common/just_knobs/src/lib.rs?lines=13-23
which has a method with two lifetime parameters. Mockall's generated code shuffled them in random order due to emitting the lifetimes in HashSet order. The generated code would randomly contain one of these two types:
`Box<dyn for<'b, 'a> FnMut(&str, Option<&'a str>, Option<&'b str>) -> Result<bool> + Send>`
`Box<dyn for<'a, 'b> FnMut(&str, Option<&'a str>, Option<&'b str>) -> Result<bool> + Send>`
Reviewed By: jsgf
Differential Revision: D30656936
fbshipit-source-id: c1a251774333d7a4001a7492c1995efd84ff22e5
Summary:
As explained in previous diff summary - we need to make those deletions
before we can fixup history. This diff adds relevant args to megarepo tool.
Reviewed By: StanislavGlebik
Differential Revision: D30456464
fbshipit-source-id: 894e27750684f86e18184e3b98beeda8dfb5b53d
Summary:
In one of our large repos the files were synced using ad-hoc scripts between
the repos before the megarepo. This made the history and blame full of
irrelevant commit messages like "sync from repo x". The new functionality will
allow us to re-merge those files with correct histories. To do that we need to
delete the:
* files with wrong history from master branch
* all other files from the source branch with correct history - so only those
are merged in
As an optimisation we're not removing the files that are the same on both
branches from branch with correct history. This will make the merge bonsai
smaller while keeping the history the same (at least when unode v2s are
enabled).
Reviewed By: StanislavGlebik
Differential Revision: D30456465
fbshipit-source-id: f14e8caf8e104c8d8069edc098cfe6419bbe407e
Summary:
The current logic in megarepo merge tool command (which creates arbitrary
merges in the repo) checks if there's intersection between the manifest leaf
entries of both commits. This doesn't allow the merges where there are exactly
same files on both side of merge.
I've updated the logic to look at fsnode manifest diff. As the old futures code
was harder to work with I've updated the code I worked with to async/await.
NOTE: I think that both new and old logic lack one important check: file vs
directory conflicts. We should address this separately.
Reviewed By: StanislavGlebik
Differential Revision: D30456466
fbshipit-source-id: 70303e8ff8ff9a42e5dcff5c1fe6e4912910d4d9
Summary:
These functions were used heavily, and used old futures.
That meant a lot of uselessly copied stuff, and harder interop.
I also took the opportunity to pass `CoreContext` around as a reference, and did some BlobRepo refactoring to use attribute traits where possible.
Reviewed By: StanislavGlebik
Differential Revision: D30368261
fbshipit-source-id: 2e63677601fafa3c2e3d9d3340df0a5f31a19a11
Summary:
The diff is giant, but it's just a one-line change to add the
nested-values feature to slog, we just have a whole bunch of projects dependent
on slog.
Reviewed By: dtolnay
Differential Revision: D30351289
fbshipit-source-id: b6c1c896b06cbdf23b1f92c0aac9a97aa116085d
Summary: `//common/rust/shed/cached_config` is the center of a dependency graph that all only uses old configerator because cached_config uses it. This diff switches all of these over to the new client.
Reviewed By: farnz
Differential Revision: D30357631
fbshipit-source-id: 9a9df74096aa38a06371c6bc787245af71175e48
Summary:
This diff adds some data to BonsaiChangeset that tells whether it is a snapshot or not.
For now, this marks every changeset as not being a snapshot. The next diff will add validation to snapshots, some tests, and mark the current `snapshot createremote` command as uploading snapshots.
Reviewed By: markbt
Differential Revision: D30158530
fbshipit-source-id: 9835450ac44e39ce8d653938f3a629f081247d2f
Summary:
Autocargo only allows 1 rust-library per Cargo.toml, but right now we have 3
per Thrift library so that doesn't work:
https://www.internalfb.com/intern/sandcastle/log/?instance_id=27021598231105145&step_id=27021602582167211&step_index=13&name=Run%20config
There's little benefit in Autocargo-ifying those rules anyway since they're of
use to Thrift servers and this doesn't work at all in our OSS builds, so let's
just see if we can just noop them. That'll make the crate not exist at all as a
dep, but even considering that it exists only to link to a C++ library that
Autocargo doesn'tk now how to build anyway, that seems OK?
drop-conflicts
Reviewed By: markbt
Differential Revision: D30304720
fbshipit-source-id: 047524985b2dadab8610267c05e3a1b3770e84e6
Summary: This adds types to FileChange thrift and rust structs to deal with additional possible snapshot states, that is, untracked and missing files. Conflicted stuff not added yet.
Reviewed By: StanislavGlebik
Differential Revision: D30103162
fbshipit-source-id: 59faa9e4af8dca907b1ec410b8af74985d85b837
Summary:
for now this changes:
```
struct FileChange {
...stuff
}
fn f(x: Option<FileChange>)
```
to
```
struct TrackedFileChange {
...stuff
}
enum FileChange {
TrackedChange(TrackedFileChange),
Deleted,
}
fn f(x: FileChange)
```
This makes it much clearer that `None` actually means the file was deleted. It will also be useful as in the next diff I will add more stuff inside FileChange (for untracked changes), and this refactor will make it easy.
(The refactor from using `Option` to putting it all inside the enum isn't really necessary, but IMO it looks much clearer, so I did it.)
Reviewed By: StanislavGlebik
Differential Revision: D30103454
fbshipit-source-id: afd2f29dc96baf9f3d069ad69bb3555387cff604
Summary: Previous diffs switched all our usage from chashmap to dashmap as dashmap upstream is more responsive. Now remove chashmap from the cargo vendoring.
Reviewed By: dtolnay
Differential Revision: D30046522
fbshipit-source-id: 111ef9375bd8095f8b7c95752ecbc1988fb0438d
Summary:
One of the things that megarepo_tool can do is to manually rewrite a commit
from one repo to another with a particular commit remapping version.
e.g.
```
source repo
X
|
P
target repo
X' <- rewritten X
|
A <- commit that exists only in target repo
|
P' <- rewritten P
```
Previously it always required manually setting the parents in the target repo
i.e. in the example above we'd need to tell that A is the new parent of
rewritten commit.
However this is not always convenient. Sometimes we just want megarepo_tool to take
parents in the source repo (i.e. P in the example above), remap them to large repo
(i.e. P' in the example above), and use P' as a target repo parent.
This diff adds a special option that lets us do so.
Reviewed By: farnz
Differential Revision: D30040016
fbshipit-source-id: 116dbe1803857053336ca76d0a65dbca8b14bd73
Summary:
The use of dyn traits of the Thrift-generated server traits was emitting future compatibility warnings with recent versions of rustc, due to a fixed soundness hole in the trait object system:
```
error: the trait `x_account_aggregator_if::server::XAccountAggregator` cannot be made into an object
|
= this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
note: for a trait to be "object safe" it needs to allow building a vtable to allow the call to be resolvable dynamically; for more information visit <https://doc.rust-lang.org/reference/items/traits.html#object-safety>
```
This diff pulls in https://github.com/dtolnay/async-trait/releases/tag/0.1.51 which results in the Thrift-generated server traits no longer hitting the problematic pattern.
Reviewed By: zertosh
Differential Revision: D29979939
fbshipit-source-id: 3e6e976181bfcf35ed453ae681baeb76a634ddda
Summary:
The patches to these crates have been upstreamed.
allow-large-files
Reviewed By: jsgf
Differential Revision: D29891894
fbshipit-source-id: a9f2ee0744752b689992b770fc66b6e66b3eda2b
Summary:
This is just updating the os_info crate to my fork with a fix for Centos
Stream: https://github.com/stanislav-tkach/os_info/pull/267
Reviewed By: quark-zju
Differential Revision: D29410043
fbshipit-source-id: 3642e704f5a056e75fee4421dc59020fde13ed5e
Summary: There is a regression in 1.7.0 (which we're on at the moment) so we might as well update.
Reviewed By: zertosh, farnz
Differential Revision: D29358047
fbshipit-source-id: 226393d79c165455d27f7a09b14b40c6a30d96d3
Summary:
Pull in a patch which fixes writing out an incorrect entsize for the
`SHT_GNU_versym` section:
ddbae72082
Reviewed By: igorsugak
Differential Revision: D29248208
fbshipit-source-id: 90bbaa179df79e817e3eaa846ecfef5c1236073a
Summary: Update versions for several of the crates we depend on.
Reviewed By: danobi
Differential Revision: D29165283
fbshipit-source-id: baaa9fa106b7dad000f93d2eefa95867ac46e5a1
Summary: revert the zstd crates back to previous version
Reviewed By: johansglock
Differential Revision: D29038514
fbshipit-source-id: 3cbc31203052034bca428441d5514557311b86ae
Summary:
Same as previous diff, but for a different command.
Building stuff through RepoFactory makes it easier to change how the build works *everywhere* later.
See D28877887 for goal
**This is the last diff of this type**. With this, I am able to delete `fetch_skiplist_index`, and now all skiplist building goes through factories.
Reviewed By: StanislavGlebik
Differential Revision: D28898839
fbshipit-source-id: af17d87b5890e09dbce317d52798bf78b14bc95c
Summary:
Same as previous diff, but for a different command.
Building stuff through RepoFactory makes it easier to change how the build works *everywhere* later.
See D28877887 for goal
Reviewed By: StanislavGlebik
Differential Revision: D28898653
fbshipit-source-id: 3db00cd99e50044d1acfa3d65065e5305280ec02
Summary:
Same as previous diff, but for a different command.
Building stuff through RepoFactory makes it easier to change how the build works *everywhere* later.
See D28877887 for goal
Reviewed By: StanislavGlebik
Differential Revision: D28898240
fbshipit-source-id: ede29b190c7bcf3f0a9c53c80149764a23cc4ef6
Summary:
Same as previous diff, but for a different command.
Building stuff through RepoFactory makes it easier to change how the build works *everywhere* later.
I changed from passing down a `BlobRepo` and a `LeastCommonAncestorsHint` to simply passing down an `InnerRepo`.
See D28877887 for goal
Reviewed By: StanislavGlebik
Differential Revision: D28897045
fbshipit-source-id: af546ae4702813bac2a0f938e7dec9d570c8cbf7
Summary:
The important change on this diff is in this file: `eden/mononoke/cmdlib/src/args/mod.rs`
On this diff I change that file's repo-building functions to be able to build both `BlobRepo` and `InnerRepo` (added on D28748221 (e4b6fd3751)). In fact, they are now able to build any facet container that can be built by the `RepoFactory` factory, so each binary can specify their own subset of needed "attributes" and only build those ones.
For now, they're all still using BlobRepo, this diff is only a refactor that enables easily changing the repo attributes you need.
The rest of the diff is mostly giving hints to the compiler, as in several places it couldn't infer it should use `BlobRepo` directly, so I had to add type hints.
## High level goal
This is part of the blobrepo refactoring effort.
I am also doing this in order to:
1. Make sure every place that builds `SkiplistIndex` uses `RepoFactory` for that.
2. Then add a `BlobstoreGetOps` trait for blobstores, and use the factory to feed it to skiplist index, so it can query the blobstore while skipping cache. (see [this thread](https://www.internalfb.com/diff/D28681737 (850a1a41b7)?dst_version_fbid=283910610084973&transaction_fbid=106742464866346))
Reviewed By: StanislavGlebik
Differential Revision: D28877887
fbshipit-source-id: b5e0093449aac734591a19d915b6459b1779360a
Summary: Update to latest version. This includes a patch to async-compression crate from [my PR updating it](https://github.com/Nemo157/async-compression/pull/125), I will remove once the crate is released.
Reviewed By: mitrandir77
Differential Revision: D28897019
fbshipit-source-id: 07c72f2880e7f8b85097837d084178c6625e77be
Summary:
When syncing merge commits with two parents it would be nice if it was the first parent that comes from the unified branch. In **case of octopus merges** we really don't want
the parent in unified branch to be third (that would turn the sync into
non-forward move!). Let's add a way to tell the commit rewriter which parent
needs to be first.
Reviewed By: farnz
Differential Revision: D28885488
fbshipit-source-id: 57a081ce2d285ba2b6d6d98110cd1c64a241548e
Summary:
Like it says in the title. The API between Bytes 1.x has changed a little bit,
but the concepts are basically the same, so we just need to change the
callsites that were calling `bytes()` and have them ask for `chunk()` instead.
This diff attempts to be as small as it can (and it's already quite big). I
didn't attempt to update *everything*: I only updated whatever was needed to
keep `common/rust/tools/scripts/check_all.sh` passing.
However, there are a few changes that fall out of this. I'll outline them here:
## `BufExt`
One little caveat is the `copy_to_bytes` we had on `BufExt`. This was
introduced into Bytes 1.x (under that name), but we can't use it here directly.
The reason we can't is because the instance we have is a `Cursor<Bytes>`, which
receives an implementation of `copy_from_bytes` via:
```
impl<T: AsRef<[u8]>> Buf for std::io::Cursor<T>
```
This means that implementation isn't capable of using the optimized
`Bytes::copy_from_bytes` which doesn't do a copy at all. So, instead, we need
to use a dedicated method on `Cursor<Bytes>`: `copy_or_reuse_bytes`.
## Calls to `Buf::to_bytes()`
This method is gone in Bytes 1.x, and replaced by the idiom
`x.copy_to_bytes(x.remaining())`, so I updated callsites of `to_bytes()`
accordingly.
## `fbthrift_ext`
This set of crates provides transports for Thrift calls that rely on Tokio 0.2
for I/O. Unfortunately, Tokio 0.2 uses Bytes 0.5, so that doesn't work well.
For now, I included a copy here (there was only one required, when reading from
the socket). This can be removed if we update the whole `fbthrift_ext` stack to
Bytes 1.x. fanzeyi had been wanting to update this to Tokio 1.x, but was blocked on `thrift/lib/rust` using Bytes 0.5, and confirmed that the overhead of a copy here is fine (besides, this code can now be updated to Tokio 1.x to remove the copy).
## Crates using both Bytes 0.5 & Bytes 1.x
This was mostly the case in Mononoke. That's no coincidence: this is why I'm
working on this. There, I had to make changes that consist of removing Bytes
0.5 to Bytes 1.x copies.
## Misuse of `Buf::bytes()`
Some places use `bytes()` when they probably mean to use `copy_to_bytes()`. For
now, I updated those to use `chunk()`, which keeps the behavior the same but
keeps the code buggy. I filed T91156115 to track fixing those (in all
likelihood I will file tasks for the relevant teams).
Reviewed By: dtolnay
Differential Revision: D28537964
fbshipit-source-id: ca42a614036bc3cb08b21a572166c4add72520ad
Summary:
Just as with rewrite_commit function that I moved a to commit_transformation
not so long ago (D28259214 (df340221a0)), let's also move copy_file_contents. The motivation
is because we are going to use it in the next diff for sync_changeset method.
Reviewed By: ikostia
Differential Revision: D28352402
fbshipit-source-id: 12288a51540c9793d988e4063735bcbc1c3b1a7f
Summary:
This is going to be use to rewrite (or transform) commits from source to
target. This diff does a few tihngs:
1) adds a MultiMover type and a function that produces a mover given a config. This is similar to Mover type we used for fbsource<-> ovrsource megarepo sync, though this time it can produce a few target paths for a given source path.
2) Moves `rewrite_commit` function from cross_repo_sync to megarepo_api, and make it work with MultiMover.
Reviewed By: ikostia
Differential Revision: D28259214
fbshipit-source-id: 16ba106dc0c65cb606df10c1a210578621c62367
Summary:
This is going to enable the background update in SegmentedChangelog to log
entries to Scuba.
The scuba sample builder is not fundamentally different than other elements of
the environment. It is used slightly differently to, for example, Logger,
because it has to cloned in all places that want to log rows but otherwise it
has the same characteristics.
Reviewed By: krallin
Differential Revision: D28210008
fbshipit-source-id: 68468868d13f29dddf21095bd7526cb4ff690786
Summary: Upstream crate has landed my PR for zstd 1.4.9 support and made a release, so can remove this patch now.
Reviewed By: ikostia
Differential Revision: D28221163
fbshipit-source-id: b95a6bee4f0c8d11f495dc17b2737c9ac9142b36
Summary:
We used to carry patches for Tokio 0.2 to add support for disabling Tokio coop
(which was necessary to make Mononoke work with it), but this was upstreamed
in Tokio 1.x (as a different implementation), so that's no longer needed. Nobody
else besides Mononoke was using this.
For Hyper we used to carry a patch with a bugfix. This was also fixed in Tokio
1.x-compatible versions of Hyper. There are still users of hyper-02 in fbcode.
However, this is only used for servers and only when accepting websocket
connections, and those users are just using Hyper as a HTTP client.
Reviewed By: farnz
Differential Revision: D28091331
fbshipit-source-id: de13b2452b654be6f3fa829404385e80a85c4420
Summary:
This used to be used by Mononoke, but we're now on Tokio 1.x and on
corresponding versions of Gotham so it's not needed anymore.
Reviewed By: farnz
Differential Revision: D28091091
fbshipit-source-id: a58bcb4ba52f3f5d2eeb77b68ee4055d80fbfce2
