Summary:
this relies on local changes to make it so cargo metadata ACTUALLY finds this
binary: https://github.com/tokio-rs/console/pull/146 is where I try to upstream
it
Reviewed By: jsgf
Differential Revision: D30944630
fbshipit-source-id: 5d34a32a042f83eff7e7ae7445e23badf10fffe3
Summary:
Allow the mononoke_api user to choose whether they want
oprerations to be sped up using WBC or not.
Reviewed By: yancouto
Differential Revision: D30958908
fbshipit-source-id: 038cf77735e7c655f6801d714762e316b6817df5
Summary:
Some crates like mononoke_api depend on warm bookmark cache to speed up the
bookmark operations. This prevents them from being used in cases requiring
quick and low overhead startup like CLIs.
This diff makes it possible to swap out the warm bookmark cache to a
implementation that doesn't cache anything. (See next diffs to see how it's
used in mononoke_api crate).
Reviewed By: yancouto
Differential Revision: D30958910
fbshipit-source-id: 4d09367217a66f59539b566e48c8d271b8cc8c8e
Summary:
This method was added before the more generic list method was added. Let's get
rid of it for simplicity and to discourage listing all the bookmarks.
Reviewed By: yancouto
Differential Revision: D30958911
fbshipit-source-id: f4518da3f34591c313657161f69af96d15482e6c
Summary:
0.4.24 is incompatible with crates that use `deny(warnings)` on a compiler 1.55.0 or newer.
Example error:
```
error: unused borrow that must be used
--> common/rust/shed/futures_ext/src/stream/return_remainder.rs:22:1
|
22 | #[pin_project]
| ^^^^^^^^^^^^^^
|
= note: this error originates in the derive macro `::pin_project::__private::__PinProjectInternalDerive` (in Nightly builds, run with -Z macro-backtrace for more info)
```
The release notes for 0.4.28 call out this issue. https://github.com/taiki-e/pin-project/releases/tag/v0.4.28
Reviewed By: krallin
Differential Revision: D30858380
fbshipit-source-id: 98e98bcb5a6b795b93ed1efd706a1711f15c57db
Summary:
Some time ago (see D25910464 (fca761e153)) we've started using Background session class
while deriving data. This was done to avoid overloading blobstore sync queue - if Background
session class is set then multiplex blobstore waits for all blobstores to
finish instead of writing to the blobstore sync queue right away. However if any of the
blobstores fails then we start writing to the blobstore sync queue. In theory it should have avoided the problem of overloading blobstore sync queue while having the same multiplex reliability (i.e. if only a single blobstore fails the whole multiplex put doesn't fail)
Unfortunately there was a flaw - if blobstore put of a single blobstore wasn't
failing but was just too slow, then the whole multiplexed put operation becomes
too slow. This diff fixes this flaw by adding a timeout - if multiplexed put is
taking too long then we fallback to writing entries to the blobstore sync
queue.
Note that I added a new session class - BackgroundUnlessTooSlow -
because I figured that in some cases we are ok with waiting a long time but not
writing to the sync queue. Skiplist builder might be a good example of that -
since it's doing overwrites then we don't want to write to the blobstore sync
queue at all, because healer doesn't process overwrites correctly.
Reviewed By: farnz
Differential Revision: D30892377
fbshipit-source-id: 69ac1795002b124e11daac13d8bfe59895191168
Summary:
Add impls for Layer for Box/Arc<L: Layer> and <dyn Layer>. Also a pile of other
updates in git which haven't been published to crates.io yet, including proper
level filtering of trace events being fed into log.
Reviewed By: dtolnay
Differential Revision: D30829927
fbshipit-source-id: c01c9369222df2af663e8f8bf59ea78ee12f7866
Summary: Implement serverside graph endpoint for fetching the mapping of commits to commit parents for the missing segment of a commit graph. This implementation uses the find_commits_to_send method from the get_bundle_response library. What may be missing from pull and the old bundle protocol now is mutation markers.
Reviewed By: yancouto
Differential Revision: D30485672
fbshipit-source-id: ba3a30d9e482d60831cbe7a8e89f20dab947d9a1
Summary:
Since the find_commits_to_send method was added, common is already a
hashset not a vector, so it doesn't needed to be converted to a hashset.
Reviewed By: quark-zju
Differential Revision: D30622028
fbshipit-source-id: e5d1b6c60115d13c906b25142043652ba9e89d70
Summary:
Like it says in the title, this updates us to use Daemonize 0.5, though from
Github and not Crates.io, because it hasn't been released to the latter yet.
The main motivation here is to pull in
https://github.com/knsd/daemonize/pull/39 to avoid leaking PID files to
children of the daemon.
This required some changes in `hphp/hack/src/facebook/hh_decl` and `xplat/rust/mobium` since the way to
run code after daemonization has changed (and became more flexible).
Reviewed By: ndmitchell
Differential Revision: D30694946
fbshipit-source-id: d99768febe449d7a079feec78ab8826d0e29f1ef
Summary:
Manual component version update
Bump Schedule: https://www.internalfb.com/intern/msdk/bump/?schedule_fbid=342556550408072
Package: https://www.internalfb.com/intern/msdk/package/181247287328949/
Oncall Team: rust_foundation
NOTE: This build is expected to expire at 2022/09/01 09:14AM PDT
---------
New project source changes since last bump based on D30663071 (08e362a355e0a64a503f5073f57f927394696b8c at 2021/08/31 03:47AM -05):
| 2021/08/31 04:41AM -05 | generatedunixname89002005294178 | D30665384 | [MSDK] Update autocargo component on FBS:master |
| 2021/08/31 07:14PM PDT | kavoor | D30681642 | [autocargo] Make cxx-build match version of cxx |
| 2021/09/01 04:05PM BST | krallin | D30698095 | autocargo: include generated comment in OSS manifests |
---------
build-break (bot commits are not reviewed by a human)
Reviewed By: farnz
Differential Revision: D30717040
fbshipit-source-id: 2c1d09f0d51b6ff2e2636496cf22bcf781f22889
Summary:
The mockall crate's `automock` attribute previously created nondeterministic output, which leads to frequent random "Found possibly newer version of crate" failures in Buck builds that involve cache.
The affected trait in Conveyor is:
https://www.internalfb.com/code/fbsource/[4753807291f7275a061d67cead04ea12e7b38ae2]/fbcode/conveyor/common/just_knobs/src/lib.rs?lines=13-23
which has a method with two lifetime parameters. Mockall's generated code shuffled them in random order due to emitting the lifetimes in HashSet order. The generated code would randomly contain one of these two types:
`Box<dyn for<'b, 'a> FnMut(&str, Option<&'a str>, Option<&'b str>) -> Result<bool> + Send>`
`Box<dyn for<'a, 'b> FnMut(&str, Option<&'a str>, Option<&'b str>) -> Result<bool> + Send>`
Reviewed By: jsgf
Differential Revision: D30656936
fbshipit-source-id: c1a251774333d7a4001a7492c1995efd84ff22e5
Summary:
this library is a more general version of the `panic_unpack` lib I
made in fbcode. I made this library, its mit-apache licensed
Reviewed By: dtolnay
Differential Revision: D30607308
fbshipit-source-id: ee4fad3924fdae021753055cd3fd88c99cb99512
Summary:
This config option was used to slowly roll out LFS for a repo.
However, it is no longer used and can therefore be removed.
Reviewed By: StanislavGlebik
Differential Revision: D30511880
fbshipit-source-id: 59fe5925cc203aa609488fdf8ea29e9ff65ee862
Summary:
If hg sync job is pushing a commit, then we have no choice but to accept it,
even if it's too big. So let's not fail if too many commits are pushed.
Reviewed By: ahornby
Differential Revision: D30535025
fbshipit-source-id: eb607a8fbd691d6591ad990e0920411b1ad2f09c
Summary:
The diff is giant, but it's just a one-line change to add the
nested-values feature to slog, we just have a whole bunch of projects dependent
on slog.
Reviewed By: dtolnay
Differential Revision: D30351289
fbshipit-source-id: b6c1c896b06cbdf23b1f92c0aac9a97aa116085d
Summary:
We currently do repo lock checks in a loop during unbundle. However we now do a repo lock check in the bookmarks_movement::PushrebaseOntoBookmarkOp::run(), making the loop and check in repo_client unbundle redundant
Cons: It will no longer early terminate. Pros: database load should be reduced.
Reviewed By: StanislavGlebik
Differential Revision: D30331806
fbshipit-source-id: 16ee72e570184c20ac08d3fa6d8f9f333c91deb7
Summary:
Autocargo only allows 1 rust-library per Cargo.toml, but right now we have 3
per Thrift library so that doesn't work:
https://www.internalfb.com/intern/sandcastle/log/?instance_id=27021598231105145&step_id=27021602582167211&step_index=13&name=Run%20config
There's little benefit in Autocargo-ifying those rules anyway since they're of
use to Thrift servers and this doesn't work at all in our OSS builds, so let's
just see if we can just noop them. That'll make the crate not exist at all as a
dep, but even considering that it exists only to link to a C++ library that
Autocargo doesn'tk now how to build anyway, that seems OK?
drop-conflicts
Reviewed By: markbt
Differential Revision: D30304720
fbshipit-source-id: 047524985b2dadab8610267c05e3a1b3770e84e6
Summary:
for now this changes:
```
struct FileChange {
...stuff
}
fn f(x: Option<FileChange>)
```
to
```
struct TrackedFileChange {
...stuff
}
enum FileChange {
TrackedChange(TrackedFileChange),
Deleted,
}
fn f(x: FileChange)
```
This makes it much clearer that `None` actually means the file was deleted. It will also be useful as in the next diff I will add more stuff inside FileChange (for untracked changes), and this refactor will make it easy.
(The refactor from using `Option` to putting it all inside the enum isn't really necessary, but IMO it looks much clearer, so I did it.)
Reviewed By: StanislavGlebik
Differential Revision: D30103454
fbshipit-source-id: afd2f29dc96baf9f3d069ad69bb3555387cff604
Summary:
The time between logging total_unbundles and the actual successes is the time of the unbundle operation, which may be long.
This makes the alarm on D30222804 much less accurate, as success and fail for the same operation might fall in different buckets.
This diff changes two things:
- total_unbundles are logged at the end of the unbundle operation, which should make tracking more accurate when compared against successes.
- resolver_error is now logged in more cases that would previously error but not be logged
I created a wrapper function in order to make sure it always logs, as before it could not log if there were some early errors.
Differential Revision: D30248117
fbshipit-source-id: 4ec0c148dd7aa818b6d204fafecacacf4d267be7
Summary: Previous diffs switched all our usage from chashmap to dashmap as dashmap upstream is more responsive. Now remove chashmap from the cargo vendoring.
Reviewed By: dtolnay
Differential Revision: D30046522
fbshipit-source-id: 111ef9375bd8095f8b7c95752ecbc1988fb0438d
Summary:
Let's log repo name since it's clearer for people than repo ids. And in my mind logging
repo ids was a mistake - repo id is an implementation detail (we use repo ids
because they are more efficient to store in xdb table than strings), and
Mononoke users shouldn't need to care about repo ids. So let's start loggin
repo names.
Reviewed By: krallin
Differential Revision: D30040409
fbshipit-source-id: 71c2794d8122e616850662cda27c8092d382de7a
Summary:
The use of dyn traits of the Thrift-generated server traits was emitting future compatibility warnings with recent versions of rustc, due to a fixed soundness hole in the trait object system:
```
error: the trait `x_account_aggregator_if::server::XAccountAggregator` cannot be made into an object
|
= this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
note: for a trait to be "object safe" it needs to allow building a vtable to allow the call to be resolvable dynamically; for more information visit <https://doc.rust-lang.org/reference/items/traits.html#object-safety>
```
This diff pulls in https://github.com/dtolnay/async-trait/releases/tag/0.1.51 which results in the Thrift-generated server traits no longer hitting the problematic pattern.
Reviewed By: zertosh
Differential Revision: D29979939
fbshipit-source-id: 3e6e976181bfcf35ed453ae681baeb76a634ddda
Summary:
The patches to these crates have been upstreamed.
allow-large-files
Reviewed By: jsgf
Differential Revision: D29891894
fbshipit-source-id: a9f2ee0744752b689992b770fc66b6e66b3eda2b
Summary:
implement uploading of hg changesets
For now, reuse the upload code path from unbundle but calling it with empty filenodes and manifests.
Those are used for parents validation but this is not needed for us because we load trees and filenodes and their parents to construct the bonsai cs.
We might want to rewrite it to a cleaner code later and separate from unbundle but for now reusing the fucntion is the easiest way because we know the implementation is correct and also has logging.
Reviewed By: markbt
Differential Revision: D29644798
fbshipit-source-id: 27217d3061ab8d9712417facdbfbbc7e3aebfc5b
Summary:
Update Mononoke server to use the new `rate_limiting` crate. This diff
also removes the old rate limiting library.
Differential Revision: D29396507
fbshipit-source-id: 05adb9322705b771a739c8bcaf2816c95218a42d
Summary:
The rate limits for commits support the ability to apply a specific
prefix to the load limiting category. However, we haven't used this
functionality. Remove it to make subsequent work on the rate limiting logic
easier to implement.
Reviewed By: StanislavGlebik
Differential Revision: D29396506
fbshipit-source-id: ac518ccd74f6fac49ab85f87f1500787b5db955e
Summary:
This is just updating the os_info crate to my fork with a fix for Centos
Stream: https://github.com/stanislav-tkach/os_info/pull/267
Reviewed By: quark-zju
Differential Revision: D29410043
fbshipit-source-id: 3642e704f5a056e75fee4421dc59020fde13ed5e
Summary: There is a regression in 1.7.0 (which we're on at the moment) so we might as well update.
Reviewed By: zertosh, farnz
Differential Revision: D29358047
fbshipit-source-id: 226393d79c165455d27f7a09b14b40c6a30d96d3
Summary:
Pull in a patch which fixes writing out an incorrect entsize for the
`SHT_GNU_versym` section:
ddbae72082
Reviewed By: igorsugak
Differential Revision: D29248208
fbshipit-source-id: 90bbaa179df79e817e3eaa846ecfef5c1236073a
Summary: Update versions for several of the crates we depend on.
Reviewed By: danobi
Differential Revision: D29165283
fbshipit-source-id: baaa9fa106b7dad000f93d2eefa95867ac46e5a1
Summary: revert the zstd crates back to previous version
Reviewed By: johansglock
Differential Revision: D29038514
fbshipit-source-id: 3cbc31203052034bca428441d5514557311b86ae
Summary:
BlobRepo is already easily cloneable (actually holds an arc inside).
This should also make code a little prettier, where we needed to use `(*blob_repo).clone()` instead of just `blob_repo.clone()`.
Reviewed By: StanislavGlebik
Differential Revision: D28930384
fbshipit-source-id: 59f95d10576a3f71808d0d26d36358421673351e
Summary: Update to latest version. This includes a patch to async-compression crate from [my PR updating it](https://github.com/Nemo157/async-compression/pull/125), I will remove once the crate is released.
Reviewed By: mitrandir77
Differential Revision: D28897019
fbshipit-source-id: 07c72f2880e7f8b85097837d084178c6625e77be
Summary:
Time 0.2 is current, and 0.1 is long obsolete. Unfortunately there's a
large 0.1 -> 0.2 API change, so I preserved 0.1 and updated the targets of its
users. Also unfortunate that `chrono` has `oldtime` as a default feature, which
makes it use `time-0.1`'s `Duration` type. Excluding it from the features
doesn't help because every other user is specifying it by default.
Reviewed By: dtolnay
Differential Revision: D28854148
fbshipit-source-id: 0c41ac6b998dfbdcddc85a22178aadb05e2b2f2b
Summary:
Currently bookmark warmers receives a `BlobRepo`. This diff makes it receive an `InnerRepo`, but does no logic changes.
This will be useful as fields are moved from `BlobRepo` to `InnerRepo`, and will also be useful for accessing skiplists from warmers, as I plan to do on the next diff.
Reviewed By: StanislavGlebik
Differential Revision: D28796543
fbshipit-source-id: dbe5bec9fc34da3ae51e645ea09b03e2bb620445
Summary:
Like it says in the title. The API between Bytes 1.x has changed a little bit,
but the concepts are basically the same, so we just need to change the
callsites that were calling `bytes()` and have them ask for `chunk()` instead.
This diff attempts to be as small as it can (and it's already quite big). I
didn't attempt to update *everything*: I only updated whatever was needed to
keep `common/rust/tools/scripts/check_all.sh` passing.
However, there are a few changes that fall out of this. I'll outline them here:
## `BufExt`
One little caveat is the `copy_to_bytes` we had on `BufExt`. This was
introduced into Bytes 1.x (under that name), but we can't use it here directly.
The reason we can't is because the instance we have is a `Cursor<Bytes>`, which
receives an implementation of `copy_from_bytes` via:
```
impl<T: AsRef<[u8]>> Buf for std::io::Cursor<T>
```
This means that implementation isn't capable of using the optimized
`Bytes::copy_from_bytes` which doesn't do a copy at all. So, instead, we need
to use a dedicated method on `Cursor<Bytes>`: `copy_or_reuse_bytes`.
## Calls to `Buf::to_bytes()`
This method is gone in Bytes 1.x, and replaced by the idiom
`x.copy_to_bytes(x.remaining())`, so I updated callsites of `to_bytes()`
accordingly.
## `fbthrift_ext`
This set of crates provides transports for Thrift calls that rely on Tokio 0.2
for I/O. Unfortunately, Tokio 0.2 uses Bytes 0.5, so that doesn't work well.
For now, I included a copy here (there was only one required, when reading from
the socket). This can be removed if we update the whole `fbthrift_ext` stack to
Bytes 1.x. fanzeyi had been wanting to update this to Tokio 1.x, but was blocked on `thrift/lib/rust` using Bytes 0.5, and confirmed that the overhead of a copy here is fine (besides, this code can now be updated to Tokio 1.x to remove the copy).
## Crates using both Bytes 0.5 & Bytes 1.x
This was mostly the case in Mononoke. That's no coincidence: this is why I'm
working on this. There, I had to make changes that consist of removing Bytes
0.5 to Bytes 1.x copies.
## Misuse of `Buf::bytes()`
Some places use `bytes()` when they probably mean to use `copy_to_bytes()`. For
now, I updated those to use `chunk()`, which keeps the behavior the same but
keeps the code buggy. I filed T91156115 to track fixing those (in all
likelihood I will file tasks for the relevant teams).
Reviewed By: dtolnay
Differential Revision: D28537964
fbshipit-source-id: ca42a614036bc3cb08b21a572166c4add72520ad
