Summary:
Fix the PathRelativizer APIs to accept `Path` and even `str` arguments instead
of just `PathBuf`. The old code required a `PathBuf`, which often forced
callers to make a copy of the path data.
Reviewed By: quark-zju
Differential Revision: D19958505
fbshipit-source-id: 6fa40dd4b75df4e3faf9ad2ae4f0e4e6595669f6
Summary:
This should give us a slightly better idea of what hosts are doing to
troubleshoot duplicate derivation.
Also, let's make the logging a bit less confusing.
Reviewed By: StanislavGlebik
Differential Revision: D20070619
fbshipit-source-id: 91cc264b7043b8fc8c21c007832fba328ef0017d
Summary:
This updates our multiplexed blobstore configuration to carry its own DB
config. The upshot of this change is that we can move the blobstore sync queue
(a fairly unruly table) to its own DB.
Another nice side effect of this is that it cleans up a bunch of other code, by
finally decoupling the blobstore config from the DB config. For examples,
places that need to instantiate a blobstore can now to do even without a DB
config (such as wireproto logging).
Obviously, this cannot land until we update the configs to include this. I'll
do so in Configerator prior to landing the diff.
Reviewed By: HarveyHunt
Differential Revision: D19973905
fbshipit-source-id: 79e4ff92cdb989aab4532decd3fe4fd6c55e2bb2
Summary:
I'd like to refactor our multiplex blob to store its DB using a different
shard. In preparation of doing so, let's:
- Extract parsing DB configs from storage configs
- Tidy up some related places that take a reference when they actually need
ownership (which is sort of wasteful).
Reviewed By: StanislavGlebik
Differential Revision: D19973906
fbshipit-source-id: 82baceb892e9e24e5fd0349ffa5503884c177a7a
Summary:
Most of EdenFS's main logging is done through folly::logging, however a number
of libraries that we use do logging through glog. Previously we set glog's
`--minloglevel` setting to `0`, and we use the default `--v=0` setting.
This enabled glog `VLOG` messages, only for at VLOG level `0` messages.
Now that the Rust backing store code can fetch directly from memcache this now
links in some additional memcache library code that has some `VLOG(0)`
messages that are logged fairly frequently. These aren't useful for us to
have in our logs, so reduce the `minloglevel` to `1` for now, which disables
all `VLOG` messages.
Reviewed By: genevievehelsel
Differential Revision: D20050589
fbshipit-source-id: 167e301d61e46ae3c19975e0c9233eda371495c0
Summary: Now it no longer depends on mononoke_types, we can build it with cargo
Reviewed By: krallin
Differential Revision: D20070438
fbshipit-source-id: 1b2f9cc3640c58fd38e962c7c738d08cbb22a71d
Summary:
The bytes 0.5 is a depencency of newer tokio, it's also newer, and thus better.
Staying on 0.4 means that copies between Bytes 0.4 and 0.5 need to be done,
this will be especially bad in the LFS code since 10+MB buffer will have to be
copied...
One main API change is for the configparser. The code used to take Into<Bytes>
for the keys, I switched it to AsRef<[u8]>.
For hg_memcache_client, an extra copy is performed to build a Delta, since this
code uses an old tokio, and is being replaced right now, the effort of
switching to a new tokio and new bytes was not deemed worth it, the copy will
do for now.
Reviewed By: dtolnay
Differential Revision: D20043137
fbshipit-source-id: 395bfc3749a3b1bdfea652262019ac6a086e61e0
Summary:
The SAI diag shell implementation requires Python3.7.6.
Previous diff in the stack added Python3.7.6 manifest.
Make use of it to build diag shell and link with Python lib.
Differential Revision: D20064315
fbshipit-source-id: c855f8a9f554066e8150ce1d867828a84161961b
Summary:
As titled.
FBOSS depends on Python3.7.6, subsequent diff in the stack adds Python3.7.6
depedency for FBOSS.
Differential Revision: D20064316
fbshipit-source-id: 27b328dc25e326f3927ea6cb003fb7bb45732d61
Summary:
This is the second (and last) step on removing RocksDB as a blobstore.
Check the task for more description.
Context for OSS:
> The issue with rocksblob (and to some extent sqlite) is that unless we
> introduce a blobstore tier/thift api (which is something I'm hoping to avoid
> for xdb blobstore) we'd have to combine all the mononoke function like hg,
> scs, LFS etc into one binary for it to have access to rocksdb, which would be
> quite a big difference to how we deploy internally
(Note: this ignores all push blocking failures!)
Reviewed By: farnz
Differential Revision: D20001261
fbshipit-source-id: c4b2b2a393b918d17680ad483aa1d77356f1d07c
Summary:
Add option to start the roots of the walk from any graph node, rather than just bookmarks.
This is useful when reproducing issues loading a key, validating a changeset/filenode etc, or to get consistent results on things like sizing where specifying root by bookmark would result in changes between runs.
Reviewed By: farnz
Differential Revision: D19886707
fbshipit-source-id: b7361cbec894aba08b6f702ff0731b9b201224d3
Summary:
Add `scsc export`. Analogous to `svn export`, this exports the contents of a
directory within a commit to files on disk, without a local checkout.
Reviewed By: mitrandir77
Differential Revision: D20006307
fbshipit-source-id: 5870712172cd8a030e85dbff75273c28ab0c332c
Summary: so that it can be shared more easily with AsyncUDPSocket
Reviewed By: yangchi
Differential Revision: D19851480
fbshipit-source-id: ec8cdb852519724db6f89cf70c4a4169de5028b6
Summary:
`treedirstatemap._repacked` is sometimes set in write(), but does not appear
to be used anywhere. Remove it. (I noticed this since Pyre complains about
it if you enable type checking for `write()`)
Reviewed By: xavierd
Differential Revision: D19958219
fbshipit-source-id: a55e237865160191d814ed950f69c3113bec4f64
Summary:
Add type annotations for the propertycache type.
Unfortunately at the moment Pyre still can't properly type check code that
uses this class, as it does not understand the special `__get__()` method.
It looks like support for this is hopefully coming in D19206575.
Reviewed By: xavierd
Differential Revision: D19958223
fbshipit-source-id: 0f8f15fc6935ec3feaef41d3be373a85225276fe
Summary:
Add type annotations for `dirstate.status()` and
`filesystem.pendingchanges()`
Unfortunately Pyre appears to choke when processing the `dirstate.status()`
function, and currently does not actually report type errors inside this
function at the moment. I've let the Pyre team know about this.
(If Pyre did work correctly it would report one issue since it doesn't realy
understand the `rootcache` decorator applied to `dirstate._ignore`)
Reviewed By: xavierd
Differential Revision: D19958226
fbshipit-source-id: a1cd4b9402a0a449481035cee819533c56b9b336
Summary:
This module previously used to handle deciding how a particular module should
be imported if it had multiple versions (e.g., pure Python or native).
However, as of D18819680 it was changed to always import the native C version.
Lets go ahead and remove it entirely now. Using `policy.importmod` simply
makes it harder for type checkers to figure out the actual module that will be
used.
The only functionality that `policy.importmod()` still provided was verifying
that the module contained a "version" field that looked like what was
expected. In practice these version numbers are not bumped often, so this
doesn't really seem to provide much value in checking that we imported the
correct version that we expected to be shipped with this release.
Reviewed By: xavierd
Differential Revision: D19958227
fbshipit-source-id: 05f1d027d0a41cf99c4aa93cb84a51e830305077
Summary:
Add *.pyi type stub files for most of the native C extensions.
This allows Pyre to type check functions that use these extensions.
These type annotations likely aren't complete, but contain enough information
to allow Pyre to pass cleanly on the existing type-checked locations in the
code using these modules.
Reviewed By: xavierd
Differential Revision: D19958220
fbshipit-source-id: 85dc39a16e595595a174a8e59e419c418d3531be
Summary:
This moves the build rules for the extensions in mercurial/cext into a TARGETS
file in this directory.
This will allow us to start writing `*.pyi` files that contain type
information for these modules, and store them alongside the corresponding `.c`
files. By having the build rules in the top-level `eden/scm` directory we
would have needed to keep the `.pyi` files for these modules directly in the
`eden/scm` directory instead, as the namespace for the `pyi` files is assumed
to be the basemodule plus their path relative to the TARGETS file.
Reviewed By: xavierd
Differential Revision: D19958222
fbshipit-source-id: fdc26ead16663036ffa2562a96eb1649f91cba81
Summary:
Mercurial wishes to use this crate, but pulling in mononoke_types brings way
too many dependencies. Since the only reason mononoke_types is brought in is
for the Sha256 type, let's just hardcode it to [u8; 32].
Reviewed By: krallin
Differential Revision: D20003596
fbshipit-source-id: 53434143c61cd1a1275027200e1149040d30beae
Summary:
The last diff fixed this for fsmonitor. Let's skip these same paths for
non-fsmonitor.
Reviewed By: quark-zju
Differential Revision: D20014808
fbshipit-source-id: 02e3cd9aa29d9c024ba3e8e42a46e21a7c8dfc30
Summary:
This hook was implemented to prevent incorrect users from moving a
bookmark. However, it doesn't work and the functionality is now implemented by
`is_allowed_user` in the pushrebase pipeline.
Remove the unused hook.
Reviewed By: johansglock
Differential Revision: D20030479
fbshipit-source-id: bcbc9508eebe77cffbc7936382ba4d345b76f74f
Summary:
Watchman may report invalid utf-8 filenames, even after they've been
deleted. Let's skip them, and print a warning.
Reviewed By: sfilipco
Differential Revision: D20012187
fbshipit-source-id: b13550918a8330ef3eb5c546105d1e054dcb7724
Summary:
Error strings were being converted to unicode if they contained certain
characters. This caused python 2 Mercurial to throw various errors when it tried
to turn them into strings to report errors.
Let's return cpython_ext::Str instead of String.
Reviewed By: sfilipco
Differential Revision: D20012188
fbshipit-source-id: af6fa7d98d68e3c188292e4972cfc1bdb758dbdf
Summary:
We're working towards sharding Bonsais. Let's make them easier to cache by also
not allowing arbitrarily large commit messages.
Reviewed By: StanislavGlebik
Differential Revision: D20002994
fbshipit-source-id: b2319ac9d5709e968121d4299396e03a90df4a06
Summary:
Let's populate the bonsai<->git mapping on pushrebase of the commits that are
coming from git. By this being a pushrebase hook we can have the accuare mappings
being available as soon as the bonsai commit is available.
Corresponding configerator change: D19951607
Reviewed By: krallin
Differential Revision: D19949472
fbshipit-source-id: b957cbcdd0f14450ceb090539814952db9872576
Summary: During the pushrebase hook phase we'll need to reuse existing transaction.
Reviewed By: krallin
Differential Revision: D19949473
fbshipit-source-id: 7c53308724bec6df6d40933405f703c86be15a7a
Summary:
By having it in blobrepo we can ensure that all parts of mononoke can access it
easily
Reviewed By: StanislavGlebik
Differential Revision: D19949474
fbshipit-source-id: ac3831d61177c4ef0ad7db248f2a0cc5edb933b1
Summary:
We need a table to store git<->bonsai mappings and a crate that would abrstract operations on it:
* it's going to be useful immediately to store git hashes for configerator
commits and doing the hash translations via SCS.
* it's going to be useful further down the line for real git support.
NOTE: I'm explicitly using the name `SHA1` all over the place to minimize the
confusion if we'll ever want to support other hashing schemes for git commits.
(Git Community is working on SHA256 support nowdays).
The corresponding AOSC diff: D19835975
Reviewed By: krallin
Differential Revision: D19835974
fbshipit-source-id: 113640f4db9681b060892a8cedd93092799ab732
