Summary:
`eden du --clean` currently fails with
```
Fatal Python error: initfsencoding: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'
```
Full error: P149352812
It looks like this is because Buck expects to run with a different python, so
here I clear out the PYTHONHOME variable before we start buck.
This reuses out logic used elsewhere to clean up the environment before
calling buck.
Reviewed By: wez
Differential Revision: D24904105
fbshipit-source-id: 73587c52aff3ea00f68339eb44e7042329de2e44
Summary: `lru-disk-cache` depends on an old version of `linked-hash-map` which contains UB in 1.48 (see https://github.com/mozilla/sccache/issues/813). They updated the deps in their repo months ago, but haven't pushed a new version. This diff makes us get `lru-disk-cache` directly from their GitHub instead.
Reviewed By: dtolnay
Differential Revision: D25134582
fbshipit-source-id: 05fd63a76b7095ebeea458645b92a83bbd8c4614
Summary: The new library firmware_storage needs the yaml-cpp. For OSS build this diff adds the yaml-cpp library to manifest
Reviewed By: shri-khare
Differential Revision: D25115387
fbshipit-source-id: 9ed1c374df7e87f284c48d366095a20183719534
Summary:
It was mistakenly mapping deleted manifest id to a unode id.
I'm surprised the walk.rs code that constructs the DeletedManifestToDeletedManifestChild edge built with this bug. The DeletedManifestId and ManifestUnodeId types presumably coerce to each other.
Reviewed By: markbt
Differential Revision: D25120994
fbshipit-source-id: 1b53037808779c345c163ef32324961938078fc7
Summary:
Soon we are going to use hg sync job for configerator repos, and they might use
Push bookmark move. Let's allow it in sync job
Reviewed By: ikostia
Differential Revision: D25121176
fbshipit-source-id: f6000617b42be8392730ffc56be1947fbdbfff76
Summary:
This adds support for optionally not uploading commits we already have when
they arrive via infinitepush. This can happen if we're replaying bundles.
This works by filtering the commits we have. We still get some futures created
for the underlying uploads, but we never poll them because when we visit
what futures are needed for what commits, we don't select uploads that are
only reachable from commits we filtered out.
Obviously, this isn't super efficient, since the client still has to send us
all this data, but it's better than having them send us all that data then
having us take hours overwriting it all.
Reviewed By: mitrandir77
Differential Revision: D25120844
fbshipit-source-id: f96881d0d98ec622259bedb624fd94b600a2bf1d
Summary: Remove 'static requirement for async methods of Blobstore, propagate this change and fixup low hanging fruits where the code can become 'static free easily.
Reviewed By: ahornby, farnz
Differential Revision: D24839054
fbshipit-source-id: 5d5daa04c23c4c9ae902b669b0a71fe41ee6dee6
Summary:
It feels like invalidating the entry before the directory makes slightly more
sense, so do it in that order.
Reviewed By: chadaustin
Differential Revision: D24800817
fbshipit-source-id: ed053d07bbae6954c276d1ad7a1ff247e5c055d9
Summary:
It turns out the hggit tests weren't passing in Python 3, despite us
having removed them from our py3-broken list. Woops. This fixes them and enables
the tests.
Reviewed By: sfilipco
Differential Revision: D25095189
fbshipit-source-id: acffca34b0d5defa7575ede60621ca2ce0a2afe4
Summary:
Back in March we forced all extras to be strings. It turns out hggit
needs to write binary extras since it stores some binary patches in extras.
To support that, let's encode commit extras using surrogateescaped. That will
allow hggit to put binary data in extras in a later diff.
Reviewed By: sfilipco
Differential Revision: D25095190
fbshipit-source-id: 330bf06b15fc435f70119490557b8c22341b6ed9
Summary:
Apparently shards for some DBs start from 1 (production_filenodes) and for some - from 0 (production_blobstore).
This diff fixed the issue for mysql connections.
Long term we might want to query SMC for the list of shards instead of hardcoding different values in the different places.
Reviewed By: farnz
Differential Revision: D25057136
fbshipit-source-id: 9201a2ec8afe0b66a246a2ee91cc9389630f5ddf
Summary:
Add a TraceBus to HgQueuedBackingStore and allow tracing import events over Thrift.
This powers a new `eden trace hg` command that allows a live view of
tree and blob imports.
Reviewed By: genevievehelsel
Differential Revision: D24732059
fbshipit-source-id: 525152fe39047160a68c1706217a06a00a6dbef1
Summary:
We got a few ubns because one bookmark validator in a single region wasn't able
to connect to mysql and was reporting errors.
This diff fixes by separating logical and infra errors
Reviewed By: ikostia
Differential Revision: D25092364
fbshipit-source-id: 93f4be1a7e0467051b7b8d927eef9b4f5cd6a983
Summary:
This isn't actually being consulted anywhere save for a single test, so let's
just remove it (it's not like the test checks anything important — that field
might not as well exist given we never read it).
Reviewed By: farnz
Differential Revision: D25093494
fbshipit-source-id: 5f4a53f8666fc0e8a89ceade44baa96e71fb813f
Summary:
This is a bit unnecessary as it stands — we roundtrip the path through
execute() just to return it back. This path was used for trace logging, but
given we literally never look at this log, let's just simplify this logging a
little bit.
Reviewed By: StanislavGlebik
Differential Revision: D25089344
fbshipit-source-id: 15b0f1cce8c9b2938429de19ff063e5677794912
Summary:
With segmented changelog, rev can exceed f64 safe range
(Number.MAX_SAFE_INTEGER in Javascript, 9007199254740991, 0x1fffffffffffff).
If rev is used in JSON, then the JSON cannot be parsed with precise rev
information.
This diff adds a compatibility mode so template will map the out-of-range revs
to safe range, and the mapped revs can be detected, and mapped back to be
resolved correctly.
Reviewed By: sfilipco
Differential Revision: D25079001
fbshipit-source-id: 52a5a2c8345a76355b8bf81435034d90c7c31610
Summary: Tests the behaviour of collecting the raw bundles.
Reviewed By: krallin
Differential Revision: D25025255
fbshipit-source-id: 114da273a28d131f5dd24047ed28ea23d076f235
Summary:
The `getdesignatednodes` function returns a boolean indicating whether the requested nodes were actually fetched.
In the case of SSH, this is needed because the server may not support the `designatednodes` capability (only supported by Mononoke). If the fetch fails, Mercurial will fall back to an expensive complete tree fetch.
The HTTP version of this function accidentally omitted `return True`, which meant it implicitly returned `None`, which triggered the fallback path.
Reviewed By: sfilipco
Differential Revision: D25074067
fbshipit-source-id: 089d5382dd566db89ee732cdcb82762c8d43e21a
Summary:
Paths are not necessarily legal glob syntax. In particular, backslash
is used for escaping. This caused problems on Windows, where we tried
to pass a backslash-delimited path into `eden prefetch --no-prefetch`.
Reviewed By: xavierd
Differential Revision: D25072784
fbshipit-source-id: 9ce8e5ccc8f28581512c39d04922889da0bc1bf6
Summary: The test doesn't compile on Windows, let's just ifdef it.
Reviewed By: genevievehelsel
Differential Revision: D25033804
fbshipit-source-id: 4f312f010f9d0db42cc9ae19df3f668e8e1c4665
Summary:
Converting back and forth between folly::fs::path and AbsolutePath appears to
be problematic on Windows as NUL bytes appears in the paths, causing the tests
to fail. Instead of doing this conversion, let's simply use AbsolutePath everywhere.
Reviewed By: chadaustin
Differential Revision: D25033803
fbshipit-source-id: 6c45c2a20fc4bf18cecc838b219faacfeb8386d8
Summary:
Querying bookmarks in all repos at the exact same time results in us making a
bunch of concurrent queries to MySQL, which in turn results in MyRouter
allocating a bunch of connections.
This is for reporting the age of bookmarks: that stuff can wait... It doesn't
need to be super fast. Let's make it run on repos one at a time to avoid
allocating dozens of connections every 10 seconds (which is more often than
MyRouter seems to close them).
Reviewed By: ikostia
Differential Revision: D25057432
fbshipit-source-id: 8b65ef65752fc9762a26d835ac80f61573003dd7
Summary:
Skip the need to go through SSH/Mercurial servers. Directly connect to Mononoke
via TLS with X509 certificates.
Reviewed By: markbt
Differential Revision: D24756947
fbshipit-source-id: a782c894956d6a6be055eaa78287c040ceb57da7
Summary: Add a test case for walking hg data from non-public changeset
Reviewed By: ikostia
Differential Revision: D25023130
fbshipit-source-id: 34295f77926b32c77095f7c10d6daa8ef59d9550
Summary: Add command line option to exclude derived data types from blobimport so we can use it to create non-public commits without filenode data for tests
Differential Revision: D25023116
fbshipit-source-id: d8e5d6955f11cebec0de2075c22981bf6c6f4af3
Summary: Add DeletedManifest to walker types with edges to linked changeset and its sub-manifests so that they can be scrubbed.
Differential Revision: D25000319
fbshipit-source-id: f146e6132fde0fb13e630d315484cc2c0a964bdc
