Commit Graph

65773 Commits

Author SHA1 Message Date
Xavier Deguillard
9621d533f4 nfs: use makeImmediateFutureWith instead of a manual version of it
Summary:
Now that makeImmediateFutureWith exists, we can simply use it instead of
constructing a ImmediateFuture<folly::Unit> and calling thenValue on it.

Reviewed By: chadaustin

Differential Revision: D28518059

fbshipit-source-id: 0041cf863fb32efab274f11c77c76109ca9b454f
2021-05-25 15:16:59 -07:00
Yipu Miao
814c4c2bcf Fix unused parameters on EdenServiceHandler
Summary: There are some unused warning on Windows, should be an easy fix.

Reviewed By: xavierd

Differential Revision: D28665465

fbshipit-source-id: 9281de7fd62f38e09d91435bb53c819bb98fb4ec
2021-05-25 10:30:29 -07:00
Yipu Miao
eb8deb4f51 Fix unused parameters on inodes on Windows
Summary: There are some unused warning on Windows, should be an easy fix.

Reviewed By: xavierd

Differential Revision: D28595085

fbshipit-source-id: abc03d210b2e9c5aa19a8925be6d4c426311e826
2021-05-25 10:30:29 -07:00
Yipu Miao
8f05086834 Fix unused parameters on StartupLogger.cpp on Windows
Summary: There are some unused warning on Windows, should be an easy fix.

Reviewed By: xavierd

Differential Revision: D28594404

fbshipit-source-id: f3dec92403739d67df3ecd091f2d8283a11ea0db
2021-05-25 10:30:29 -07:00
Xavier Deguillard
5a84f7ac85 utils: add an makeImmediateFutureWith function
Summary: This will be used to replace calls to folly::makeFutureWith.

Reviewed By: chadaustin

Differential Revision: D28515786

fbshipit-source-id: 2c2c542392e8e57b8f865173d6878cb9d00ba376
2021-05-25 09:45:47 -07:00
Mateusz Kwapich
7f0144d872 add claimed_by to requests table
Summary:
For debugging and better error messages it would be nice to know who worked on
a given request.

I'll do AOSC schema change once accepted.

Reviewed By: krallin

Differential Revision: D28441673

fbshipit-source-id: ba146d7f43dde26d9433f76af7fe982da14b5b82
2021-05-25 09:11:43 -07:00
Mateusz Kwapich
fb901786da add repo and bookmark to the async requests table
Summary:
Those will be used for two purposes:
 * to limit a scope of given tailer to just given repo (this way we could use
   different tailer binaries for different repos or disable processing for a
   single repo etc...)
 * to enforce a single in-flight request per repo (to prevent client from
   accidentally scheduling duplicate requests etc...)

I'll do AOSC schema change once accepted.

Reviewed By: krallin

Differential Revision: D28441670

fbshipit-source-id: 7b35a1c7034707d7cf54220c559edd6e03f430c3
2021-05-25 09:11:43 -07:00
Mateusz Kwapich
be32915b9b move the queue to its own module
Summary: I want to put more things in the async_requests crate.

Reviewed By: krallin

Differential Revision: D28441671

fbshipit-source-id: 19233c2c5b697cc1e27107cd9904666baf8f10b7
2021-05-25 09:11:43 -07:00
Alex Hornby
405b1e9f44 mononoke: fix warnings in fastlog
Summary: Fix these rustc warnings.

Reviewed By: farnz

Differential Revision: D28645895

fbshipit-source-id: 1d23c8e115fe43056f2ad2e3fe935e182c596eff
2021-05-25 01:56:35 -07:00
Jan Mazur
0e960695e0 put command renewing certificate in the error message
Summary: I have modified the places where most of the errors were raised that users reported and were resolved by renewal of certificates.

Reviewed By: krallin

Differential Revision: D28568561

fbshipit-source-id: 44fb127a49bde83efee1c934e0435b31f8602a8d
2021-05-25 01:05:16 -07:00
Zeyi (Rice) Fan
a8e007bbf6 hg: promote batch size to a configuration option
Summary: This diff removes the gflag based batch size option and promote it into a EdenFS Configuration so we can experiment with different batch size easily.

Reviewed By: chadaustin

Differential Revision: D28555280

fbshipit-source-id: 6d3a7be3cd880f0aaa3f427c0328222efa2d37ea
2021-05-24 19:17:17 -07:00
svcscm svcscm
bed0fa7c34 Updating submodules
Summary:
GitHub commits:

97183ba540
ec71afa895
a41db66f3d
43778e064b
a492ec64dc
6097866e85

Reviewed By: jurajh-fb

fbshipit-source-id: bcd13864c76bcc69531ce7001b0f7eac778f8826
2021-05-24 18:18:08 -07:00
svcscm svcscm
51763a1222 Updating submodules
Summary:
GitHub commits:

34e9fb6203
6779b8eeed
28a0b74ee5
d3bd20cffe
33d84df82b
4648a5658a
65c80c3f81
8c605d06c7
7a307c4d48

Reviewed By: jurajh-fb

fbshipit-source-id: be57dc2f0823a83faea98e4deafa6af2e5a6ca45
2021-05-24 17:58:10 -07:00
svcscm svcscm
518a268dcf Updating submodules
Summary:
GitHub commits:

3d9d855634
48cd1a65ce
18824d9355
56eb6ea2cd
50379d06bb
c0a6638054
0cfe8059bd
6c39bb2a45
2f6ef4b442
ca2f2f4ad3
9555348ea3

Reviewed By: jurajh-fb

fbshipit-source-id: 047184a0d5b7e9b9a21b328e63f08c638b7708c1
2021-05-24 17:32:58 -07:00
svcscm svcscm
ab6a87af8c Updating submodules
Summary:
GitHub commits:

0b4f440ab1
15ddc84722
60113595a2
99fbca1df1
52bbcd9764
cabd44f885
d4e3ee3ebe
8924d1a170
5253cd03e8
d4c2807cbf

Reviewed By: jurajh-fb

fbshipit-source-id: 81b999feac001d51a308e56d30e8e28eebdc7f84
2021-05-24 17:02:58 -07:00
Chad Austin
e44b403d53 whereami: always read the dirstate, even in EdenFS checkouts
Summary:
The telemetry wrapper didn't validate the SNAPSHOT header, which makes
migrating to a new format harder. Fortunately, it doesn't even need to
read the SNAPSHOT file. The dirstate file is maintained even in EdenFS
checkouts.

Reviewed By: quark-zju

Differential Revision: D28650333

fbshipit-source-id: 174cf7039adcbb28224ec528c2462e0a9232b6cd
2021-05-24 16:52:32 -07:00
Jun Wu
fc9912dbfe test-metalog-migration: remove the test
Summary: Upcoming changes will force enable metalog so there will be no way to migrate down.

Reviewed By: DurhamG

Differential Revision: D28595290

fbshipit-source-id: a130b3c60c5b553d024868f28a28e48c50d44783
2021-05-24 16:40:30 -07:00
svcscm svcscm
35d438b393 Updating submodules
Summary:
GitHub commits:

c94ee29a43
dfdd00185e
be6a88da90
aeb713e65b
189b730c8e
948feeb231
f3c6a63a45
cd6df76914
1147ebd31b
7ca015fd37
3223115dd0

Reviewed By: jurajh-fb

fbshipit-source-id: 89e9fcf1a2f8952e49bd26f5c58c6e7b1d3c7a0d
2021-05-24 16:22:44 -07:00
svcscm svcscm
2b63409a20 Updating submodules
Summary:
GitHub commits:

544e94eb5c
e5bc1bd7df
9551a4f7d1
85d4e767f2
b2fd85fe23
cd3ce74e52
a6394f44cd
41008f1515
8e81393d5b
a291f5e49a
2b5c3b88d5

Reviewed By: jurajh-fb

fbshipit-source-id: 7d11ededf543aba25cf1362bb5d942308511a673
2021-05-24 15:51:31 -07:00
Yipu Miao
c4fe54e1d5 Bump the waiting time for DaemonStartupLoggerTest.daemonClosesStandardFileDescriptors
Reviewed By: chadaustin

Differential Revision: D28499240

fbshipit-source-id: d04fc25ab3a823620c72c1355f897a274703efb5
2021-05-24 15:40:49 -07:00
Jun Wu
a586d3f636 test-encodedstore-long: remove the test
Summary:
It was added by D8527475 (72c3d8afc1) to workaround hgsql with no-fncache and long file
names synced from svn. Upcoming changes will force fncache to simplify
configuration and the hgsql server code was forked. So let's just delete
the test.

Reviewed By: DurhamG

Differential Revision: D28595291

fbshipit-source-id: 60d2449cca7af46b8b5b3c3b557a36507ff1576e
2021-05-24 15:24:18 -07:00
svcscm svcscm
2abfbc102d Updating submodules
Summary:
GitHub commits:

cc3fee7300
cd9426de42
f2e8440cd8
0c57185fb3
0eb4d6e578

Reviewed By: jurajh-fb

fbshipit-source-id: fa2e386bf208b250370115a74d6533135f2ba7e1
2021-05-24 15:10:09 -07:00
Jun Wu
2aa432e134 clone: add a config to clone with lazy changelog
Summary: This will be used by fbclone to ship lazy commit hash backend.

Reviewed By: DurhamG

Differential Revision: D28554445

fbshipit-source-id: a263ae7683124b3b86f4025b02c7de20dcb9813e
2021-05-24 14:51:58 -07:00
svcscm svcscm
8d48eda885 Updating submodules
Summary:
GitHub commits:

ed8c2e1426
8c26adf835
4bd5fcf6b4
992088f6ae
c95c2ca5c8
4de2bace72

Reviewed By: jurajh-fb

fbshipit-source-id: 9fce4169ed57d7afe5739118694821b928e3701d
2021-05-24 14:48:24 -07:00
svcscm svcscm
8741831163 Updating submodules
Summary:
GitHub commits:

a952a90c7e
d1f3264b57
43739b0607
78a7cede01
b73e0709f3
30f1dffd7c
8e37682d7e
6934d09079
9d25b6f956

Reviewed By: jurajh-fb

fbshipit-source-id: f0077cff9612090e74a31a1148fe6d7d24ef9024
2021-05-24 14:29:07 -07:00
svcscm svcscm
f5088de610 Updating submodules
Summary:
GitHub commits:

a0b7b06204
ef7aea5536
71627925c8
db34635b88
142d304d0b
ea29e4b9b5
c7d41fa32d
94d079fe1a
e62fbe895f
80e97453bc
639203e0e6

Reviewed By: jurajh-fb

fbshipit-source-id: 99e50123058bf0296db81b2b625a75f19199acfd
2021-05-24 14:07:58 -07:00
svcscm svcscm
97f3dab4d1 Updating submodules
Summary:
GitHub commits:

8ab5913501
b4ede024ec
b87a40a9ed
48f6f9fed0
999355e3c8
a607b88240
a2fe649ac3
5b9d76621a
9bed02c3ab
5cdb09e881
8bb3bc3d0c
2c30962515
5e86b7c49e

Reviewed By: jurajh-fb

fbshipit-source-id: ddc8d753d81bc0011fd1e7e7d29d1cf87c844570
2021-05-24 13:43:03 -07:00
Matthew William Edwards
f4212a2ab3 Fix openr.thrift Python Module Build
Summary:
Add Dockerfile build for openr.thrift python module.

The python module is built by:
1. Building and installing Facebook libraries with fbcode_builder
2. Building Open/R
3. Generating Cython files from thrift files with the FB thrift compiler
4. Generating C++ files from the Cython modules with the Cython compiler
5. Compiling the C++ modules into shared objects

Future work for building and distributing Breeze:

- Fix the hacks in build_breeze.sh, see comments therein
- Use a staged Dockerfile build for the Open/R and Breeze build
- Install openr.thrift. The openr.thrift shared objects are build and
  stored in the Docker image generated by Dockerfile, but are unused.
- Install all the openr python submodules in a single openr site-package
- Add cross-compilation to the openr.thrift build. This is needed for
  Terragraph
- Upload the openr python package to PyPi

Reviewed By: saifhhasan

Differential Revision: D28614443

fbshipit-source-id: 38b7e7c5594fd4bb5a338f19c69e5fc3b3b95863
2021-05-24 13:13:38 -07:00
Jun Wu
5edc1f6433 metalog: expose compact as debugcompactmetalog command
Summary: This makes it possible to use non-debugshell to compact the metalog.

Reviewed By: DurhamG

Differential Revision: D28550902

fbshipit-source-id: 789830ba35243d248397e6a52ee343584c1e01a9
2021-05-24 11:44:55 -07:00
Jun Wu
318f2ae812 metalog: expose compact and path API to Python
Summary:
The "compact" API rebuilds the metalog by removing older history. It could be
useful to reduce the size overhead of the metalog.

This is also useful if we're doing other "rebuild" work, such as rebuilding the
changelog.

Reviewed By: DurhamG

Differential Revision: D28550903

fbshipit-source-id: 56f875bd955247181236a976dcce6163d126a4b6
2021-05-24 11:44:55 -07:00
Simon Farnsworth
f125000b2b Do not alter rate limit inside derived data utils
Summary: I'm going to reuse this for AOSP import logic speedups, and I do not want my low QPS limit overridden by a higher QPS limit set for backfilling. Push the rate limiter out

Reviewed By: StanislavGlebik

Differential Revision: D28638180

fbshipit-source-id: ef3a783d4b1993614a146f534337f719958a1f36
2021-05-24 11:43:01 -07:00
Alex Hornby
6d9926ab78 mononoke: short circuit PutBehaviour::IfAbsent in sqlblob
Summary: We don't need to load the chunk data or update the chunk generation if the blobstore key is already present in IfAbsent mode.

Reviewed By: farnz

Differential Revision: D28640820

fbshipit-source-id: 3eab255ebfc896d4950935e3d7350b19f9a280b9
2021-05-24 11:08:38 -07:00
Jun Wu
bd1fb8e337 zipimport: workaroud pyc invalidation due to timezone settings
Summary:
The zipimport logic requires the pyc mtime to match its source. However, the
Windows system time zone can invalidate it and cause slow startups.

Workaround it by making the zipimport mtime function return a fallback value so
the mtime check is then bypassed.

    # zipimport.py, _unmarshal_code
    source_mtime, source_size = \
        _get_mtime_and_size_of_source(self, fullpath)

    if source_mtime:  # if source_mtime is false, then the check is bypassed.
        # We don't use _bootstrap_external._validate_timestamp_pyc
        # to allow for a more lenient timestamp check.
        if (not _eq_mtime(_unpack_uint32(data[8:12]), source_mtime) or
                _unpack_uint32(data[12:16]) != source_size):
            _bootstrap._verbose_message(
                f'bytecode is stale for {fullname!r}')
            return None

Change my Windows time zone from GMT-7 to GMT-4. Set PYTHONVERBOSE and
PYTHONDEBUG to 1. Ran `hg init -h` and check its stderr. It prints:

  # bytecode is stale for 'edenscm.traceimport'

and alike before this change, and no longer after replacing the `__init__.py`
in the zip with the new version.

Reviewed By: DurhamG

Differential Revision: D28622287

fbshipit-source-id: bb3e8e378ea168e4f83f4b6aa9713103b2c90ef8
2021-05-24 09:36:37 -07:00
svcscm svcscm
91078f9d51 Updating submodules
Summary:
GitHub commits:

22fe579c15
79b4d47997
5266a01461
0e31e7310a

Reviewed By: yns88

fbshipit-source-id: 72580ef41e49349ddaf248188ef939ff3ec029b2
2021-05-24 08:43:44 -07:00
svcscm svcscm
c6566421b4 Updating submodules
Summary:
GitHub commits:

d1ae4f30d1
408fead8d1

Reviewed By: yns88

fbshipit-source-id: b67b0347d3f5b7352490fb6b7d0514160fa1a9ad
2021-05-24 08:10:36 -07:00
svcscm svcscm
51bdc5b5d8 Updating submodules
Summary:
GitHub commits:

53b5d2834b
ba7bab214f
a331aa2872
bc896a8d61

Reviewed By: yns88

fbshipit-source-id: d0d5f31d372cd719b46c2b14cc694629ad47db0f
2021-05-24 06:46:37 -07:00
svcscm svcscm
fff239e4d9 Updating submodules
Summary:
GitHub commits:

dd6df0ab64
7c4c87a41e
d736580150
e43c29b5dd
bab773ace9
e722ad30d6

Reviewed By: yns88

fbshipit-source-id: 5df46e1d64f161dd14da70cf520926e157ecd60a
2021-05-24 06:05:25 -07:00
Alex Hornby
6b36c5a69f mononoke: fix walker chunk bounds when chunk smaller than fetch and OldestFirst
Summary:
Fix and add tests for two problems when running in OldestFirst mode.

1. The computed chunk bounds were incorrect when the walker chunk size was smaller than the bulkops fetch size.

2. when loading a checkpoint in OldestFirst direction the chunk bounds to continue from were reversed

Reviewed By: farnz

Differential Revision: D28624622

fbshipit-source-id: 23d3a3505631447da6607bc472f625a34d0b8752
2021-05-24 05:58:18 -07:00
Johan Schuijt-Li
112008bf7a mononokepeer: fix error message
Summary:
Would previously fail with:
  AttributeError: 'timeout' object has no attribute 'message'

Reviewed By: mzr

Differential Revision: D28636534

fbshipit-source-id: 44df954b659cfbfbe0b468798e7eb5142478be67
2021-05-24 05:34:10 -07:00
svcscm svcscm
b4adaf7fb9 Updating submodules
Summary:
GitHub commits:

867540bc87
6605e62998
5b1ce78490
059091a42e
82368206b7
49229bcf7e
dfc39e3293
ded6ff0c62
26362adeb9

Reviewed By: yns88

fbshipit-source-id: f2be3fd00c695aa34c9bddefe7eb252321253a34
2021-05-24 05:31:23 -07:00
svcscm svcscm
4df2c0486d Updating submodules
Summary:
GitHub commits:

7727ed119b
0d9bbf48e7
db2e894073
e8a5b72c28
bb418eb2ae
056d233c7f
9f42eb4f76
03d9339cbc
d26ecd03e7
fddeea9b81
a6a91b3d12

Reviewed By: yns88

fbshipit-source-id: 5799b7336079f2df54f7a3e4429f2754ceaedb45
2021-05-24 04:47:56 -07:00
Thomas Orozco
21a7095af7 mononoke/wireproto_sink: track last failed io not just last failed flush
Summary:
In practice, if the client has disconnected it's unlikely that the thing that
will fail is a flush (instead if will probably be the poll that checks if we
can write before). So, let's track that separately.

This isn't super important since we can already infer the timing from when the
error was logged but it'll make the logs less ambiguous.

Reviewed By: johansglock

Differential Revision: D28637258

fbshipit-source-id: 3bc9c9aaa9fc8cf7a2d2514fb520cb1729f4c560
2021-05-24 04:27:36 -07:00
svcscm svcscm
01f82d44d7 Updating submodules
Summary:
GitHub commits:

0fee9dfce9
9c8053bd72

Reviewed By: yns88

fbshipit-source-id: 275dcf17cc5d3d3dc70da84c30f69cab7b36ca57
2021-05-24 04:24:34 -07:00
Mark Juggurnauth-Thomas
6333939fa7 blame_v2: add "skip past this change" metadata
Summary:
Extend the `blame_v2` format to include metadata about the location in the
parent commit that a blamed line replaces.  This can be used to implement
accurate "skip past this change" in clients.

Most ranges only need the range of lines that the original blame range
replaces.  For ranges that are inserts, the parent range is of zero length and
the offset indicates the line that the range was inserted before.

For renames, we must include the path of the file before the rename, so that
the file can be found in the parent.

For merge commits, if the file is present in more than one parent, then lines
that are introduced in the merge commit itself have multiple possibilities for
the parent range.  We select and record the first parent that contains the file
as the provider of the parent range for these lines.  This favours the p1
history of the file, but allows "skip past this change" to work when files
are merged in.

Reviewed By: farnz

Differential Revision: D28546768

fbshipit-source-id: 2af1e95a0d27fb25aeea51682177fbac2c41b029
2021-05-24 03:49:27 -07:00
Simon Farnsworth
b40bd2aad3 Use bulk derivation (as used by derived data tailers) for faster derived data
Summary:
The derived data tailers use batch derivation aided by a graph structure. This does batches of derivation in-memory, then writes out the result of a batch.

Use this mechanism in `branch_forest_updater`

Reviewed By: mitrandir77

Differential Revision: D28614817

fbshipit-source-id: 351007a87302fb357e0f6db386e4493bb7879c78
2021-05-24 03:38:40 -07:00
Jan Mazur
d219688a5d reenable -debuginfo packages
Summary: Reenabling *-debuginfo packages

Reviewed By: krallin

Differential Revision: D28634015

fbshipit-source-id: 8f49c114598a63981c48c23fb6da0d6ce3e74ec1
2021-05-24 02:04:36 -07:00
svcscm svcscm
8a3bb03416 Updating submodules
Summary:
GitHub commits:

b085afde1d
070ed349c1
c1a373ae78
467351aa7d

Reviewed By: yns88

fbshipit-source-id: 75f6513b2c3427cd804b23eaabd0c59ee1011d69
2021-05-23 22:13:27 -07:00
svcscm svcscm
9abbbee59f Updating submodules
Summary:
GitHub commits:

e2813cef8a
01231c1c83
95d2c44c06
28138ebc68
44faeeae1e
3765c6aff2

Reviewed By: yns88

fbshipit-source-id: 88b10f57a17422bf3227c33633a6b1d8ff75ce2c
2021-05-23 21:46:52 -07:00
svcscm svcscm
25a136521c Updating submodules
Summary:
GitHub commits:

3d53fb6c67
05d305f8a0
34c427d749
8e399bb0af
44f3056104
e73bbf7f95
b9524790d6
83c7734124
5b94df2ea6

Reviewed By: yns88

fbshipit-source-id: 8592b56e9abd4e363176c15384c4ba76d374adde
2021-05-23 21:16:28 -07:00
svcscm svcscm
b1ef432c96 Updating submodules
Summary:
GitHub commits:

54c3d06fd2
7ad54d87e1
7404a81314
eb91bcc4cf
ece63f86ba
0e63fe78ee
dc43a3d5b5
3bda65152c
be277dc73b
ac37a8915b

Reviewed By: yns88

fbshipit-source-id: f592dc218470d062552843ff723895fcdff3ffea
2021-05-23 20:57:24 -07:00