Commit Graph

56606 Commits

Author SHA1 Message Date
Xavier Deguillard
deffd9a477 minibytes: address clippy warnings
Summary: Using ptr.add is shorter and preferred to ptr.offset.

Reviewed By: quark-zju

Differential Revision: D20452752

fbshipit-source-id: 1dc2fdbc392267d2d690673c10dcc161ecd00dfa
2020-03-16 14:58:22 -07:00
Xavier Deguillard
67c8cf22a3 hgtime: address clippy warnings
Summary:
These warnings are fairly trivial, as it recommends using single quote (char)
for single characters search instead of a double quote (str).

Reviewed By: quark-zju

Differential Revision: D20452408

fbshipit-source-id: b2951e133e57633a8e766536e22969fa9ac0ecee
2020-03-16 14:58:22 -07:00
Xavier Deguillard
bb30c40375 types: address clippy warnings
Summary:
Clippy had 3 sources of warnings in this crate:
 - from_str method not in impl FromStr. We still have 2 of them in path.rs, but
   this is documented as not supported by the FromStr trait due to returning a
   reference. Maybe we can find a different name?
 - Use of mem::transmute while casts are sufficient. I find the cast to be
   ugly, but they are simply safer as the compiler can do some type checking on
   them.
 - Unecessary lifetime parameters

Reviewed By: quark-zju

Differential Revision: D20452257

fbshipit-source-id: 94abd8d8cd76ff7af5e0bbfc97c1e106cdd142b0
2020-03-16 14:58:21 -07:00
Xavier Deguillard
82d3c7f544 configparser: address clippy warnings
Summary:
Clippy complains about 3 things:
 - Using raw pointers in a public function that is not declared as unsafe. This
   happens for C exported ones, this feels like a warning, so I haven't changed
   it.
 - Using .map(...).unwrap_or(<default value constructed>). The recommendation
   is to use .unwrap_or_default().
 - Single match instead of if let, the latter makes code much shorter.

Reviewed By: quark-zju

Differential Revision: D20452751

fbshipit-source-id: 8eeff7581c119c651ca41d8117f1f70f15774833
2020-03-16 14:53:45 -07:00
Stefan Filip
1fb5acf242 dag: use IdDagStore in IdDag with type parameter
Summary: Make IdDag storage generic by depending on IdDagStore.

Reviewed By: quark-zju

Differential Revision: D20471712

fbshipit-source-id: 3a2668f301758a3c880db35c9f0db6887ef1dd38
2020-03-16 14:41:41 -07:00
Stefan Filip
236292c0fd dag: add the GetLock trait
Summary: Used to generalize `get_lock` functionality.

Reviewed By: quark-zju

Differential Revision: D20471710

fbshipit-source-id: e44d5b22ecacdb653170ef83914354f521f82dfc
2020-03-16 14:41:40 -07:00
Stefan Filip
66436b4a3c dag: add the IdDagStore trait
Summary: Abstract the storage functionality required by IdDag.

Reviewed By: quark-zju

Differential Revision: D20449122

fbshipit-source-id: fc3c7d7b88d74f7a93670d310be2e680f35e8ce7
2020-03-16 14:41:40 -07:00
Stefan Filip
1239628ef8 dag: move IdDag storage details to the iddagstore module
Summary:
Right now the module has one implementation IndexedLogStore. The name could
be more specific in the context of the crate.

The goal will be to add a trait for storage requirements of IdDag and
make IndexedLogStorage one implementation of that trait.

Reviewed By: quark-zju

Differential Revision: D20446042

fbshipit-source-id: 7576e1cc4ad757c1a2c00322936cc884838ff710
2020-03-16 14:41:40 -07:00
Harvey Hunt
d1b4f83bf5 mononoke: Log number of possible LFS fetches for a getpack request.
Summary:
Update the `getpack` code to calculate how many files (and their total
size) would be served over LFS.

NOTE: The columns have `Possible` in their names as we might not have LFS
enabled, in which case we aren't actually fetching this many blobs from an LFS
server.

Reviewed By: farnz

Differential Revision: D20444137

fbshipit-source-id: 85506d8c468cfdc470684dd216567f1848c43d08
2020-03-16 14:11:49 -07:00
Chad Austin
97f2921deb fix an assertion violation in GlobBenchmark
Summary:
In dev mode, the glob benchmark failed inside of
folly::Range::operator[] because asserting null termination
technically violates the bounds check.

Reviewed By: simpkins

Differential Revision: D20268416

fbshipit-source-id: ee9b16a6eb9882e850631aa9d83fffe7b6fb67c3
2020-03-16 13:28:24 -07:00
svcscm
5f7a50bdb4 Updating submodules
Summary:
GitHub commits:

b0f50dfd3f
db0e476dad
d66908091d
e1bd93ccd4
002ac44046

Reviewed By: zpao

fbshipit-source-id: f32ab73a49485237d281b3b89c8ac30dcae00ae1
2020-03-16 13:23:18 -07:00
Jun Wu
1f64b4ec50 nameset: fix LazySet iteration
Summary:
The `next` method forgot to increase the iteration index, causing infinite
iteration.

Reviewed By: ikostia

Differential Revision: D20473206

fbshipit-source-id: 82a95de1b1c12ac4e9e4d328a0adba7145d7b24c
2020-03-16 13:00:35 -07:00
Xavier Deguillard
181981452d build: have make local run on Centos8
Summary:
On Centos8, `python` no longer exist, instead `python2` or `python3` must be
used.

The onetime-setup.py will have to be re-worked a bit too for Centos8
differences.

Reviewed By: quark-zju

Differential Revision: D20284986

fbshipit-source-id: bb1d9c8b211f1e50ce379fcb6bd92d677323935b
2020-03-16 12:54:01 -07:00
Stanislau Hlebik
72f788f808 mononoke: add is_chunked method
Reviewed By: ikostia

Differential Revision: D20470896

fbshipit-source-id: 91949dd6e77e17ef06d3144fb9ade89d10423151
2020-03-16 12:33:08 -07:00
svcscm
bbd969c3ec Updating submodules
Summary:
GitHub commits:

5f7b4076ff
ed2e12f885

Reviewed By: zpao

fbshipit-source-id: 767412bda9e005a12eec1b8808dc157681f38f51
2020-03-16 10:52:53 -07:00
Jun Wu
1734cfe169 metalog: allow overridding root using HGFORCEMETALOGROOT
Summary:
Together with `debugmetalogroots`, this allows some kind of "time travel" to
investigate repo states in the past.

Reviewed By: sfilipco

Differential Revision: D20449409

fbshipit-source-id: ed5c134f9e9ee235b24f45c1aa35867a55a71fe5
2020-03-16 10:21:48 -07:00
Jun Wu
d7fd7c15d6 metalog: add debugmetalogroots
Summary: The command lists all metalog roots. It's useful for debugging purpose.

Reviewed By: sfilipco

Differential Revision: D20449408

fbshipit-source-id: 206573b20c7740b6e536bf5d97276175c684959a
2020-03-16 10:21:47 -07:00
Jun Wu
3bf265bcd2 bookmarks: sort bookmarks before writing
Summary:
This makes the content of `bookmarks` stable.
It will make the metalog root IDs stable in tests.

Reviewed By: sfilipco

Differential Revision: D20449410

fbshipit-source-id: 969be63ab231f5865ec62e99398b5318d4257093
2020-03-16 10:21:47 -07:00
Jun Wu
8115053c00 indexedlog: implement xxd-like fmt::Debug for Log
Summary: This makes `hg debugindexedlog dump` more useful.

Reviewed By: sfilipco

Differential Revision: D20448863

fbshipit-source-id: c5cc24449ae00ee329ce02bf0adf947ff57e72ed
2020-03-16 10:21:46 -07:00
Durham Goode
a13fcd4910 workingcopy: support returning directories from the walker
Summary:
Purge needs to be able to see what directories the walker traversed, so
it can delete them if they are empty. Instead of having the walker call
match.traversedir (which it seems like a bizarre pattern to use the matcher as a
holder for a non-matching related function), let's have the walker return an
enum and have an option to return directories.

At the python layer we then translate this into match.traversedir calls, but we
can clean that up later.

Reviewed By: quark-zju

Differential Revision: D19543795

fbshipit-source-id: cc51c86c91799d3df2c65d25a7b6cfe810206d0a
2020-03-16 10:15:26 -07:00
Durham Goode
fc7739fa26 workingcopy: rename walker results
Summary:
In preparation for supporting returning directories from the walker (to
support purge), let's rename the result structure to be more generic.

Reviewed By: kulshrax

Differential Revision: D19543791

fbshipit-source-id: 9b71452c879cf397ae92533a4ef4727140ac7369
2020-03-16 10:15:26 -07:00
Durham Goode
01285a9d4a workingcopy: handle file normalization for rust walker
Summary:
On case insensitive systems we need to normalize file case. I've made a
rust case normalizer, but it requires some more tweaks. In the mean time, let's
handle this at the matching and output stages of the rust walk.

This is probably the pattern we want to follow later anyway, so the walk is
completely decoupled from normalization.

Reviewed By: kulshrax

Differential Revision: D19543797

fbshipit-source-id: 2ef8bdcecb2611a08680441fc030c64c2f4097d1
2020-03-16 10:15:25 -07:00
Durham Goode
27d1ebe9ba workingcopy: support ignore files from rust walker
Summary:
Previously it didn't handle ignoring ignored files. Let's support this
via simple matcher composition.

Reviewed By: kulshrax

Differential Revision: D19543792

fbshipit-source-id: 0c807f3c1ffc10894f13373cbe9deb2b824f1ac9
2020-03-16 10:15:25 -07:00
Durham Goode
05e09b2b89 workingcopy: report invalid file types from rust walker
Summary:
The mercurial tests print errors when they encounter 'fifo' files.
Let's handle that case.

Differential Revision: D19543796

fbshipit-source-id: f87d4b9c3f0ad8b8d8ebe2e6d18e325fc93d0ae9
2020-03-16 10:15:25 -07:00
Durham Goode
1b892e1d50 tests: don't test \1 character in hgsql tests
Summary:
\1 is not a valid character in our new RepoPath's, so let's remove it
from the tests.

Reviewed By: quark-zju

Differential Revision: D19543793

fbshipit-source-id: 383a4b3f5c54b47180fe667f234a43eda22f91b6
2020-03-16 10:15:24 -07:00
James Crooks
20afa84689 Move fbcode/eden/mononoke/cmds/populate_healer.rs to async diff2
Summary:
Another small change moving toward using the new futures. This time we
make get_resume_state() async. This pushes the implicit .compat() we would otherwise have down to where we call ThriftManifoldBlob::get() which still returns an old-style BoxFuture.

Reviewed By: farnz

Differential Revision: D20444381

fbshipit-source-id: a5aab448c5b25e71e4ce5ea031302280d586066f
2020-03-16 09:23:46 -07:00
James Crooks
1b941f3d76 Move fbcode/eden/mononoke/cmds/populate_healer.rs main to async
Summary:
Pushes the `compat` call in main() down one level into
populate_healer_queue(). This ended up being straightforward, just a fn
signature change and a couple of callsite changes.

This only affects main(), going to follow-up with future diffs to push it down
a little farther.

Remaining line deltas are just due to rustfmt.

Reviewed By: farnz

Differential Revision: D20432676

fbshipit-source-id: 98100f1b651648d6e3509fa56e94b80ee075843f
2020-03-16 09:23:46 -07:00
svcscm
61c97d8799 Updating submodules
Summary:
GitHub commits:

bddff1975b

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: d245e5caf604c0860c5f6cea1b300361a651f359
2020-03-16 08:58:04 -07:00
Stanislau Hlebik
5790f7e176 mononoke: add lfs rollout percentage
Summary:
Allow to gradually rollout lfs. A lot of the details are covered in D20441254
I won't repeat them here. I'd only mention that in order for fastreplay to
correctly calculate percentages this diff starts to log client_hostname for
fastreplay.

Reviewed By: ikostia

Differential Revision: D20441264

fbshipit-source-id: e272176f68879f6c545784609799d21daedec5eb
2020-03-16 08:18:41 -07:00
svcscm
cb933c03c4 Updating submodules
Summary:
GitHub commits:

3697fb0572
e58cb42f11

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: b871b9ca4aa01a776e9a1c4e4bd931c5c84b4204
2020-03-16 08:18:41 -07:00
svcscm
14f8fb8a7a Updating submodules
Summary:
GitHub commits:

ca3e3a7e40
8ba0a5dd52
c3e69d7a9a

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 5fcb28478c9fd4819f88eb9de7257362e3ed0980
2020-03-16 05:56:07 -07:00
Harvey Hunt
0146026587 mononoke: lfs_server: Use Identity::FromStr when parsing identities
Summary:
D20139308 added a FromStr implementation for Identity. The LFS
server uses the same code when reading identities from the command line.

Update `idents_from_values` to use the `FromStr` implementation.

Reviewed By: krallin

Differential Revision: D20441439

fbshipit-source-id: 5809ed54d1971fd778b82cc726739a382312747a
2020-03-16 05:41:49 -07:00
Thomas Orozco
89cbb08a09 mononoke: move ContextConcurrencyBlobstore to RepoBlobstore
Summary:
Right now, ContextConcurrencyBlobstore is instantiated in make_blobstore, which
makes it a lot more effective (3 times more effective, in fact) than we want it
to be, since a ticket is acquired by 3 blobstores in the chain in order to
complete a put:

- The multiplex
- The two underlying blobstores

This also has the potential to deadlock if all tickets are held by the
multiplex, which results in an eventual timeout after 600s of waiting in the
multiplex (this looks like it might be happening at least once or twice per
hour right now on the experimental tier).

In any case, the intention had always been to have one of those per repo, not
one per sub-blobstore, so let's do that. The more natural place to put this
seems to be the RepoBlobstore instantiation.

Since I anticipate I might not be the only one who gets tripped up by this at
some point, I also added a comment about this. I also updated the blobsync
tests to stop re-implementing `RepoBlobstoreArgs::new()` so that adding new
blobstores in RepoBlobstoreArgs will have minimal friction.

Reviewed By: HarveyHunt

Differential Revision: D20467346

fbshipit-source-id: a6ad2d8f04bff1c6fcaa151e947cb8af919eec07
2020-03-16 05:30:11 -07:00
svcscm
f794c682dc Updating submodules
Summary:
GitHub commits:

561425e0b1
9a0fca4e92
f9cacabd86
6163f1747b

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 6dd4bb48875c329d1f2cbfc22439b04676d8cd9a
2020-03-16 04:26:36 -07:00
Stanislau Hlebik
20b8b76b87 mononoke: introduce SessionLfsParams
Summary: For now it's identical to LfsParams - it will change in the next diff

Reviewed By: krallin

Differential Revision: D20442530

fbshipit-source-id: 8434610373bb9aefe16702207448283b34676ca2
2020-03-16 03:52:20 -07:00
Stanislau Hlebik
de7c2d335e mononoke: sync configerator thrift update
Reviewed By: farnz

Differential Revision: D20441275

fbshipit-source-id: ced24b35df19336c9d648b445f58ccc746caa360
2020-03-16 03:52:20 -07:00
svcscm
afd20ce2e7 Updating submodules
Summary:
GitHub commits:

0dd8989023
b9f15d1ab3
ace2cc78a6

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 859baf40e546a7effea1d133a618e097f896e751
2020-03-16 03:52:19 -07:00
svcscm
60d9aa6823 Updating submodules
Summary:
GitHub commits:

a944674782
13655b8a20
6a22786f00

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 6efde507d82788e58c35331c455150bf963ed461
2020-03-15 19:23:53 -07:00
svcscm
a9fe040290 Updating submodules
Summary:
GitHub commits:

e2b274cab4
f08234c43c
3a6252c242
57b03b1443
4253b4f1e4
3a4a9fcdcb

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 9a2ef0f0a1c4d1761c13888bde13cdacfbea2be0
2020-03-14 23:23:31 -07:00
svcscm
f8920aaff6 Updating submodules
Summary:
GitHub commits:

dacd7bf0a9

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 2094060228751751920db3b1bfb2ee04edc1ec8f
2020-03-13 23:19:53 -07:00
Xavier Deguillard
0fa09ddc5d gen_version.py: run it with python3
Summary: On Centos8, python no longer exist.

Reviewed By: quark-zju

Differential Revision: D20452244

fbshipit-source-id: 07dd0256c1dc227eb9259954951184d282f5c272
2020-03-13 20:08:34 -07:00
Xavier Deguillard
fc51908dba pyworker: empty flags are valid
Summary: Empty flags can be sent by Mercurial, do not error out in that case.

Reviewed By: quark-zju

Differential Revision: D20450124

fbshipit-source-id: c85af42be2afb95b09057583f6fec3a2a13d478a
2020-03-13 19:41:48 -07:00
Xavier Deguillard
a07c1b6e77 remotefilelog: pass the right config object to Rust
Summary: Fallout from D20432938.

Reviewed By: quark-zju

Differential Revision: D20449456

fbshipit-source-id: a7c3d7f6b9b2fec57b83e5ae3e789c722a45ca39
2020-03-13 19:41:48 -07:00
Xavier Deguillard
185478d122 pyworker: stabilize quickcheck tests
Summary:
In some rare cases, quickcheck may give us 2 identical paths with different
Key, Mercurial will never do that, therefore reject this.

Reviewed By: quark-zju

Differential Revision: D20451344

fbshipit-source-id: 800b31f1aeea38322052baedb918c4f45a1ec95d
2020-03-13 19:37:09 -07:00
Hezi Zhang
f9a511070e --clean option for buck clean
Summary: `buck run edenfsctl -- du --clean` would help reclaim space from buck-out directories.

Reviewed By: chadaustin

Differential Revision: D20203950

fbshipit-source-id: bda045a48ca73b0a1d3e157eff2f92b290a611f5
2020-03-13 19:23:34 -07:00
Xavier Deguillard
fd8d92f1f5 revisionstore: allow indexing LFS pointers via sha256
Summary:
While the sha256 of a blob gives access to its content, it doesn't allow
accessing its metadata, by adding a sha256 index, we can easily get the
metadata of a blob via its content hash.

Reviewed By: quark-zju

Differential Revision: D20445624

fbshipit-source-id: 42c04bd69d3c7380706c6237c5b4f4061c016cca
2020-03-13 19:03:29 -07:00
Xavier Deguillard
d9cca63444 types: add a into_inner method to Sha256
Reviewed By: quark-zju

Differential Revision: D20445623

fbshipit-source-id: d9cba7ddd16a8e89c76cd5e988ab0fb79383d0c2
2020-03-13 19:03:29 -07:00
Xavier Deguillard
60be0ac94d types: fix typo when displaying Sha256
Reviewed By: quark-zju

Differential Revision: D20445622

fbshipit-source-id: dc9a8a165ca55fdece90a5eb3a87cd3c28f444cb
2020-03-13 19:03:29 -07:00
Xavier Deguillard
6ee3a8f42f revisionstore: add metadata to FakeHgIdRemoteStore
Summary: This is necessary to properly test LFS stores.

Reviewed By: quark-zju

Differential Revision: D20445625

fbshipit-source-id: 530ddf87249e8d721957806f2d8edef3262f303c
2020-03-13 19:03:28 -07:00
Xavier Deguillard
5002d01e0a revisionstore: allow indexedlogutil users to lookup in different indices
Summary:
The OpenOptions allow for multiple indices to be added, but lookup had no way
to querying these multiple indices.

Reviewed By: quark-zju

Differential Revision: D20445627

fbshipit-source-id: 0cb754ba17b452d892b7bcb56d502d5753ef963a
2020-03-13 19:03:28 -07:00