Commit Graph

64531 Commits

Author SHA1 Message Date
Jun Wu
b0e6b21fc5 indexedlog: add RotateLog::slice_to_bytes
Summary: This will make RotateLog achieve zero-copy reading more easily.

Reviewed By: kulshrax

Differential Revision: D27724331

fbshipit-source-id: 57915516dc6bd1935838bd099a60c104f0bdef3d
2021-04-13 12:32:25 -07:00
Jun Wu
b895c335bd indexedlog: make RotateLog::iter_dirty not take mut self
Summary: This makes it more flexible.

Reviewed By: kulshrax

Differential Revision: D27724332

fbshipit-source-id: 43ad670519f0617a97e0b7d38b374f497e9c01af
2021-04-13 12:32:25 -07:00
Jun Wu
5c3d6d152f io: add config for pager wrapping mode
Summary:
This allows setting the wrapping mode. For example:

   lhg log -pv
   # copy paste long lines works.

   lhg log -pv --config pager.wrapping-mode=unwrapped
   # lines are not wrapped, ">" is shown for long lines.

   lhg log -pv --config pager.wrapping-mode=word
   # long lines wrapped at word level.

The default value matches "less" behavior.

Reviewed By: DurhamG

Differential Revision: D27720767

fbshipit-source-id: e29d6b13656407c0a1e63287fb96e2f8d914cfc8
2021-04-13 12:15:16 -07:00
Andrey Chursin
f2d873d77c checkout: check unknown files during native checkout
Summary: This is needed to prevent situation when we try writing into file that is untracked by hg

Reviewed By: DurhamG

Differential Revision: D27667152

fbshipit-source-id: 31bb9e30bd6b58e80ba96d280ff6ca1842c8caf6
2021-04-13 11:47:54 -07:00
Andrey Chursin
36b6bb98d3 checkout: bindings for check_unknown_files
Reviewed By: DurhamG

Differential Revision: D27667154

fbshipit-source-id: 0bdfb74761b01c1fe41ca8fd519c065a28381e74
2021-04-13 11:47:54 -07:00
Andrey Chursin
95e84b7302 checkout: introduce CheckoutPlan::check_unknown_files
Summary: This method checks if any of files that checkout writes is not tracked in hg and exists on disk

Reviewed By: DurhamG

Differential Revision: D27667153

fbshipit-source-id: 4ad8bc08520678ea0b51008ed14fb51ca4a98f76
2021-04-13 11:47:54 -07:00
Kostia Balytskyi
bf801f70a0 megarepo: import all_config_versions.thrift
Summary: To sync D27741269.

Reviewed By: mitrandir77

Differential Revision: D27741277

fbshipit-source-id: 52ade8d19ae434759a9b889fb82816dbf1feecfe
2021-04-13 11:38:27 -07:00
Stanislau Hlebik
0450243694 mononoke: fix sql query for streaming changelog
Summary:
Previously this query failed because it tried to convert bytes to int, and our
mysql wrapper doesn't support that.
Let's cast it instead

Reviewed By: krallin

Differential Revision: D27736863

fbshipit-source-id: 66a7cb33c0f623614f292511e18eb62e31ea582f
2021-04-13 11:15:00 -07:00
Stefan Filip
d54706d339 dynamicconfig: add hostcaps for determining Domain
Summary:
`hostcaps` abstracts the logic for determining whether we have a prod or corp
environment.

Reviewed By: DurhamG

Differential Revision: D27684641

fbshipit-source-id: 50df9a60b6a613b4cb5c9aed6cad2844aae85a6f
2021-04-13 10:42:45 -07:00
Stefan Filip
6e262c3b79 lib: hoist hostcaps to the top level of the lib directory
Summary:
We want to use it in Mercurial and the directory structure was playing bad
with Mononoke's OSS build.

Reviewed By: xavierd

Differential Revision: D27684642

fbshipit-source-id: 8827645eee58fa671f9c9e1964a34c34e3a8eeb6
2021-04-13 10:42:45 -07:00
svcscm svcscm
d03ebbe91b Updating submodules
Summary:
GitHub commits:

d5bf7f5a51
fd00f39f97

Reviewed By: yns88

fbshipit-source-id: 7be1955d8d431f4e5336463501ca0a74b431e921
2021-04-13 09:11:16 -07:00
Katie Mancini
bcf6aa465c teach edenfs to signal itself on MacOS nfs
Summary:
MacOS does not have the device field like linux that we can use to mark edenfs
nfs mounts. But there is the `f_mntfromname` field. This field more typically
might have the path which this nfs mount is mirrored from, but it should be fine
to hyjack as the edenfs indicator field.

Reviewed By: xavierd

Differential Revision: D27717945

fbshipit-source-id: 056fb39dc3273b68d79c26487fd045f4e7f93b7b
2021-04-13 09:09:02 -07:00
Katie Mancini
ccb76f1187 make device type for nfs similar to fuse
Summary:
With fuse we report "edenfs:" as the device, let's do the same thing with nfs
so watchman can recognize edenfs nfs mounts similarly.

I think its fine to use the standard "edenfs" as the server name in the mount
call rather than the address, from looking at:
https://www.systutorials.com/docs/linux/man/8-mount.nfs/

Reviewed By: xavierd

Differential Revision: D27630764

fbshipit-source-id: 9e476c90ece90e758b98d140c6bf4067dbca3661
2021-04-13 09:09:02 -07:00
Simon Farnsworth
0f817c72fb Provide an admin command for blobstore unlink
Summary: Currently just does XDB Blobstore, because the work to do other types and/or go via Packblob is significant.

Reviewed By: markbt

Differential Revision: D27735093

fbshipit-source-id: d3797017a2e0ff7c60525d1f4d4ee3e63b519d49
2021-04-13 08:38:29 -07:00
svcscm svcscm
686c3d849a Updating submodules
Summary:
GitHub commits:

f574e9eabb
9829ae1ad2
095f22b47f
3f39f8cedc

Reviewed By: yns88

fbshipit-source-id: 3018fae904bdd6f668b921e88a4839a082acfb69
2021-04-13 07:43:35 -07:00
Jan Mazur
4cb7732dae remove deprecated --readonly-storage cmdline arg
Summary: We have deprecated it in favor of argument that takes a boolean value.

Reviewed By: farnz

Differential Revision: D27709429

fbshipit-source-id: 45e9569188f2e9d017f1c5bf61f7c61bc0e5318a
2021-04-13 07:09:35 -07:00
Thomas Orozco
3c88bd8832 mononoke/timeseries: track count of valid buckets
Summary:
It's useful when operating with timeseries to know what range of data has been
populated. This diff adds support for this in mononoke/timeseries, by tracking
the number of buckets that fall within intervals where data was provided.

Reviewed By: mitrandir77

Differential Revision: D27734229

fbshipit-source-id: 3058a7ce4da67666e8ce8a46e34e277b69153ea4
2021-04-13 06:24:37 -07:00
svcscm svcscm
c854b43dc2 Updating submodules
Summary:
GitHub commits:

8732809799
13f8c315ca

Reviewed By: yns88

fbshipit-source-id: a10b329ece875bff4928d549f6644541f4fd76e0
2021-04-13 06:22:09 -07:00
Mark Juggurnauth-Thomas
a9b1b36721 admin: set background session class for skiplist build
Summary:
When building skiplists, set the session class to `Background`.  This ensures
that the blobstore writes for the new skiplist have completed fully.

Reviewed By: StanislavGlebik

Differential Revision: D27735411

fbshipit-source-id: 4ba8e8b91dafbb1aa258d15b26e7d773f63b5812
2021-04-13 06:12:20 -07:00
Thomas Orozco
a0a7091517 mononoke/lfs_server: reject Range requests that are out of bounds
Summary:
If the caller asks us for a range that extends past the end of our file, we'd
rather give them an error instead of silently returning the file.

This actually revealed one of the tests needed work :)

Note that right now we'll just end up categorizing this as 500. I'd like to
rework the errors we emit in the Filestore, but that's a somewhat bigger
undertaking so I'd like to do it separately.

Reviewed By: quark-zju

Differential Revision: D27193353

fbshipit-source-id: 922d68859401eb343cffd201057ad06e4b653aad
2021-04-13 05:10:00 -07:00
Mark Juggurnauth-Thomas
876f812e4b commitcloud: do not send backup bookmarks part
Summary:
The backupbookmarks part was used for infinitepush backup bookmarks, which were
deprecated.  Now stop sending the part entirely unless
`commitcloud.pushbackupbookmarks` is set.

Reviewed By: StanislavGlebik

Differential Revision: D27710099

fbshipit-source-id: 1eb404f106f5a8d9df6d73e11f60f89c1fa10400
2021-04-13 03:07:50 -07:00
Thomas Orozco
87aed04d37 mononoke/sql_ext: publish SQL max open connections stat
Summary:
Like it says in the title, this adds support for publishing our max open
connections to ODS. Note that this is a little more involved than I would like
for it to be, but there is no way to get direct access to this information.

This means, we need to:

- Expose how many open connections we have in flight (this is done earlier in
  this stack in the Rust MySQL bindings).
- Periodically get this information out out for MySQL, put it in a timeseries.
- Get the max out of said timeseries and publish it to a counter so that it can
  be fetched in ODS.

This is what this diff does. Note that I've only done this for read pools,
largely because I think they're the ones we tend to exhaust the most and I'd
like to see if there is value in exposing those counters before I use them.

We do the aggregation on a dedicated thread here. I contemplated making this a
Tokio task, but I figured making it a thread would make it easier to see if
it's misbehaving in any way (also: note that the SQL client allocates a bunch
of threads already anyway).

Reviewed By: HarveyHunt

Differential Revision: D27678955

fbshipit-source-id: c7b386f3a182bae787d77e997d108d8a74a6402b
2021-04-13 03:05:23 -07:00
svcscm svcscm
57d116fb8e Updating submodules
Summary:
GitHub commits:

81a15a1adb
0ebf797de8
1f4455d713
c1104a0bb2

Reviewed By: yns88

fbshipit-source-id: d6ea89f19cc5a62cffb4aaeb83748bba0353f8dd
2021-04-13 01:40:59 -07:00
svcscm svcscm
5761360f2c Updating submodules
Summary:
GitHub commits:

3e5bf7fa27
3f7ed5e783
6c9d6e1b0f
bbfb22353f
76306819c0
1ea7df884f

Reviewed By: yns88

fbshipit-source-id: a64d2c0fc85b3bfab78313f7c42d00eaa3803b8c
2021-04-12 23:39:03 -07:00
svcscm svcscm
526166a9cf Updating submodules
Summary:
GitHub commits:

9ad6fd9284
31521cda71
439aa1e9f4
256e9c4326
d2b1aaa2c7
ec4d921178
b7f6dd0f59
a9073a9bf6
de61bb3930

Reviewed By: yns88

fbshipit-source-id: ab8b39ad1d20af73ffc94dde489fbd619f7c59a4
2021-04-12 22:21:05 -07:00
svcscm svcscm
6774ad79d6 Updating submodules
Summary:
GitHub commits:

122417511f
0a2a3ea555
06de0fae90
addfe37f86
5dd826f324
04f6d26ce5
2daf5af7ec
6b9da5dfae
818a3c158f
674e0925af
9de7245269

Reviewed By: yns88

fbshipit-source-id: 1bde45c536dd34199e412b35dec826a0fd6f2358
2021-04-12 21:46:10 -07:00
svcscm svcscm
6c24587d75 Updating submodules
Summary:
GitHub commits:

d2785c1e4b
54578d07d2
e1dec6cdf1
b471fe2a91
eceb4f3ca9
018f1af489
fad054c08b
88f9f4b56c
9466fb0fc0
9e3b2e5c6b
2e4a08c351
96177fedbf
f3b2741e56
520455ce5f

Reviewed By: yns88

fbshipit-source-id: cfb71f154c7c9abf209bfacef7e1d54b43b28e29
2021-04-12 20:56:24 -07:00
Andrey Chursin
80adbe385c rename ephemeralcommit->hiddencommit
Summary: This name is more reasonable, since this commit is not actually ephemeral

Reviewed By: quark-zju

Differential Revision: D27722921

fbshipit-source-id: e2c0243d41a73341f9d0afdc79696ea37b34b8c7
2021-04-12 20:13:11 -07:00
svcscm svcscm
40534cf96b Updating submodules
Summary:
GitHub commits:

f5b12ac58f
f3ee98495e
13e5ae5707
692d08fb3c
639017e814
10a840cc07
9900cbec6d
b4ca2134fb
8af1b04db0
fb97440c0c
abdc703495
47902b22ed
74cfa32337

Reviewed By: yns88

fbshipit-source-id: e973e713e95876c0f0a979785f593e347346a6b9
2021-04-12 19:39:03 -07:00
Andres Suarez
6ccdc46775 third-party: update tracing-subscriber from 0.2.15 to 0.2.17
Reviewed By: dtolnay

Differential Revision: D27721151

fbshipit-source-id: 08a367662d506a48e0d639115c06cd4e714717b9
2021-04-12 19:24:10 -07:00
Xavier Deguillard
db2c912111 store: re-enable store tests under tsan
Summary: Running these with tsan appears to run properly, let's try to re-enable them.

Reviewed By: genevievehelsel

Differential Revision: D27723525

fbshipit-source-id: 42e61d26cf478cbe808698a6a0615015832180fa
2021-04-12 18:46:26 -07:00
Jun Wu
2480bfcef5 discovery: remove unused discovery logic
Summary:
We have `experimental.findcommonheadsnew` set to True in all tests, and
Rust commit backends force the `findcommonheadsnew` paths, which is
pretty much everywhere except hgsql repos. Remove `_findcommonheadsold`.

The fast discovery is also unnecessary. Remove them too.

Reviewed By: DurhamG

Differential Revision: D27630496

fbshipit-source-id: ab1948f03a8c84e75e3b5c9ff4769e17533447d2
2021-04-12 17:11:37 -07:00
Genevieve Helsel
1776efce9c log file accesses to edenfs_file_stats_accesses
Reviewed By: chadaustin

Differential Revision: D27660120

fbshipit-source-id: 7e8e64e9a8db62d818db5177155db77195870e0a
2021-04-12 16:58:09 -07:00
Andre Kouznetsov
a4ed5fe9c0 place a README file under scratch directory to describe what it is for
Summary: Many users have asked what scratch directory is when they look to free some disk space. Placing a README.txt under the scratch root could be helpful to explain what is in there.

Reviewed By: fanzeyi

Differential Revision: D27710277

fbshipit-source-id: e3ccd92fa1920ac4c791026b8d98aa05a1c8b268
2021-04-12 15:17:33 -07:00
Andrey Chursin
aa86902059 checkout: use scmstore in merge.py during checkout
Summary: This diff uses filescmstore for native checkout when nativecheckout.usescmstore config is set

Reviewed By: DurhamG

Differential Revision: D27658844

fbshipit-source-id: ec3442d677ccb25e8b08cc194e4c8c18c0e01fa1
2021-04-12 14:45:11 -07:00
Andrey Chursin
5cc082a2d4 checkout: py binding for filescmstore
Summary: This introduces apply_scmstore method in pycheckout bindings

Reviewed By: DurhamG

Differential Revision: D27658751

fbshipit-source-id: c09e27483bd4ecbecf9ae7193d8d030702b07315
2021-04-12 14:45:10 -07:00
Andrey Chursin
8b1ab2fc4e checkout: support ReadStore in checkout
Summary:
This diff introduces CheckoutPlan::apply_read_store to apply checkout plan using ReadStore as a data source
This requires some minor changes in apply_stream flow as ReadStore does not guarantee ordering of returned files

Reviewed By: DurhamG

Differential Revision: D27658346

fbshipit-source-id: 5a289554d8dd7b6bb4b5a996659cd0661779ad5f
2021-04-12 14:45:10 -07:00
Jun Wu
06a791384b edenapi: configparser::ConfigSet -> configmodel::Config
Summary: The latter is more lightweight.

Reviewed By: DurhamG

Differential Revision: D27641665

fbshipit-source-id: d46f62f9067eb9cb4c8517a62efa6f663d4b6732
2021-04-12 14:29:45 -07:00
Jun Wu
0d764cf718 auth: configparser::ConfigSet -> configmodel::Config
Summary: The latter is more lightweight.

Reviewed By: DurhamG

Differential Revision: D27641669

fbshipit-source-id: d907407f5a6e868862fe37f1f67fbe99ee378156
2021-04-12 14:29:45 -07:00
Jun Wu
cb58689d67 io: configparser::ConfigSet -> configmodel::Config
Summary: The latter is more lightweight.

Reviewed By: DurhamG

Differential Revision: D27641667

fbshipit-source-id: adce5a39fcb5d8e8d5d989fed46991e20ab3710d
2021-04-12 14:29:45 -07:00
Jun Wu
4d0359740c configparser: integrate with configmodel
Summary: Provides a way to read config with lighter dependencies.

Reviewed By: DurhamG

Differential Revision: D27641668

fbshipit-source-id: fc99a78f5f51e63f61d1b049af74f61f5d1916a3
2021-04-12 14:29:45 -07:00
Jun Wu
a25ec579bf lib: add configmodel
Summary:
The `configparser` is now too heavyweight. Some other crates (ex. io, auth,
revisionstore) just want to extract config values without complicated parsing /
remote hotfix logic.

Add a configmodel crate to satisfy the need.

Reviewed By: DurhamG

Differential Revision: D27641666

fbshipit-source-id: 26bd0b606ae3d286b3ec218927aef726d6802c63
2021-04-12 14:29:45 -07:00
svcscm svcscm
6290a32b04 Updating submodules
Summary:
GitHub commits:

9066fd258e
d15ace8076
ca910cb654
0ddba5afdf
aa7da5df5a
3d2a348cda
5b47e394c5
a4259cb015
a480c9737e
7a88919692
733a1cfd5b
85b6e86c4f

Reviewed By: yns88

fbshipit-source-id: 791b085d782a1c7e40c07e7d181ac54c108d56c8
2021-04-12 14:04:24 -07:00
Xavier Deguillard
1aee89eaee store: use NO_PRETTY_VALUES when discovering tests
Summary:
The CMake documentation states:

>By default, the value index of value-parameterized tests is replaced by the
>actual value in the CTest test name. If this behavior is undesirable (e.g.
>because the value strings are unwieldy), this option will suppress this
>behavior.

Which appears to be a decent default, but not when the parameter is a pointer,
in which case the test name will contain some hex values.

Reviewed By: genevievehelsel

Differential Revision: D27713222

fbshipit-source-id: 0f15b24d04817384ff975ad7b07e16b744e1eb2e
2021-04-12 13:43:09 -07:00
svcscm svcscm
99c9a41b6e Updating submodules
Summary:
GitHub commits:

97ed7b0a68
9c4bbed4f9
f45a4a0628
6ea76266cb
aa17d882e1
3284bfbd13
5b59b7d1e9
bb75092574
e6ef67c79a
ad36fe0050
60ceaa5835
736c1d4bb5
57430384f1
3ea632ee61
7423a2e1ce
c609f3c259
4821a3e660

Reviewed By: yns88

fbshipit-source-id: 9a8f0e584fa493f21b14c4e009ab7aecfe6713a8
2021-04-12 12:46:03 -07:00
Arun Kulshreshtha
f738f65d28 http-client: add verbose option to requests
Summary: Add a new `http.verbose` config option that turns on verbose output for libcurl (similar to the output printed by `curl -v`). This can be very useful for debugging HTTP issues.

Reviewed By: DurhamG

Differential Revision: D27693304

fbshipit-source-id: 2ad7a08889f40ffbcd2f14ac9c21d70433629da4
2021-04-12 12:17:12 -07:00
Arun Kulshreshtha
11697cde4c remotefilelog: cache edenapistore
Summary: Make this method match the behavior of `remotefileslog.memcachestore` and cache the `edenapistore` instead of constructing a new one each time. Right now this doesn't matter too much because we currently only call this once when setting up the Rust `revisionstore`, but it would be good to avoid creating multiple instances if we do start using this elsewhere.

Reviewed By: DurhamG

Differential Revision: D27684210

fbshipit-source-id: 7987f603c79758902b4740dd8b46d26a25baec93
2021-04-12 12:17:12 -07:00
Arun Kulshreshtha
fdfc52775d remotefilelog: disable memcache when cachekey is empty
Summary:
This diff causes memcache to be disabled when `remotefilelog.cachekey` is set to the empty string, thereby allowing memcache to be disabled on the command line with `--config remotefilelog.cachekey=''`. This is useful when testing data-layer changes.

Previously, the only way to do this was to add `%unset cachekey` to the `remotefilelog` section of your `hgrc`, which was a bit tedious compared to just using a `--config` flag.

Reviewed By: DurhamG

Differential Revision: D27683782

fbshipit-source-id: 3e0434e98a32db916a07935e8b26f70317f50286
2021-04-12 12:17:12 -07:00
Stanislau Hlebik
85469374e9 mononoke: add leases to x-repo-sync
Reviewed By: ikostia

Differential Revision: D27677897

fbshipit-source-id: e86220f23f5950130f2f4ead2743f9a9b86abed7
2021-04-12 12:08:34 -07:00
Stefan Filip
b48f998a61 mononokepeer: update handling of pipe read errors
Summary: The error is not going to have a message in many cases leading to crashes.

Reviewed By: quark-zju

Differential Revision: D27637119

fbshipit-source-id: 135b133371916dddf0c47a84f00957a8b8fdfe92
2021-04-12 11:07:55 -07:00