Summary: this is the last diff to complete the `reachabilityindex` module migration
Reviewed By: farnz
Differential Revision: D22049853
fbshipit-source-id: d8510f2d1b1c12a1ba903c2afda37096aadd8347
Summary:
When inserting multiple rows using `INSERT IGNORE`, it's possible to hit
deadlocks if two competing transactions insert the same rows in a different
order.
This can be triggered in the mutation store if two clients are pushing
overlapping sets of commits, and the arbitrary changeset order from the
`HashSet` gives the commits in a different order. For example the first
transaction might be inserting "A, B, C", be holding the lock on row A, and be
waiting for the lock on row C, meanwhile another transaction is inserting "C,
D, A", holding the lock on row C, and waiting for the lock on row A.
To avoid these deadlocks, always insert the mutation changesets in a stable
order (sorted by changeset hash or successor hash).
Reviewed By: krallin
Differential Revision: D22067338
fbshipit-source-id: aa54c8a9c0cac4e4ae35855b44e759f8b6cb4f59
Summary:
Add logging of the repo root and repo shared root to the sampling extension.
Reduce the number of different strings that are logged by replacing any path
component that equals the user's username (e.g. their home directory) with
`$USER`.
Reviewed By: farnz
Differential Revision: D22066053
fbshipit-source-id: 39221e5182b3d4df51afa386497003e637a50b13
Summary:
async the blobstore test functions, in preparation for adding test coverage for BlobstoreWithLink::link
The boxable test was duplicating the simple roundtrip test which seemed unnecessary, now it just checks that we can Box the blobstore type.
Reviewed By: krallin
Differential Revision: D21974471
fbshipit-source-id: a3cd71a7503a3a670b3c6223812a2e870a20a16e
Summary: update memblob imports to futures_old in preparation for using new futures to implement BlobstoreWithLink
Reviewed By: krallin
Differential Revision: D21973642
fbshipit-source-id: dea4a686db41a0d618395c4dc411603f922a9e22
Summary: Update imports in preparation for using new futures for the BlobstoreWithLink mixin trait
Reviewed By: StanislavGlebik
Differential Revision: D21973587
fbshipit-source-id: 094aa82abc21848d43f50d6d169087e19339ba04
Summary: Switch blobstore imports to use futures_old naming in preparation for using new futures in BlobstoreWithLink (and later BlobstoreWithEnumeration) traits.
Reviewed By: krallin
Differential Revision: D21972842
fbshipit-source-id: 6022c98e7b7254f7c7beb46dc1c7f83609810853
Summary: Add basic packblob store which wraps blobs in the thrift StorageEnvelope type on put() and unwraps them on get()
Reviewed By: farnz
Differential Revision: D21924405
fbshipit-source-id: 81eccf5da6b5b50bdb9aae13622301a20dca0eac
Summary:
Thift definitions for a new blobstore layer known as packblob, which is an enabling part of compression at the mononoke blobstore layer. Its storage formats will support raw data, individual blob compression and blob delta compression.
The included README.md has more detail on the expected use of these types.
Reviewed By: farnz
Differential Revision: D21761723
fbshipit-source-id: a277800f4b4aed5972c4a5bb277e984ff12660f8
Summary:
I'm going to change it in the next diffs, so let's migrate them to new futures
first.
Reviewed By: krallin
Differential Revision: D22065216
fbshipit-source-id: b06fcac518d684f40b1d19fcef9118ca51236873
Summary: Without this, the most lagged shard will slow down all shards; with this, a single lagging shard doesn't slow down other shards.
Reviewed By: ahornby
Differential Revision: D22018264
fbshipit-source-id: c23f25ea1d0b0b6891d2a6f76dfed467101d2f4d
Summary: I'll want the delay operation to respect shard_id - move it into the stores. While here, do some refactoring that will reduce the size of the next diff.
Reviewed By: ahornby
Differential Revision: D22018317
fbshipit-source-id: d2333dfc6749b2a13bc8a67bfa953ed25cedf847
Summary:
test_no_units_are_active was failing on CentOS 8. Presumably systemd
has some new default services. I believe genevievehelsel is planning on
replacing all of this code, so, rather than understand the failure,
just ignore the additional service names.
Reviewed By: genevievehelsel
Differential Revision: D22055992
fbshipit-source-id: b6f313350d0e1b107fe6ea3d7ed7f5b4eb025ef3
Summary:
From a simple `hg trace log -r .`, it appears that importing the module takes
16ms, which dominates the time it takes to load the treemanifest extension.
Since the module isn't necessary, don't import it.
Reviewed By: farnz
Differential Revision: D22048075
fbshipit-source-id: ccaf2d53eb006faa36d97df452b0db10f4983c23
Summary: Prep for 1.44 but also general cleanups.
Reviewed By: dtolnay
Differential Revision: D22024428
fbshipit-source-id: 8e1d39a1e78289129b38554674d3dbf80681f4c3
Summary:
Python 3.8 provides the (poorly named)
unittest.IsolatedAsyncioTestCase base class that gives each test an
asyncio event loop and runs it. This allows us to write async
integration tests, required to use modern Thrift Python.
Reviewed By: genevievehelsel
Differential Revision: D21955584
fbshipit-source-id: 8366dc7c72b31e1667e72fe1e359587b74e2c614
Summary:
In text mode, Python will use '\r\n' as line endings instead of '\n' on
Windows. This has the negative side effect of making some tests fails due to
the file content not being exactly what is being written.
Reviewed By: fanzeyi
Differential Revision: D21944456
fbshipit-source-id: f7f271e2ef9e2f21bd78c76233e644ce411b63df
Summary: Replaces usages of whitelist/blacklist with include/exclude/allow. These terms are more descriptive and less likely to contribute to racial stereotyping. More context: https://fb.workplace.com/groups/sourcecontrolteam/permalink/2926049127516414/
Reviewed By: farnz
Differential Revision: D22039299
fbshipit-source-id: d5b4cdeca681e1f4c992f7ef0b9f772b2ae1abd3
Summary:
Add a `--record` flag, which uses `git` to track changes in `$TESTTMP`.
I use it to track changes with C revlog vs the Rust revlog.
Reviewed By: farnz
Differential Revision: D22007501
fbshipit-source-id: a8027aa4b2b3b97c3cf5f642434451826fcbded9
Summary:
Interactive smartlog history attempts to access `sltimestamp` as a field of
`slinfo`. The field is actually called `timestamp`.
Reviewed By: quark-zju
Differential Revision: D22042316
fbshipit-source-id: 7c1d53dfc1c8d661a248d8412f7d780c83723721
Summary:
Rename the `subtree` endpoint on the EdenAPI server to `complete_trees` to better express what it does (namely, fetching complete trees, in contrast to the lighter weight `/trees` endpoint that serves individual tree nodes). This endpoint is not used by anything yet, so there isn't much risk in renaming it at this stage.
In addition to renaming the endpoint, the relevant request struct has been renamed to `CompleteTreeRequest` to better evoke its purpose, and the relevant client and test code has been updated accordingly. Notably, now that the API server is gone, we can remove the usage of this type from Mononoke's `hgproto` crate, thereby cleaning up our dependency graph a bit.
Reviewed By: krallin
Differential Revision: D22033356
fbshipit-source-id: 87bf6afbeb5e0054896a39577bf701f67a3edfec
Summary: Replace usages of whitelist/blacklist with include/exclude/filter/allow. These terms are more descriptive and less likely to contribute to racial stereotyping. More context: https://fb.workplace.com/groups/sourcecontrolteam/permalink/2926049127516414/
Reviewed By: kulshrax
Differential Revision: D22039298
fbshipit-source-id: 255c7389ee5ce5e54bbccdfb05ffa4cafc6958e5
Summary: Replaces usages of whitelist/blacklist with include/exclude/allowed. These terms are more descriptive and less likely to contribute to racial stereotyping. More context: https://fb.workplace.com/groups/sourcecontrolteam/permalink/2926049127516414/
Reviewed By: kulshrax
Differential Revision: D22039297
fbshipit-source-id: ab6e55889e6cf42aed35c856475b2ff92f4cb802
Summary: Enable the most common autocomplete use cases in zsh hg histedit autocomplete
Reviewed By: fanzeyi
Differential Revision: D22032612
fbshipit-source-id: 1fd47d30564c51a010785d7d0553e3a5d16f163a
Summary:
This diff updated the `eden du` command. Instead of printing
```Reclaim space from the LFS cache directory by running:
hg -R /data/users/akushner/eden-repos/opsfiles gc
Reclaim space from the LFS cache directory by running:
hg -R /data/users/akushner/eden-repos/megarepo_test gc
Reclaim space from the LFS cache directory by running:
hg -R /data/users/akushner/eden-repos/instagram-server_test gc
```
We now print "Reclaim.." only once and put available commands together
```
LFS cache detected in backing repo. To reclaim space from the LFS cache directory, run:
hg -R /data/users/akushner/eden-repos/opsfiles gc
hg -R /data/users/akushner/eden-repos/megarepo_test gc
hg -R /data/users/akushner/eden-repos/instagram-server_test gc
```
and did the same for working copy under backing repositories.
Reviewed By: fanzeyi
Differential Revision: D22038555
fbshipit-source-id: c19869db6f91e90a0627b3bb4b4f7e95142ae198
Summary: Replaces usages of whitelist/blacklist with includelist/excludelist/incompatiblelist/. These terms are more descriptive and less likely to contribute to racial stereotyping. More context: https://fb.workplace.com/groups/sourcecontrolteam/permalink/2926049127516414/
Reviewed By: kulshrax
Differential Revision: D22039296
fbshipit-source-id: d4c1fd09e40d5ebeaab99515ed19aa5df6c2ecc4
Summary: This diff fixed small problems in previous `eden du` summary printing and added a reminder of option `--clean` to users.
Reviewed By: fanzeyi
Differential Revision: D22038552
fbshipit-source-id: 7dbac858b74833e59d8d3f1c6fb1f5d8944935d0
Summary:
Those tests show up as "skipped". I removed some rarely used ones:
- test-check-jshint: we seldom write javascript.
- test-check-shbang: a slow test that does not do much.
- test-chg: chg is being tested in other tests by default.
- test-docker-packaging: probalby already broken, we don't use docker.
- test-verify-repo-operations: a slow test that is too expensive to run with
limited value.
Reviewed By: markbt
Differential Revision: D21989103
fbshipit-source-id: 8080ad68dc75b59f87336b94af38c8231f4f10f4
Summary:
I think LZ4 was disabled accidentally in our rocksdb getdeps build in
D21319896. Enable it again on macOS and Linux, because otherwise this
breaks people with EdenFS mounts containing LZ4-compressed proxy
hashes.
Reviewed By: xavierd
Differential Revision: D21990356
fbshipit-source-id: b9166c2992ae51f09de3fa9a4f114143aa008f43
Summary:
After being trimmed down to store only a single edge the skiplist update
algoritm was working incorrectly. This diff introduces a way of fixing things
up.
Reviewed By: StanislavGlebik
Differential Revision: D21976995
fbshipit-source-id: 3bc4678011b332419d9f6f7b983a33fa066674bf
Summary: Make most of the handlers on the EdenAPI server return streaming responses.
Reviewed By: krallin
Differential Revision: D22014652
fbshipit-source-id: 177e540e1372e7dfcba73c594614f0225da3a10f
Summary:
In production we trim skiplists to hold a single entry which causes
the update algorithm to behave incorrectly
Reviewed By: StanislavGlebik
Differential Revision: D21976996
fbshipit-source-id: 4946be11512ac9577bdd3320913fe092d513ecdd
Summary:
Support for this was added in D20763778, however they were reverted later in
D21762641. Turned out that all cancelled requests were counting as still
"active" even though they were cancelled a long time ago. It accumulated over
time, and at some point (to be precise, at ~65K of "active" connections) thrift
server started to reject new connections.
The problem was that original diff D20763778 didn't delete ReplyState
structure, which had pointers to a few other C++ structures whose destructor
decremented the number of active connections. To be precise:
1) https://fburl.com/diffusion/qxsu4dqe - ReplyState is allocated here, and was
previously deallocated here - https://fburl.com/diffusion/9fj9ksgx
2) ReplyState has a pointer to ResponseChannelRequest
(https://fburl.com/diffusion/6au1duav), which is in our case is an instance of
Cpp2Request, whose destructor decrements the number of active requests
(https://fburl.com/diffusion/ylp2108m).
This diff actually does two things. It adds back the changes that were added in
D20763778, and also adds a fix that explicitly deletes ReplyState structure.
For convinience, you can see just the fix in D22017238.
Previously we were deleting ReplyState in send_reply function, however now
freeReplyState function should be explicitly called from the Rust code, which
should also fix a memory leak we had when `processor.call()` method returned an error
(https://fburl.com/diffusion/ymnwc3jx).
This is quite unfortunate that we have to do a manual actions to clean
up the memory - I'll try to improve it in the next diff.
Reviewed By: markbt
Differential Revision: D22018113
fbshipit-source-id: 2e9762bf7300129ad9e58e39fb93fca6f9505159
Summary:
Add logging of infinitepush (draft) commits to a separate scribe category.
The logging will also include the username and hostname of the pusher. Since
this code is shared with the public commits scribe logging, that logging will
also gain this information.
Reviewed By: farnz
Differential Revision: D21742656
fbshipit-source-id: bdbfd14db9e8aae190c634ac4bfff35b3f62bbe4
Summary: Found this wasn't used while looking at prefixblob usages
Reviewed By: farnz
Differential Revision: D22034647
fbshipit-source-id: a0d7dc02dbee9fdf685df464539323d4c200a818
Summary: incremental step on the path to modernization of this module
Reviewed By: StanislavGlebik
Differential Revision: D21937310
fbshipit-source-id: 6ae5f9eb43355efee35705cce96736bfcb19c8ca
Summary:
NOTE: this diff doesn't change the public API of the trait - that's why we do
`compat()` and `boxify()`. This change will follow later in the stack.
Reviewed By: StanislavGlebik
Differential Revision: D21937308
fbshipit-source-id: 11bef4c5087a9ac06e209685d3281483b804c394
Summary:
I'm not sure why but D14405696 added this `is_ancestor` method with a
duplicate ancestry check implementation which:
* has different behaviour when `ancestor == descendant`
* doesn't log to ODS
Let's just settle on using one implementation now.
Reviewed By: StanislavGlebik
Differential Revision: D21937312
fbshipit-source-id: 0f4e2390cb0b984f92cd4fe645d9313fa4bba70e
Summary: incremental step on the path to modernization of this module
Reviewed By: StanislavGlebik
Differential Revision: D21937311
fbshipit-source-id: 9db84c7cb166084edfc5a0638c1c884964f9984c
Summary: incremental step on the path to modernization of this module
Reviewed By: StanislavGlebik
Differential Revision: D21937309
fbshipit-source-id: 61b1b4302782cea35d3f65e6fe982f530293dd81
