Summary: Factor out common async limit code in throttledblob to throttled_access function.
Reviewed By: StanislavGlebik
Differential Revision: D24254357
fbshipit-source-id: 8f3277ccc065e3eb93f089edb2f3bd12e80ef352
Summary: Add --blobstore-put-behaviour argument so we can try running some workloads with modified behaviour, firstly with logging on overwrites to gather metrics, and then not able to overwrite at all.
Reviewed By: StanislavGlebik
Differential Revision: D24109292
fbshipit-source-id: bbea31eb40604fdedc3a0db7a84e99b5b1fe7a23
Summary:
Remove BlobstorePutOps::put_behaviour and the default implementation of BlobstorePutOps::put_with_status as they did not make sense for multiplexed stores
This resolves the corresponding TODO in multiplexblob/base.rs
Reviewed By: StanislavGlebik
Differential Revision: D24162629
fbshipit-source-id: aa175bbedac473093dd1862226e910dea15c1299
Summary:
Implement BlobstorePutOps for multiplex blobstores, so restoring the ability to have multiplexes of multiplexes.
Note that this makes BlobstorePutOps::put_behaviour accessor problematic as the inner stores could have different put_behaviour, will remove it in next diff so as to keep this diff reasonable size.
Reviewed By: StanislavGlebik
Differential Revision: D24162235
fbshipit-source-id: 2ace3af5f60607996e449451316c5c0720351f82
Summary:
Add overwrite status logging to multiplexblob puts now that the inner types held all implement BlobstorePutOps
This has the effect of making any configuration that specifies a multiplex of multiplexes invalid, which is addressed in next diff in stack.
Reviewed By: StanislavGlebik
Differential Revision: D24159958
fbshipit-source-id: dd1f70a636dfb36686d796af7afd8d5da8797a23
Summary: Delegate put to put_with_status for logblob now that the blobstore factory can provide right inner type.
Reviewed By: StanislavGlebik
Differential Revision: D24109296
fbshipit-source-id: 1493421f632708d8ec24bd9f7c5cd5745f17ae68
Summary: Delegate put to put_with_status for packblob now that the blobstore factory can provide right inner type
Reviewed By: StanislavGlebik
Differential Revision: D24109288
fbshipit-source-id: 745fe3f6398833c9ac51e70f4d1c2959fceba0d5
Summary:
Make blobstore_factory PutBehaviour aware by layering all except the final multiplex as BlobstorePutOps
This makes it so all the components that go into a multiplex are BlobstorePutOps, which is a prerequisite for making the multiplex logging include the Overwrite status.
Reviewed By: StanislavGlebik
Differential Revision: D24109289
fbshipit-source-id: 23f4cedbaebadae194e41cfbff9ef46b651e3fd4
Summary:
log whether puts overwrite keys by implementing BlobstoreWithPutBehaviour for Logblob.
It logs a count of type of overwrite per put so we can sum them up.
Reviewed By: StanislavGlebik
Differential Revision: D24079272
fbshipit-source-id: 81944d92a56b0d3349ef390eb83f9e5bf4ee3d39
Summary:
If Phabricator takes more than 10 seconds to respond, a `socket.timeout`
exception may be thrown. Treat this like other networking errors, and simply
report the diff phabricator status as `Error`. Previously this exception was
unhandled, causing the entire command to abort.
Reviewed By: singhsrb
Differential Revision: D24272274
fbshipit-source-id: f646d111a91f901e09d9f94a1e0102d6dd4d0952
Summary:
This change introduces two new metadata types, Category and Transience, and a mechanism for Category to provide a default Fault and Transience, which can be overriden by the user.
Also introduces a mechanism for attempting to log exceptions which occur during exception logging, falling back to the previous behavior of just swallowing the exception on failure.
Reviewed By: DurhamG
Differential Revision: D22677565
fbshipit-source-id: 1cf75ca1e2a65964a0ede1f072439378a46bd391
Summary:
It only has benchmark code that led to the use of mincode. Now hgcommits is the
main crate for commit storage. `commitstore` without `hg` in its name was
initially planned to support other kinds of commits including git and bonsai.
However we don't have immediate goal for that at present. So let's just remove
the commitstore directory.
Reviewed By: singhsrb
Differential Revision: D24263618
fbshipit-source-id: 84b4861ae490817377e69d8c2006c63331e3db1f
Summary:
On the hg servers we're seeing crashes due to signals during syscalls.
Rolling back to prior to the signal changes seems to have fixed it, though we
haven't bisected enough to be sure this is the cause.
The ui.threaded option is already used to opt-out of running on a background
thread, let's also use it to opt-out of the custom signal registering in hopes
that it fixes the server issue, until they can be deprecated.
Reviewed By: quark-zju
Differential Revision: D24254804
fbshipit-source-id: 50e4fc8c7e3a88b5baa49394f6f1edffc946187d
Summary:
Since D23247941, .clang-format is no longer behind a symlink, which means that
we can get clang-format niceness on Windows without having our own
.clang-format file.
Thus, let's just remove eden/scm/.clang-format
Reviewed By: ahornby
Differential Revision: D24255987
fbshipit-source-id: 8bbb966949cf0d1c0ad76053f699dd524700183e
Summary: Need to add new quickcheck tests, verify that remove `Serialize` from `TreeEntry` is okay.
Reviewed By: kulshrax
Differential Revision: D23457777
fbshipit-source-id: aa94ed7aa81b41924eba4a8bd1bdc2c737365b77
Summary:
Change
abort: repository repo: timed out waiting for lock held by <lockinfo ...>
to:
abort: repository repo: timed out waiting for lock held by process '842210' on host 'hostname'
Reviewed By: singhsrb
Differential Revision: D24214462
fbshipit-source-id: 65056ebb9764651b2f0126061fafdfdefaa4e9c6
Summary: The rev numbers are almost gone, remove them from the test.
Reviewed By: sfilipco
Differential Revision: D24255156
fbshipit-source-id: 5cbc4a71c2d3f773c5b474d1edce84ceceb50bf9
Summary: Added sqlblob to the common blobstore tests to make sure it doesn't diverge from expectations for other stores.
Reviewed By: StanislavGlebik
Differential Revision: D24079254
fbshipit-source-id: 49ef1c372171a1594ba298c66d3473ef682d24cc
Summary:
Add CountedBlobstoreOps so that blobstore layers that need BlobstoreOps can still use counting
This unblocks adding sqlblob to blobstore-test in next diff in stack
Reviewed By: farnz
Differential Revision: D24079256
fbshipit-source-id: 6a6505aff8c8405353a1f10d79f6e6e08911228a
Summary: Add BlobstorePutOps so that blobstore layers that need BlobstorePutOps can still use PrefixBlob as a wrapper.
Reviewed By: farnz
Differential Revision: D24109298
fbshipit-source-id: 710571e6c30fa8a432d463eedfab5fcc0389baa3
Summary:
Add predicate based PutBehaviour logic to manifoldblob.
This will prevent overwrites of keys when in IfAbsent mode, and will generate useful logging in OverwriteAndLog and IsAbsent mode.
This change factors our part of the put logic to put_check_conflict, so that it can use re-used from each of the PutBehaviour cases.
Reviewed By: StanislavGlebik
Differential Revision: D24021170
fbshipit-source-id: d2e71afadada3d5e661634449108e6c9f8dc5907
Summary:
We don't have any Preserved entries anymore - now all preserved entries will be
rewritten with "noop" mapping.
This diff removes it completely
Reviewed By: mitrandir77, ikostia
Differential Revision: D24173538
fbshipit-source-id: f2d6238633cea8dc3c06f2e607b2abd76edfca6b
Summary: This state is going to be removed soon, so no need for tests anymore
Reviewed By: mitrandir77
Differential Revision: D24221363
fbshipit-source-id: 19dce04549ccbfe59255463a73e56c70f1c8bc4d
Summary: Just as with the previous diff, let's remove it from cross_repo_sync_test_utisl
Reviewed By: mitrandir77
Differential Revision: D24220618
fbshipit-source-id: 95c5ddc955f720101f6576b34e9787435b6deb4c
Summary:
This will allow Eden to control what data is in Mercurial's memory and
what is on disk. This will let it persist the hg_import_helper process longer,
and prevent slow startup times when needing to restart it.
Reviewed By: xavierd
Differential Revision: D24231131
fbshipit-source-id: a4f743740b44957e8d2dd93f07e9831eadfee7ab
Summary:
I observed an integration test fail because Mercurial aborted with an
error about not handling EINTR, but it had no traceback. Always run
Mercurial with --traceback to try to debug these.
Reviewed By: xavierd
Differential Revision: D24204308
fbshipit-source-id: 44960bc645e5f97f61761e511b372328430fcda7
Summary:
This was added in D16078908 (1bc6ecf8fe). It turns out fsync followed by a hardlink
on btrfs results in very slow hardlink performance (100-600ms). Since fewer and
fewer of our files use this atomic write code path and since this affects almost
every hg write command, let's roll this back.
This will increase the chance of data loss during a hard reboot, but commit
cloud is good enough to allow recovering from that in almost every situation.
Reviewed By: sfilipco
Differential Revision: D24230056
fbshipit-source-id: aae1a5612eda5f62bb5ec1442b1312ed45c42786
Summary:
With invalidatelinkrev, linkrev stored in revlog shouldn't be used.
This makes some tests pass with segmented changelog.
Reviewed By: singhsrb
Differential Revision: D24201944
fbshipit-source-id: 6473c30266c79aa97a955b1a6c867411cc67de2b
Summary:
The old code assumes `0..len(repo)` are valid revs, which is no longer true
with segmented changelog.
Reviewed By: singhsrb
Differential Revision: D24201948
fbshipit-source-id: b882a215701c57ccdf4af340c889586b040772da
Summary:
Copy segmented changelog, metalog and mutation store for local clones.
This mainly affects tests.
Reviewed By: singhsrb
Differential Revision: D24201941
fbshipit-source-id: c60da9e2bf982a6f66004415e45178749157745e
Summary: This is needed to move our hg servers to python 3.
Reviewed By: quark-zju
Differential Revision: D24204056
fbshipit-source-id: cbaf97893f8f77b535952ac290766f0fd5e14f0c
Summary:
From an OSS perspective, I think that the log tables have a place. However for
daily use perspective, next to scuba they don't add much except retention and
instead feel more heavy weight to manage. This change probably simplifies
things and makes the Segmented Changelog component easier to maintain.
Reviewed By: krallin
Differential Revision: D24213548
fbshipit-source-id: 48a4ea57e3f3911c3bf82b0cc51f118d72119e19
Summary:
This diff is a first step in a preparation for removing Preserved state from
CommitSyncOutcome. It does two things:
1) Remove unsafe_preserve_commit method and instead rewrite commit with noop
config version - this is exactly how we are going to do it in production
2) While there also select correct mover for validation - previously we were
always picking the current mover
I've also added a bunch of print statements, since it was tricky to debug what
was going on in the test.
Reviewed By: ikostia
Differential Revision: D24216168
fbshipit-source-id: 86e81aea61e638d93bdb33e7c9fd713f7b5e6c3b
Summary:
All the new entries in our mapping tables should have their version set. Let's
enforce it in the code
Reviewed By: ikostia
Differential Revision: D24217688
fbshipit-source-id: 95f01d8929a9c3a19b84434c91db6d08a6e5f863
Summary:
To give a bit of context - we are getting rid of Preserved state in
CommitSyncOutcome completely since it can be fully replaced with with
EquivalentWorkingCopyAncestor/RewrittenAs states.
ikostia@ did a similar change in D24142837 (2035a34a0e) to consider Preserved only if
mapping is None. However this diff diff it only for mapping.get() method. Let's do
the same for mapping.get_equivalent_working_copy().
Reviewed By: ikostia
Differential Revision: D24216455
fbshipit-source-id: f1f8d46263de54cb2e11d33b6c17f371b79e80f9
