Summary:
This reverts D24066918 (a684e53a31). That diff changed the phabstatus code to use the last
created revset iterator rather than the first for peekahead purposes.
The first revset iterator does appear to be the correct one to wrap. After
D24066918 (a684e53a31) I consistently see `hg log -r 'draft()' -T'{phabstatus}\n'` make
Phabricator queries for only one diff at a time. Reverting the changes from
that diff fixes the issue.
Note that the revset argument passed to `hg log` appears to make a difference
in the behavior here. When using simple revsets including just specific
commit ranges then the peekahead behavior works as expected both before and
after D24066918 (a684e53a31). However, when using `draft()` in the revset specifier then
the behavior works before D24066918 (a684e53a31), but not after it.
Reviewed By: singhsrb
Differential Revision: D24269910
fbshipit-source-id: c2c2abf6360fe3129a1fc73b5b433b763bce1fd2
Summary:
In D19023924 (e9df42c192) ikostia allowed non-prefix free movers, however he left the
safeguard in just in case. Well, now we finally need to use non-prefix-free
mapping, so let's remove the safeguard.
Reviewed By: ikostia
Differential Revision: D24277040
fbshipit-source-id: 4d658ad813171ab0dcb23656e95e3e443ec9961a
Summary:
This is a tool that's handy to use right before the binding. It accepts a list
of large repo changesets, and for each commit does not yet have a mapping it
inserts a NotSyncCandidate. This is exactly what we want to do right before the
bind - we'd like large repo to have most of its commits marked as
NotSyncCandidate.
Reviewed By: ikostia
Differential Revision: D24276642
fbshipit-source-id: b040e5dee981c06da8a4c58dacbb8c2660f0917d
Summary:
The infinitepush sqlindex was returning bytes instead of strings, which
caused various formatters to serialize as "b'sadf'" instead of "sadf". Let's fix
the type converter to produce strings on python 3. Note, this is the opposite to
what we do in the hgsql converter. hgsql works with binary data, so we default
to bytes there. The sqlindex doesn't deal with any binary data, so defaulting to
strings is better.
Also updates a comment on the hgsql converter to be accurate. I confirmed there
are no other customer converters in the code base that need updating.
Reviewed By: krallin
Differential Revision: D24282899
fbshipit-source-id: 708732c17626f6ebc2c3cbe7f00afa5771f31d7b
Summary:
Likely `MPath` constructor got more clever since this was written. In any case
I am just happy that it builds.
Reviewed By: farnz
Differential Revision: D24278551
fbshipit-source-id: 2378a43bd01b1826b7f81d294a035053fba5322f
Summary:
Bookmark filler doesn't make much sense outside of FB. In fact the commit
filler is already in the `facebook/` dir.
D24253307 contains the fbpkg change that has to be landed in-sync with this one.
Reviewed By: lukaspiatkowski
Differential Revision: D24253070
fbshipit-source-id: 52734ae34779801b4cae4882a6d0880586ef505f
Summary:
Include the client correlator string from the `clienttelemetry` extension in each EdenAPI HTTP request via the `X-Client-Correlator` header.
The `ClientIdentityMiddleware` in `gotham_ext` already understands this header (as it is already used by the LFS server), and `gotham_ext`'s `ScubaMiddleware` will automatically include the provided correlator in the server's Scuba samples.
Reviewed By: farnz
Differential Revision: D24282244
fbshipit-source-id: 13d04e706eda38893cff6e740bd1d7bf104e43dd
Summary: Add option to redirect the eden rage output to stderr
Reviewed By: chadaustin
Differential Revision: D24210031
fbshipit-source-id: 736af81de6a41e49c2459a1cd8124123d6928323
Summary: For the purposes of megarepo, we need to treat certain paths differently depending on whether the push is native to a large repo, or is push-redirected from a small repo. Let's add a separate patterns list to the `deny_files` hook. Of course, we can also add a list, specific for push-redirected pushes, but it seems unneeded atm.
Reviewed By: StanislavGlebik
Differential Revision: D24278409
fbshipit-source-id: 9fd815940bb656ceac6ab234f3a0647a5c57db06
Summary:
If we fail to read from MySQL, we immediately go back to attempting to read.
That's the exact opposite of what we should be doing. This fixes that by not
changing our poll schedule if we hit an error.
Reviewed By: farnz
Differential Revision: D24279086
fbshipit-source-id: 2c05ec9c33a1e0cbdfb63dda63a68f2a93615512
Summary:
This diff adds all third party dependencies that are required by getdeps to be able to build and runn Mononoke's integration tests.
Also add a stub Makefile with no-op steps that will be filled in next diff.
Reviewed By: ahornby
Differential Revision: D24251894
fbshipit-source-id: 67384ecfd0ced6762dddc3c6e61feb1240b1162d
Summary: Reorder scrub walk expansion to reduce queue depth. The bounded_traversal_stream inserts to its unscheduled queue at the front, therefore reversing the order of the children, so we want to add the easiest to complete Nodes to the list of edges last.
Reviewed By: farnz
Differential Revision: D24247625
fbshipit-source-id: 59caa5898e7f38f41cc04a15723370de38f8474f
Summary:
Delegate chaosblob put to put_impl to remove some duplication.
This is possible as all chaosblob construction has an inner BlobstorePutOps.
Reviewed By: StanislavGlebik
Differential Revision: D24258401
fbshipit-source-id: dee9fce888e5ef9c2f34865c97921b9cc87ac3bf
Summary:
This change adds some wiring to allow the hooks to treat native vs push-redirected pushes differently. This is needed because `deny-files` needs to block native pushes to `.ovrsource-rest` and `arvr-legacy`, while it needs to allow push-redirection into these directories.
The plan further is to change the actual hook body for `deny_files` to have different handling of the two cases.
Reviewed By: StanislavGlebik
Differential Revision: D24257454
fbshipit-source-id: 2f5931149115210aeeeebb3294a6512effd36350
Summary:
The logic inside `eden/scm/tests/features.py` script enables certain
features of HG on tests based on the name of the test. Mononoke's integration
tests suit reuses eden/scm's tests suit and as a consequence it triggers the
same `features.py` logic. It was fine until D24201934 (f5988c415c) introduced a feature for
test that is named in the same way as one of the Mononoke tests - the
`test-clienttelemetry.t`.
In order to fix this problem Mononoke will pass a `--nofeatures` flag to the
`eden/scm/run-tests.py` invocations that will turn off the usage of
`features.py`.
Reviewed By: farnz
Differential Revision: D24276294
fbshipit-source-id: eb28ed55a05de3b012e37407603c2370adaaad16
Summary:
mononoke_api crate which provides high-level abstraction over most of mononoke
has been used in externally facing scs_server so far. That's why all of it's entry
points are validating user identity first.
In D24136239 (42a783999d) I've started using it for iternal sync job. In this world all
operations are authorized and there's no need for extra ACL checks.
I could just add additional arg to the new method but I don't want to make it
easy to accidentally skip ACL checks - that's why I went for separate method
with descriptive namea.
Reviewed By: krallin
Differential Revision: D24261117
fbshipit-source-id: 32f601d970d286cd383ba5c488596d64d09a535e
Summary: Factor out common async limit code in throttledblob to throttled_access function.
Reviewed By: StanislavGlebik
Differential Revision: D24254357
fbshipit-source-id: 8f3277ccc065e3eb93f089edb2f3bd12e80ef352
Summary: Add --blobstore-put-behaviour argument so we can try running some workloads with modified behaviour, firstly with logging on overwrites to gather metrics, and then not able to overwrite at all.
Reviewed By: StanislavGlebik
Differential Revision: D24109292
fbshipit-source-id: bbea31eb40604fdedc3a0db7a84e99b5b1fe7a23
Summary:
Remove BlobstorePutOps::put_behaviour and the default implementation of BlobstorePutOps::put_with_status as they did not make sense for multiplexed stores
This resolves the corresponding TODO in multiplexblob/base.rs
Reviewed By: StanislavGlebik
Differential Revision: D24162629
fbshipit-source-id: aa175bbedac473093dd1862226e910dea15c1299
Summary:
Implement BlobstorePutOps for multiplex blobstores, so restoring the ability to have multiplexes of multiplexes.
Note that this makes BlobstorePutOps::put_behaviour accessor problematic as the inner stores could have different put_behaviour, will remove it in next diff so as to keep this diff reasonable size.
Reviewed By: StanislavGlebik
Differential Revision: D24162235
fbshipit-source-id: 2ace3af5f60607996e449451316c5c0720351f82
Summary:
Add overwrite status logging to multiplexblob puts now that the inner types held all implement BlobstorePutOps
This has the effect of making any configuration that specifies a multiplex of multiplexes invalid, which is addressed in next diff in stack.
Reviewed By: StanislavGlebik
Differential Revision: D24159958
fbshipit-source-id: dd1f70a636dfb36686d796af7afd8d5da8797a23
Summary: Delegate put to put_with_status for logblob now that the blobstore factory can provide right inner type.
Reviewed By: StanislavGlebik
Differential Revision: D24109296
fbshipit-source-id: 1493421f632708d8ec24bd9f7c5cd5745f17ae68
Summary: Delegate put to put_with_status for packblob now that the blobstore factory can provide right inner type
Reviewed By: StanislavGlebik
Differential Revision: D24109288
fbshipit-source-id: 745fe3f6398833c9ac51e70f4d1c2959fceba0d5
Summary:
Make blobstore_factory PutBehaviour aware by layering all except the final multiplex as BlobstorePutOps
This makes it so all the components that go into a multiplex are BlobstorePutOps, which is a prerequisite for making the multiplex logging include the Overwrite status.
Reviewed By: StanislavGlebik
Differential Revision: D24109289
fbshipit-source-id: 23f4cedbaebadae194e41cfbff9ef46b651e3fd4
Summary:
log whether puts overwrite keys by implementing BlobstoreWithPutBehaviour for Logblob.
It logs a count of type of overwrite per put so we can sum them up.
Reviewed By: StanislavGlebik
Differential Revision: D24079272
fbshipit-source-id: 81944d92a56b0d3349ef390eb83f9e5bf4ee3d39
Summary:
If Phabricator takes more than 10 seconds to respond, a `socket.timeout`
exception may be thrown. Treat this like other networking errors, and simply
report the diff phabricator status as `Error`. Previously this exception was
unhandled, causing the entire command to abort.
Reviewed By: singhsrb
Differential Revision: D24272274
fbshipit-source-id: f646d111a91f901e09d9f94a1e0102d6dd4d0952
Summary:
This change introduces two new metadata types, Category and Transience, and a mechanism for Category to provide a default Fault and Transience, which can be overriden by the user.
Also introduces a mechanism for attempting to log exceptions which occur during exception logging, falling back to the previous behavior of just swallowing the exception on failure.
Reviewed By: DurhamG
Differential Revision: D22677565
fbshipit-source-id: 1cf75ca1e2a65964a0ede1f072439378a46bd391
Summary:
It only has benchmark code that led to the use of mincode. Now hgcommits is the
main crate for commit storage. `commitstore` without `hg` in its name was
initially planned to support other kinds of commits including git and bonsai.
However we don't have immediate goal for that at present. So let's just remove
the commitstore directory.
Reviewed By: singhsrb
Differential Revision: D24263618
fbshipit-source-id: 84b4861ae490817377e69d8c2006c63331e3db1f
Summary:
On the hg servers we're seeing crashes due to signals during syscalls.
Rolling back to prior to the signal changes seems to have fixed it, though we
haven't bisected enough to be sure this is the cause.
The ui.threaded option is already used to opt-out of running on a background
thread, let's also use it to opt-out of the custom signal registering in hopes
that it fixes the server issue, until they can be deprecated.
Reviewed By: quark-zju
Differential Revision: D24254804
fbshipit-source-id: 50e4fc8c7e3a88b5baa49394f6f1edffc946187d
Summary:
Since D23247941, .clang-format is no longer behind a symlink, which means that
we can get clang-format niceness on Windows without having our own
.clang-format file.
Thus, let's just remove eden/scm/.clang-format
Reviewed By: ahornby
Differential Revision: D24255987
fbshipit-source-id: 8bbb966949cf0d1c0ad76053f699dd524700183e
Summary: Need to add new quickcheck tests, verify that remove `Serialize` from `TreeEntry` is okay.
Reviewed By: kulshrax
Differential Revision: D23457777
fbshipit-source-id: aa94ed7aa81b41924eba4a8bd1bdc2c737365b77
Summary:
Change
abort: repository repo: timed out waiting for lock held by <lockinfo ...>
to:
abort: repository repo: timed out waiting for lock held by process '842210' on host 'hostname'
Reviewed By: singhsrb
Differential Revision: D24214462
fbshipit-source-id: 65056ebb9764651b2f0126061fafdfdefaa4e9c6
Summary: The rev numbers are almost gone, remove them from the test.
Reviewed By: sfilipco
Differential Revision: D24255156
fbshipit-source-id: 5cbc4a71c2d3f773c5b474d1edce84ceceb50bf9
Summary: Added sqlblob to the common blobstore tests to make sure it doesn't diverge from expectations for other stores.
Reviewed By: StanislavGlebik
Differential Revision: D24079254
fbshipit-source-id: 49ef1c372171a1594ba298c66d3473ef682d24cc
Summary:
Add CountedBlobstoreOps so that blobstore layers that need BlobstoreOps can still use counting
This unblocks adding sqlblob to blobstore-test in next diff in stack
Reviewed By: farnz
Differential Revision: D24079256
fbshipit-source-id: 6a6505aff8c8405353a1f10d79f6e6e08911228a
Summary: Add BlobstorePutOps so that blobstore layers that need BlobstorePutOps can still use PrefixBlob as a wrapper.
Reviewed By: farnz
Differential Revision: D24109298
fbshipit-source-id: 710571e6c30fa8a432d463eedfab5fcc0389baa3
Summary:
Add predicate based PutBehaviour logic to manifoldblob.
This will prevent overwrites of keys when in IfAbsent mode, and will generate useful logging in OverwriteAndLog and IsAbsent mode.
This change factors our part of the put logic to put_check_conflict, so that it can use re-used from each of the PutBehaviour cases.
Reviewed By: StanislavGlebik
Differential Revision: D24021170
fbshipit-source-id: d2e71afadada3d5e661634449108e6c9f8dc5907
Summary:
We don't have any Preserved entries anymore - now all preserved entries will be
rewritten with "noop" mapping.
This diff removes it completely
Reviewed By: mitrandir77, ikostia
Differential Revision: D24173538
fbshipit-source-id: f2d6238633cea8dc3c06f2e607b2abd76edfca6b
Summary: This state is going to be removed soon, so no need for tests anymore
Reviewed By: mitrandir77
Differential Revision: D24221363
fbshipit-source-id: 19dce04549ccbfe59255463a73e56c70f1c8bc4d
