Summary:
We got a user request to add this output after a restart. This (` run "cd / && cd -" to
update your shell's working directory.`) is the output from eden doctor when it detects stale mount points. I don't think it hurts to add, especially since new customers may not make the connection/have the context to deduce that "programs" in this message also apply to the user's shell.
Reviewed By: kmancini
Differential Revision: D20971070
fbshipit-source-id: 3448651cae90b2b5c8c4d2432b946521cdc5a292
Summary:
The ID in Hgsql is supposed to the next globalrev to assign, not the last one
that was assigned. We would have otherwise noticed during the rollout since
we'd have seen that the counter in Mercurial wasn't `globalrev(master) + 1`
(and we could have fixed it up manually before it had any impact), but let's
fix it now.
Reviewed By: StanislavGlebik
Differential Revision: D21089653
fbshipit-source-id: 0a37e1b7299a0606788bd87f788799db6e3d55f4
Summary:
At the moment MemWritesBlobstore stores both writes and reads. This is not
always desirable - in particular, in the next diff I'd like to add --dry-run
mode to backfill_derived_data which would keep everything that was derived in
memory. In that case storing all data that we read from blobstore might be too
much.
Given that we don't need this functionality let's just remove it
While there also add a method to get access to the underlying cache - again, it
will be used in the next diff.
Reviewed By: krallin
Differential Revision: D21088794
fbshipit-source-id: 91c8729d748d8ad8d9a70e6f8d5e15afe5021e8c
Summary:
This adds a special mode of deriving fsnodes that will be used in
backfill_derived_data. From my experiments it looks like it got 5-10 times
faster.
I tried to explain how it works in the comments - lmk if that's not enough.
Reviewed By: mitrandir77
Differential Revision: D21067817
fbshipit-source-id: ff72a079754a2c15f65852c28d80f723961b53c4
Summary: See comments for more details
Reviewed By: krallin
Differential Revision: D21088800
fbshipit-source-id: b4c187b5d4d476602e69d26d71d3fe1252fd78e6
Summary: I think it's more readable this way
Reviewed By: krallin
Differential Revision: D21088598
fbshipit-source-id: 1608c250701ae6870094f0f61c0c2ce4e2c12ebf
Summary:
In the next few diffs I'm going to add more functionality to backfill derived
data. The file has grown quite big already, so I'd rather put this new
functionality in a separate file. This diff does the first step - it just moves
a file to a separte directory.
Reviewed By: farnz
Differential Revision: D21087813
fbshipit-source-id: 4a8e3eac4b8d478aa4ceca6bb55fa0d2973068ba
Summary:
When a file goes from being a symlink to a regular file, a regular update
action ("g") is used, and the Python code implicitely remove the symlink before
writing to it. In the Rust code, we don't and as a consequence write through
the symlink, not the intended behavior.
An alternative way of fixing this would be to perform an lstat(2) before
writing to a file, but the cost of doing that will be fairly high for a very
unlikely situation especially since the manifest diff can give us exactly this
information.
This whole merge code feels extremely fragile, so I'm definitively not sure if
I got all the places that needs updating :(.
Reviewed By: DurhamG
Differential Revision: D21082733
fbshipit-source-id: 4f36a67363915c9b67d5a0b290a226075a9f1d31
Summary:
Previously our jobs would have exit with error code 0 even if there was an
actual error. The reason for this was because error was just ignored (or rather
just printed to stderr).
This is not a huge problem but it makes tw output confusing - it shows that
the task was "Completed" while in reality it "Failed"
Reviewed By: ahornby
Differential Revision: D20693297
fbshipit-source-id: 4f615e2ef11f2edbb9bdbcf49cb1635929fdae89
Summary: For some reason, direct connections don't work from devvm4263.prn3.facebook.com, complaining about an IP address mismatch. Rather than debug that, just use MyRouter.
Reviewed By: StanislavGlebik
Differential Revision: D21089203
fbshipit-source-id: 88489a6a3b83de885c7c6e9405b325b56b807e12
Summary:
I was getting weird results from SQLBlob, which I traced to connections being closed a certain amount of time after the first query.
Fix this by recreating the blob store (thus using new connections) each time.
Reviewed By: StanislavGlebik
Differential Revision: D21089041
fbshipit-source-id: e94f8993d64dbd81d9f122f92d64aa92dad8514f
Summary: That is helpful when e.g. benchmarking on the most humongous commits.
Reviewed By: farnz
Differential Revision: D21064716
fbshipit-source-id: 62973d8e4f0352a2d963bb2e8a87bdced6dedc85
Summary:
This makes it easier to test performance on a specific set of commits. As part
of that, I've also updated our file reading to be async since why not.
Reviewed By: farnz
Differential Revision: D21064609
fbshipit-source-id: d446ab5fb5597b9113dbebecf97f7d9b2d651684
Summary:
Let's try and make sure this doesn't bitrot again by adding a smoke test. Note
that there are no hooks configured here, so this exercises everything but the
actual hook running, but for now this is probably fine.
Note that this required updating the hook tailer to use the repository config
for the hook manager, since you can't start AclChecker in a test otherwise.
Reviewed By: StanislavGlebik
Differential Revision: D21063378
fbshipit-source-id: c7336bc883dca2722b189449a208e9381196300e
Summary: There's no reason to force using Hg Changeset IDs here.
Reviewed By: StanislavGlebik
Differential Revision: D21063377
fbshipit-source-id: e4c3943449c33340159afbef819dd3dbf786bf5c
Summary:
I'd like to make sure that performance of our hooks is now linear with regard
to the changeset size after D21039811.
Reviewed By: farnz
Differential Revision: D21043172
fbshipit-source-id: a36edc5cfdb26fc63160dfdbc47be157b7506523
Summary:
Rather than buffer everything, let's stream outcomes as we go. Also, let's
track the number of changesets we accepted or rejected, as opposed to the hook
instance count (my goal is to output all that in a CSV if we want more detail).
Reviewed By: StanislavGlebik
Differential Revision: D21043173
fbshipit-source-id: 1b20339a52ac95a0a771b9ef469d19dd14ffc2c3
Summary:
We'll be adding more data to the filesystem layer, so let's move this
out of lib.rs.
Also made a slight tweak to expose File metadata in the walk results, which will be used by the future pending changes logic to avoid re-stating the file.
Reviewed By: xavierd
Differential Revision: D20546903
fbshipit-source-id: 70456055b0da601990e6d6ff535678d2df6c50ba
Summary: Use constants for flag names instead of duplicating them as hardcoded strings.
Reviewed By: xavierd
Differential Revision: D21072045
fbshipit-source-id: 4617d169d034e05dcf11eb138ad0b6eaf915edec
Summary: RSRoutingHandler is now added by default
Reviewed By: bithree
Differential Revision: D21058633
fbshipit-source-id: 7d7607a5c5c7c3a5ed70259496c5f036c6fa8e4c
Summary:
Purging treemanifest packfiles always made little sense to me, as the reason we
have tons of them, is because we fetched tons of them. Remove all of them would
just lead to the same situation again. Let's just do a repack instead.
Reviewed By: markbt
Differential Revision: D21065723
fbshipit-source-id: a7cdee1de1d7897481f75c13d8aceeb8c1a68550
Summary:
Now that we've upgraded mysql-connector-python, we can enable these
tests.
Reviewed By: quark-zju
Differential Revision: D21009185
fbshipit-source-id: e9ae62cadcc8d0a291381ab2cfb5c7bc04606d9e
Summary: Add a simple integration test for the EdenAPI server which just starts up a server and hits its health_check endpoint. This will be expanded in later diffs to perform actual testing.
Reviewed By: krallin
Differential Revision: D21054212
fbshipit-source-id: a3be8ddabb3960d709a1e83599bc6a90ebe49b25
Summary:
Add TLS session data middleware to the EdenAPI server to allow inspecting HTTPS traffic to the server for debugging purposes.
An additional side effect of setting this up is that we use `bind_server_with_socket_data` to bind to the listening socket, wherein we construct a `TlsSocketData`, which constructs a `TlsCertificateIdentities` containing the client certificates, which are later added to the server's State by Gotham. Having the client certificate information in the State is necessary in order for the ClientIdentityMiddleware to work, and will allow enforcement of ACLs based on the provided client identities.
Reviewed By: krallin
Differential Revision: D21054213
fbshipit-source-id: 7002c73b7458f21e3c4a51a3029d27d1dea7a927
Summary:
We used to implicitly do this when creating the sync queue (though it wasn't
needed there - if we don't wait we crash later when checking for replication
lag), but we no longer do after the SqlConstruct refactor.
This fixes that so now we can start the healer again.
Reviewed By: farnz
Differential Revision: D21063118
fbshipit-source-id: 24f236d10b411bc9a5694b42c19bf2afa352a54c
Summary:
Being told `Input/output error: Connection refused (os error 111)` isn't very
helpful when things are broken. However, being told:
```
Execution error: While waiting for replication
Caused by:
0: While fetching repliction lag for altoona
1: Input/output error: Connection refused (os error 111)
```
Is nicer.
Reviewed By: farnz
Differential Revision: D21063120
fbshipit-source-id: 1408b9eca025b120790a95d336895d2f50be3d5d
Summary:
This turns out quite nice because we had some futures there that were always
`Ok`, and now we can use `Output` instead of `Item` and `Error`.
Reviewed By: ahornby
Differential Revision: D21063119
fbshipit-source-id: ab5dc67589f79c898d742a276a9872f82ee7e3f9
Summary:
I'd like to do a bit of work on this, so might as well convert it to async /
await first.
Reviewed By: ahornby
Differential Revision: D21063121
fbshipit-source-id: e388d59cecf5ba68d9bdf551868cea79765606f7
Summary: I'm going to want to be able to test against a single ephemeral shard, as well as production use against a real DB. Use the standard config to make that possible.
Reviewed By: ahornby
Differential Revision: D21048697
fbshipit-source-id: 644854e2c831a9410c782ca1fddc1c4b5f324d03
Summary:
This is generally something I wanted to have for a long time: instead of having to open a writable db shell, now we can just use the admin command. Also, this will be easier to document in the oncall wikis.
NB: this is lacking the `delete` functionality atm, but that one is almost never needed.
Reviewed By: krallin
Differential Revision: D21039606
fbshipit-source-id: 7b329e1782d1898f1a8a936bc711472fdc118a96
