Summary:
Use the new API to clean up stale logs at open time. This hopefully helps
releasing disk space on Windows if the normal rotation fails to remove
old files being mmap-ed by other processes.
Reviewed By: xavierd
Differential Revision: D25894282
fbshipit-source-id: a3d8247b737dd451ee68b58cc5a38fdd2822c0c3
Summary:
Previously rotation only happens at flush time and file deletion is a best
effort (it might fail on Windows). For use-cases that are sensitive about
space usage that's suboptimal. This diff adds an API to remove old files
manually so high level logic can choose to clean up old files after open().
Reviewed By: xavierd
Differential Revision: D25894283
fbshipit-source-id: fbffff426544b39349ddf3537d46954d3cab5d12
Summary:
Previously, the fsmonitor state update logic will skip updating treestate if the wlock
cannot be obtained. D17468790 (8d4d0a66a2) made it wait for wlock for the painful "watchman fresh
instance" case. But things can still suck if it's not a "fresh instance" but there are just
too many nonnormal files.
This diff makes it that exceeding a threshold of nonnormal files will trigger a fsmonitor
state write as an attempt to to reduce the number of nonnormal files. In additional,
`--debug` was changed to print more internal states for debugging.
This would hopefully address issues where people have a large "nonnormal"
treestate, suffers from the bad performance issue and cannot recover from it
automatically.
Reviewed By: DurhamG
Differential Revision: D25794083
fbshipit-source-id: 741426cf31484d9318f9cfcab11d38da33ab5067
Summary:
We are going to write docs on how to unbind repositories and bind them later.
This integration test is a "test for documentation" i.e. it should run the same
steps as we'd run ourselves whenever we need to rebind.
The key part here is that we do a merge of unbound small repo into large repo commits
Reviewed By: ikostia
Differential Revision: D25883873
fbshipit-source-id: fac1d7871e52f4e1aa7f15d32d39f2755b803cd3
Summary:
Add a command that will unconditionally rewrite commits from one repo to
another. This is similar to what x-repo lookup thrift api does, however this
command doesn't do a few safety checks (in particular, it allows to rewrite
public commits).
This is going to be used for unbinding procedure.
Reviewed By: ikostia
Differential Revision: D25883874
fbshipit-source-id: a40eda9aa40ef7ad63e2403d956871940dd1601d
Summary: We don't use obsstore. Its size is irrelevant.
Reviewed By: singhsrb
Differential Revision: D25876066
fbshipit-source-id: 5657c3ca08f5ed1cb5a3d1a5b3395ab74756b7e5
Summary: It's used for malloc / realloc which take size_t instead of unsigned int.
Reviewed By: StanislavGlebik, xavierd
Differential Revision: D25890562
fbshipit-source-id: e2787786e9b995431c50b411d77cbce438a82c98
Summary: This is a bit nicer to read
Reviewed By: ahornby
Differential Revision: D25881919
fbshipit-source-id: 3c97390a96410a18e8fdc6cb6279b2d46e407cd3
Summary:
I think it's nice to group them together. Also while there I sorted both groups
lexicographically
Reviewed By: ahornby
Differential Revision: D25881918
fbshipit-source-id: 05bb9f07ee7799c3d5c19a8ffaabadaca635fef2
Summary: Certain doctor items are not meaningful outside Facebook. This diff adds the ability to have Facebook-specific doctors.
Reviewed By: xavierd
Differential Revision: D25789275
fbshipit-source-id: 94160be741a8fc3e8d01e86beaa0d2428952db21
Summary:
Before this diff, we did DNS lookups using a crate called `dns_lookup`. This crate is a thin layer over libc DNS lookups. Those lookups are blocking (i.e. they hold a thread), so they're not very friendly to asynchronous code. We currently offload them on a dedicated thread pool to mitigate the issue, but this isn't ideal: if we experience e.g. slow DNS responses, we could saturate this thread pool pretty easily.
I updated it to use the trust-dns crate, which provides an asynchronous implementation of DNS lookups, and is currently used in other parts of Mononoke.
Reviewed By: krallin
Differential Revision: D25849872
fbshipit-source-id: 826ab4e5618844f3b48e5def4ad9bd163753ebb1
Summary:
D25850262 (772680f2f7) updated a dep but only did a partial regen. This broke
other diffs.
This diff does a full regen to sync up again.
Differential Revision: D25872444
fbshipit-source-id: b5749b3d634ffe355a5a3cf6e94066ca5158001f
Summary:
The `pull` commmand has a lot of tech debt (with issues like inefficiency, race
conditions, named branches, etc). The new `repo.pull` API isn't designed to
support all legacy usecases.
This diff switches a subset of `pull` command that the new API can support to
the new API. It should solve race condition or `visibility.add` inefficiency
issues (ex. 20s on addgroup, 187s on visibility.add - P154836357).
Logic related to remotenames was ported from the remotenames extension.
The selectivepull tests seem strong enough to check their behaviors.
The new pull API is used by commit cloud for many months. So I turned the
new code path on by default. It can be turned off by changing
`commands.new-pull`.
There are a few test changes. The new behavior seems more correct overall:
- test-commitcloud-switch-workspace.t
- "M/F" swap is caused by rev number: both are correct
- "S: public remote/stable" is more correct than "S: draft"
- test-commitcloud-sync-rb-deletion.t
- "draft1: draft remote/scratch/draft1" is more correct because
"remote/scratch/draft1" is listed in the "book --list-subs"
command above.
- test-commitcloud-sync-rb-enabling.t
- "public1: draft" not showing up is more correct.
- test-fb-hgext-remotefilelog-lfs-prefetch.t
- Difference seems to be caused by rev order.
Reviewed By: markbt
Differential Revision: D25562089
fbshipit-source-id: ac22b2f0492ab53517d580d706dfbc823fd0e0cc
Summary:
Make the `pull` API update visible heads even if all commits exist locally.
This is a more expected behavior, and will make the "pull" command using the
pull API simpler.
Reviewed By: markbt
Differential Revision: D25562091
fbshipit-source-id: 8a43cfe4afd31d3cb9ad5369a6081de120043168
Summary:
Filters out some obviously public heads at the end of repo.pull so they don't
get passed to `visibility.add`. Note: this just removes some obviously "public"
commits without considering the graph. A stronger guarantee would be removing
public heads at metalog commit time.
Reviewed By: markbt
Differential Revision: D25562092
fbshipit-source-id: cc6a19252fcfe305e3a14895b61ab0d6b84a007e
Summary: Update name to match usage of Try as tri-state, since this method also throws if the Try is empty
Reviewed By: yfeldblum
Differential Revision: D25737810
fbshipit-source-id: a4166153362f07353d212216fbaf7105867eef2a
Summary:
On Windows, since Mercurial doesn't yet build with Buck, we need to test
against the system Mercurial, thus remove the dependencies to //eden/scm:hg for
the tests. Also remove various dependencies that don't build yet on Windows.
This allows for the tests to run, but fail while trying to execute edenfsctl.par.
Reviewed By: kmancini
Differential Revision: D25807727
fbshipit-source-id: c2533eedc361cc6db9fdf2190476c3d52833139d
Summary: Add a method to get just the ids. This is a new best case for the fetching, so updated the benchmark as well.
Reviewed By: StanislavGlebik
Differential Revision: D25804027
fbshipit-source-id: ccc4573c8a4ebc07db854a0ffa737f572087019e
Summary: Simplify the tests for bulkops, no need to map to hg to check the expected changesets. Can compare to expected bonsai directly instead as its simpler
Reviewed By: StanislavGlebik
Differential Revision: D25804020
fbshipit-source-id: eb4381c37ed6a4fb1e213f0397ffb2136ddee473
Summary: When scrubbing repos it is preferable to scrub newest data first. This diff adds Direction::NewestFirst to bulkops for use in scrubbing and updates existing call sites to Direction::OldestFirst so as not to change behaviour
Reviewed By: StanislavGlebik
Differential Revision: D25742279
fbshipit-source-id: 363a4854b14e9aa970b2c1ec491dcaccac7a6ec9
Summary: Preparation for adding OldestFirst vs NewestFirst direction of fetching in next diff.
Reviewed By: StanislavGlebik
Differential Revision: D25742281
fbshipit-source-id: 655f297efc2094d4325717d97cce53e697c35597
Summary: Add a benchmark tool for bulkops that uses the criterion benchmarking framework. This is so we can measure effect of optimizations later in stack.
Differential Revision: D25804026
fbshipit-source-id: 71b8addf1145c0ecb69d6392b4602172f2b52080
Summary:
We used to get those in the old (Python) LFS extension, but didn't have them in
the new one. However, this is helpful to correlate requests to LFS with data in
hg logs. It's also convenient to be able to identify whether a set of requests
are part of the same session or not.
This diffs threads the client correlator through to the LFS store from the
Python, similarly to how it's done in EdenAPI.
Reviewed By: DurhamG
Differential Revision: D25804930
fbshipit-source-id: a5d5508617fa4184344834bbd8e3423816aa7668
Summary: The dependency that we want to share is 0.14 instead of 0.15.
Reviewed By: singhsrb
Differential Revision: D25871110
fbshipit-source-id: 16e9f8a858ee04a47867c2916909edfc996f8bc4
Summary:
Add `manifest_node` to the crdump output, which is the root manifest node of the commit.
This is useful for detecting commits that have the same tree content but
different metadata (e.g. if only the commit message has been edited).
Reviewed By: singhsrb
Differential Revision: D25782674
fbshipit-source-id: dfdf426833533140b676eee82e123a0cba23c77a
Summary:
We had a mix of callsites using fetch_all_public_changesets directly vs those using the PublicChangesetBulkFetch::fetch.
Update callsites to use PublicChangesetBulkFetch::fetch for consistency. This also has the nice side effect to removing some explicit config_store usage.
Reviewed By: StanislavGlebik
Differential Revision: D25804019
fbshipit-source-id: 5a88888dd915d1d693fb26ffe3bb359c9e918d5c
Summary:
Right now, we have zero visibility HTTP level errors (such as broken pipes),
because this is all set up on Gotham with no logging. This diff moves the
bootstrapping from Gotham to us to fix this.
There's a bit of code I'd like to deduplicate here, but it's tricky to do given
that the code is a little different in the HTTP vs. HTTPS branches. For now,
this will give us some logging we need without too much effort. We can make it
more robust (and route it to Scuba or give it session IDs) if this proves
useful.
Reviewed By: StanislavGlebik
Differential Revision: D25851426
fbshipit-source-id: 4ca5d1ecb3931715f04af735aa1b7cfdac87846d
Summary:
This updates Gotham. Under the hood I rebased our fork, you can see the diff
here: P161171514.
The stuff that is relevant is that Gotham got rid of its dependency on
`failure` and now uses `anyhow` instead, and I also added a little bit to our
existing socket data patch by making a few things public so that we can get
access to a few more internals.
Reviewed By: StanislavGlebik
Differential Revision: D25850262
fbshipit-source-id: 25ebf5d63be39e3e93208705d91abc5c61c90453
