Summary: This diff implements the newly added methods to `IOverlay` so we can build & test `SqliteOverlay` on devserver.
Reviewed By: chadaustin
Differential Revision: D25283647
fbshipit-source-id: 50d0c2dd47cd8f74c860a131ffad1d6342a86ea2
Summary: Was fixed at 2 reads, add an option to allow testing read performance more throughly.
Reviewed By: farnz
Differential Revision: D27043234
fbshipit-source-id: 4bb5f49007a4fa67c42e872e236417fa5ce5c9a0
Summary:
Right now, fileblob crashes if those two things are on different devices
(because you cannot atomically rename across devices), so you need to make sure
your TMPDIR is on the same volume as your fileblob.
This is kinda annoying and kinda unnecessary. Let's just fix default to putting
the temporary files into the same location as the blobstore.
Note: while the temp files will be in the same directory as the rest of our
blobs, they don't have the `blob-` prefix (their prefix will be `.tmp`), so
they cannot be read by accident as if they were blobs.
Reviewed By: farnz
Differential Revision: D27046889
fbshipit-source-id: c2b47cd6927eef34ac19325f87f446a6f6532eaf
Summary: Tidy up a bit and use the constants
Reviewed By: krallin
Differential Revision: D27043233
fbshipit-source-id: 3208e2f35c67b4b22bb5f8189cd8c5b399604833
Summary:
Prefetch profiles has been tested to work on windows. Prefetching will be a little slower
on windows as process creation is more expensive, but it is still functional, and should
be ready for us to gather some early numbers on windows sandcastle.
Reviewed By: xavierd
Differential Revision: D27016292
fbshipit-source-id: ed0c8b732b1dfb8bf0a301ae5e76469d4e59f8d2
Summary: This makes it easier to spot problematic segments.
Differential Revision: D27025039
fbshipit-source-id: 44ae9eaf03ca54751b64c0cf977b78737eaa58f0
Summary: The Rust commit graph does not hardcode nullid. This avoids crashes in some cases.
Differential Revision: D27022659
fbshipit-source-id: 0f61ffd9da89de7a477470384c6015d9edfaaaa0
Summary:
Our current straming changelog updater logic is written in python, and it has a
few downsides:
1) It writes directly to manifold, which means it bypasses all the multiplexed
blobstore logic...
2) ...more importantly, we can't write to non-manifold blobstores at all.
3) There are no tests for the streaming changelogs
This diff moves the logic of initial creation of streaming changelog entry to
rust, which should fix the issues mentioned above. I want to highligh that
this implementation only works for the initial creation case i.e. when there are no
entries in the database. Next diffs will add incremental updates functionality.
Reviewed By: krallin
Differential Revision: D27008485
fbshipit-source-id: d9583bb1b98e5c4abea11c0a43c42bc673f8ed48
Summary:
We saw the `unwrap()` panic at `iddag.rs:453`:
let (new_idx, low, high, parent_count, has_root) = candidate.unwrap();
Reading the code, it seems it can only happen if one of `parents` is `high` in a segment.
Parents in a segment should all be < low. Add some validations for that.
Reviewed By: sfilipco
Differential Revision: D27021394
fbshipit-source-id: 2c51713dfb8f07f2921229d8866f7e8a0bd9490f
Summary: Add the EdenAPI endpoint for resolving bookmarks. This is a first pass that just takes a bookmark name as a path variable, to make sure that this is on the right track. We'll want to add a proper request type that includes a list of bookmarks and a response type that can indicate that no bookmark was found. Then the hg bookmark command will also need support for prefix listing capabilities.
Reviewed By: kulshrax
Differential Revision: D26920845
fbshipit-source-id: 067db6a636a75531ee5953392b734c038a58efb6
Summary:
For example, `hg log > a.txt`, writing to `a.txt` does not need to clear the
progress bars.
Reviewed By: andll
Differential Revision: D26886281
fbshipit-source-id: 392c50e62a854e03ea0c400a1dac8bfb900815a2
Summary:
The Rust progress bars now integrates with hg-http and all requests get
progress bars for free. There is no need for a separate bar.
Reviewed By: andll
Differential Revision: D26970747
fbshipit-source-id: c5e59c745e75a93e5da0541baab21c3f23dc1322
Summary:
Render the progress bars in a Rust thread.
Also make it compatible with traditional Python progress bars (not the most efficient way because the Python still have its own progress thread, but the code change is minimal).
Reviewed By: andll
Differential Revision: D26886282
fbshipit-source-id: ad1f862ced278cc1cb90c37f576eb05f52a50b13
Summary: 100 and 5k seems too small - requests are too frequent.
Reviewed By: andll
Differential Revision: D26886276
fbshipit-source-id: cb5cce1111120f8a437fbc8c60906940588f38bc
Summary:
Split `dispatch()` into 2 parts: config parsing and command execution.
This allows callsite to do extra work based on the config before running the
command.
Reviewed By: andll
Differential Revision: D26853749
fbshipit-source-id: 459918b43341f7e14df6958056b6e07453bb691b
Summary:
Scuba stats provide a lot of context around the workings of the service.
The most interesting operation for segmented changelog is the update.
Reviewed By: krallin
Differential Revision: D26770846
fbshipit-source-id: a5250603f74930ef4f86b4167d43bdd1790b3fce
Summary:
STATS!!!
Count, success, failure, duration. Per instances, per repo.
I wavered on what to name the stats. I wondered whether it was worth being more
specific that "mononoke.segmented_changelog.update" with something like
"inprocess". In my view the in process stats are more important than the tailer
stats because the tailer is more simple and thus easier to understand. So I add
extra qualifications to the tailer stats and keep the name short for inprocess
stats.
Reviewed By: krallin
Differential Revision: D26770845
fbshipit-source-id: 8e02ec3e6b84621327e665c2099abd7a034e43a5
Summary: Currently unused. Will add stats the reference it.
Reviewed By: krallin
Differential Revision: D26770847
fbshipit-source-id: d5694cd221c90ba3adaf89345ffeb06fa46b9e7b
Summary:
Handle (ignore) git-submodules in gitimport.
git-sub-modules are represented as ObjectType::Commit inside the tree. For now we do not support git-sub-modules but we still need to import repositories that has sub-modules in them (just not synchronized), so ignoring any sub-module for now.
Reviewed By: StanislavGlebik
Differential Revision: D26999625
fbshipit-source-id: eb32247d4ad0325ee433e21a516ac4a92469fd90
Summary: Record some more stats so we can see last finish time. Also record update stats for run and chunk number so can see how far along a run is.
Differential Revision: D26949482
fbshipit-source-id: 5e7df4412c25149559883b6e15afa70e1c670cdc
Summary:
We don't have data on how often this repo-corrupted event occurs, so
begin logging it.
Reviewed By: xavierd
Differential Revision: D26968638
fbshipit-source-id: 95e62a595f74f22fdec8a60ebb0671e53cbe199e
Summary:
It's always annoyed me that HgProxyHash has a constructor which knows
how to load itself from a LocalStore. Add an explicit load() function,
and clean up some other stuff about the class while I'm in there.
Reviewed By: xavierd
Differential Revision: D26769231
fbshipit-source-id: f0ea9f16c3f1fbcd3d4361bcc34845901094b282
Summary: adds a new subcommand `disable` which will update the user's local eden config with a flag to disable prefetching
Reviewed By: kmancini
Differential Revision: D26964514
fbshipit-source-id: 043c89a5848f827377744ebc46fe93c431efee3e
Summary: This method should be preferred to use from python binding. It is similar to block_on but allow to interrupt future on Ctrl+C
Reviewed By: quark-zju
Differential Revision: D26894586
fbshipit-source-id: 02d8fbb2f9c4d9e585f8fbee51bfc400c402d0d2
Summary:
During sparse profile change, there might be a case when file has changed between checkouts, and also is added due to sparse profile change
Currently this would lead to creating duplicated update action that would waste resources. This diff deduplicates such action
Reviewed By: quark-zju
Differential Revision: D26953657
fbshipit-source-id: ccd9084cc3984ac0692ee6178fce018a94cd56e7
Summary:
Current store implementation returns value that may contain matadata.
We need to strip it before writing out file to disk
This is not going to be a problem with new Eden API, but it is not erady yet and rolling it out may take time
So in order to start testing native checkout now, we are incorporating metadata removal in the checkout code
Reviewed By: quark-zju
Differential Revision: D26953658
fbshipit-source-id: 05d8be55bb96e7c3162594498ef0b9944b422c90
Summary: All important jobs (SCS Server, LFS Server, Mononoke Server, derived data) have switched successfully. Roll up anything that's been missed by switching the default and letting contbuild take care of it
Reviewed By: krallin
Differential Revision: D26980991
fbshipit-source-id: 2c9f7cd56c38e9e1a2f8374c76141e7a99c88a2a
Summary:
Bounded traversal's internal book-keeping moves the futures returned from fold and unfold callbacks around while they are being queued to be scheduled. If these futures are large, then this can result in a significant portion of bounded traversal's CPU time being spent on `memcpy`ing these futures around.
This can be prevented by always boxing the futures that are returned to bounded traversal. Make this a requirement by changing the type from `impl Future<...>` to `BoxFuture<...>`.
Reviewed By: mitrandir77
Differential Revision: D26997706
fbshipit-source-id: 23a3583adc23c4e7d3607a78e82fc9d1056691c3
Summary:
Previously it was possible to use streaming clone only with xdb table. This
diff changes it
Reviewed By: farnz
Differential Revision: D27008486
fbshipit-source-id: b8d51832dd62b4343b36c3a7a96b83a327056025
Summary:
Knowing the prepushrebase changeset id is required for retroactive review.
Retroactive review checks landed commits, but verify integrity hook runs on a commit before landing. This way the landed commit has no straightforward connection with the original one and retroactive review can't acknowledge if verify integrity have seen it.
Reviewed By: StanislavGlebik
Differential Revision: D26944453
fbshipit-source-id: af1ec3c2e7fd3efc6572bb7be4a8065afa2631c1
Summary:
This tunable is not used anymore, we use
getbundle_high_low_gen_num_difference_threshold instead. Let's remove it.
Differential Revision: D26984966
fbshipit-source-id: 4e8ded5982f7e90c90476ff758b766df55644273
Summary:
The existing query to establish HgChangesetId on the path to FileContentMetadata for LFS validation is quite complex, using HgFilenode linknodes.
This change adds an optional edge from BonsaiHgMappingToHgBonsaiMapping that can be used to simplify the LFS validation case and load less data to get there.
Reviewed By: mitrandir77
Differential Revision: D26975799
fbshipit-source-id: 799acb8228721c1878f33254ebfa5e6345673e5d
