Summary:
This gives us more information about how large the hgsql operation is.
As we're here, move `sqlwriteunlock` to an earlier place so the logging is outside
the sql lock. Also document `unbundle` a bit so it's clear `repo.transaction()` only
takes sql lock in `unbundle` context.
Reviewed By: phillco
Differential Revision: D10056307
fbshipit-source-id: 5d3361b4044e6fcf01e60409ef1ecb34da34ccac
Summary:
Inside the critical section, "needsync" runs twice and each time it scans
O(bookmarks + heads), which could be slow.
If the repo is actually up-to-date, there could be a faster path for testing -
just hashing all bookmarks and tip. Note: "heads" is skipped to save some time.
Since commits are append-only, "heads" change implies "tip" movement. So
checking "tip" is enough.
As we're here, fix the empty repo case so syncing is skipped for an empty repo.
Since this is a drop-in replacement for `needsync()[0]`, turn it on by default.
Reviewed By: phillco
Differential Revision: D10022866
fbshipit-source-id: f9e304865db3575515d66444762f21071d5665a3
Summary:
When entering the critical section (creating a transaction), hgsql might wait
for SQL lock for a long time (minutes). And it currently does nothing during
that wait time. If the lock was acquired after a long time, the work of
catching up with what the database already has might be too much. That is a
waste within the precious hgsql lock held critical section.
Instead of waiting for the lock for N seconds and doing nothing, wait for k
(k < N) seconds and also try to sync with SQL periodically. This makes the
repo closer to what the database has and can reduce work needed to pull from
the database in the critical section.
This adds CPU pressure to the MySQL tier. I added a config option to control
it. Worse case, we can change `hgsql.syncinterval` to make the sync less
frequent.
As we're here, try document the configs.
Reviewed By: phillco
Differential Revision: D10002578
fbshipit-source-id: bd72d8225c919aa2bc62743de1e1d3f27cba606a
Summary:
With hgsql.verbose turned on, print useful messages like how many commits the
server is syncing, how long the lock was held or waited, to stderr so they are
visible to the client. This is useful for the client to understand why the
server "hangs" for a couple of seconds, and is useful in tests.
As we're here, drop no-flake8 comment.
Enable the flag for the hgsql-sync test to verify it.
Reviewed By: phillco
Differential Revision: D10019937
fbshipit-source-id: 8d304ce5208dbc5b92ed20f69daba02e9040c73f
Summary: This would help people unblock themselves.
Reviewed By: markbt
Differential Revision: D9443602
fbshipit-source-id: f05e2b2390a88a9280149d2164c2d7ab71c29600
Summary:
This commit just moves out the updates to the revision_references
table to a separate method so that it can be easily wrapped around.
Reviewed By: DurhamG, phillco
Differential Revision: D8406981
fbshipit-source-id: 8046a3e5b41dcf6c2f569699ac03de78611ba0bd
Summary: See the comments for more.
Reviewed By: quark-zju
Differential Revision: D7738340
fbshipit-source-id: 20660c2ff7fdba1850c0fb9f34548db0084abf3d
Summary:
- Add support for RocksDB engine (developed as a drop in replacement for innodb) to hgsql to allow new xdb.hgsql.1-10 shards to host hg repos
- Prefer MySQL test DBs in same region
- Run all hgsql unit tests also for RocksDB engine
- Allow for nested ifs to make that possible (downside if you switch off rockdb tests, innodb tests are run twice)
Reviewed By: quark-zju
Differential Revision: D7014064
fbshipit-source-id: 073c36176aa7eaf74252ef33c3f47da594920b28
Summary:
Move hgsql into the hgext directory, and the tests to tests/test-hgsql-*.
Update the tests to refer to the new places for things.
Test Plan: Run the hgsql tests and make sure they pass.
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D6660499
Tasks: T24908724