Summary:
Now that all our repos are treemanifest, let's enable the extension by
default in tests. Once we're certain no one needs it in production we'll also
make it the default in core Mercurial.
This diff includes a minor fix in treemanifest to be aware of always-enabled
extensions. It won't matter until we actually add treemanifest to the list of
default enabled extensions, but I caught this while testing things.
Reviewed By: ikostia
Differential Revision: D15030253
fbshipit-source-id: d8361f915928b6ad90665e6ed330c1df5c8d8d86
Summary:
It makes this method 25-30% faster (shaves off 250-300 ms).
Also it counts number of fetched rows correctly - fetchall method was
overriden, but looks like __iter__ method wasn't
Reviewed By: ikostia
Differential Revision: D14915472
fbshipit-source-id: 313695c1a83d05dac2fc801792226b6b64539cb5
Summary:
D13853115 adds `edenscm/` to `sys.path` and code still uses `import mercurial`.
That has nasty problems if both `import mercurial` and
`import edenscm.mercurial` are used, because Python would think `mercurial.foo`
and `edenscm.mercurial.foo` are different modules so code like
`try: ... except mercurial.error.Foo: ...`, or `isinstance(x, mercurial.foo.Bar)`
would fail to handle the `edenscm.mercurial` version. There are also some
module-level states (ex. `extensions._extensions`) that would cause trouble if
they have multiple versions in a single process.
Change imports to use the `edenscm` so ideally the `mercurial` is no longer
imported at all. Add checks in extensions.py to catch unexpected extensions
importing modules from the old (wrong) locations when running tests.
Reviewed By: phillco
Differential Revision: D13868981
fbshipit-source-id: f4e2513766957fd81d85407994f7521a08e4de48
Summary:
This gives us more information about how large the hgsql operation is.
As we're here, move `sqlwriteunlock` to an earlier place so the logging is outside
the sql lock. Also document `unbundle` a bit so it's clear `repo.transaction()` only
takes sql lock in `unbundle` context.
Reviewed By: phillco
Differential Revision: D10056307
fbshipit-source-id: 5d3361b4044e6fcf01e60409ef1ecb34da34ccac
Summary:
Inside the critical section, "needsync" runs twice and each time it scans
O(bookmarks + heads), which could be slow.
If the repo is actually up-to-date, there could be a faster path for testing -
just hashing all bookmarks and tip. Note: "heads" is skipped to save some time.
Since commits are append-only, "heads" change implies "tip" movement. So
checking "tip" is enough.
As we're here, fix the empty repo case so syncing is skipped for an empty repo.
Since this is a drop-in replacement for `needsync()[0]`, turn it on by default.
Reviewed By: phillco
Differential Revision: D10022866
fbshipit-source-id: f9e304865db3575515d66444762f21071d5665a3
Summary:
When entering the critical section (creating a transaction), hgsql might wait
for SQL lock for a long time (minutes). And it currently does nothing during
that wait time. If the lock was acquired after a long time, the work of
catching up with what the database already has might be too much. That is a
waste within the precious hgsql lock held critical section.
Instead of waiting for the lock for N seconds and doing nothing, wait for k
(k < N) seconds and also try to sync with SQL periodically. This makes the
repo closer to what the database has and can reduce work needed to pull from
the database in the critical section.
This adds CPU pressure to the MySQL tier. I added a config option to control
it. Worse case, we can change `hgsql.syncinterval` to make the sync less
frequent.
As we're here, try document the configs.
Reviewed By: phillco
Differential Revision: D10002578
fbshipit-source-id: bd72d8225c919aa2bc62743de1e1d3f27cba606a
Summary:
With hgsql.verbose turned on, print useful messages like how many commits the
server is syncing, how long the lock was held or waited, to stderr so they are
visible to the client. This is useful for the client to understand why the
server "hangs" for a couple of seconds, and is useful in tests.
As we're here, drop no-flake8 comment.
Enable the flag for the hgsql-sync test to verify it.
Reviewed By: phillco
Differential Revision: D10019937
fbshipit-source-id: 8d304ce5208dbc5b92ed20f69daba02e9040c73f
Summary: This would help people unblock themselves.
Reviewed By: markbt
Differential Revision: D9443602
fbshipit-source-id: f05e2b2390a88a9280149d2164c2d7ab71c29600
Summary:
This commit just moves out the updates to the revision_references
table to a separate method so that it can be easily wrapped around.
Reviewed By: DurhamG, phillco
Differential Revision: D8406981
fbshipit-source-id: 8046a3e5b41dcf6c2f569699ac03de78611ba0bd
Summary: See the comments for more.
Reviewed By: quark-zju
Differential Revision: D7738340
fbshipit-source-id: 20660c2ff7fdba1850c0fb9f34548db0084abf3d
Summary:
- Add support for RocksDB engine (developed as a drop in replacement for innodb) to hgsql to allow new xdb.hgsql.1-10 shards to host hg repos
- Prefer MySQL test DBs in same region
- Run all hgsql unit tests also for RocksDB engine
- Allow for nested ifs to make that possible (downside if you switch off rockdb tests, innodb tests are run twice)
Reviewed By: quark-zju
Differential Revision: D7014064
fbshipit-source-id: 073c36176aa7eaf74252ef33c3f47da594920b28
Summary:
Move hgsql into the hgext directory, and the tests to tests/test-hgsql-*.
Update the tests to refer to the new places for things.
Test Plan: Run the hgsql tests and make sure they pass.
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D6660499
Tasks: T24908724
