Summary: Implemented logic for fetching data from the scmmemcache and the server in pack files format. By default `prefetch` will still use loose files, unless the `remotefilelog.fetchpacks` config flag will be set to true.
Differential Revision: D13905733
fbshipit-source-id: cc559f5bc600067199a91efe6cf44b1bde6bf69b
Summary:
The default value is specified in configitems.py. Remove the `default`
argument.
Reviewed By: phillco
Differential Revision: D13909730
fbshipit-source-id: ca45c63be7d48e04e71a6f5b3b682f625a3030f6
Summary:
The simplecache test attempts to connect to a memcache instance on localhost.
This means the test fails if a memcache instance is not available.
Remove the use of memcache in the test. We still test local files, and assume
that memcache itself works.
Also corrupt the correct key when testing corruption handling, and document the
config options.
Reviewed By: quark-zju
Differential Revision: D13818471
fbshipit-source-id: 6ca7f575aa813f95773144be5337796f029ffd90
Summary:
Move top-level Python packages `mercurial`, `hgext` and `hgdemandimport` to
a new top-level package `edenscm`. This allows the Python packages provided by
the upstream Mercurial to be installed side-by-side.
To maintain compatibility, `edenscm/` gets added to `sys.path` in
`mercurial/__init__.py`.
Reviewed By: phillco, ikostia
Differential Revision: D13853115
fbshipit-source-id: b296b0673dc54c61ef6a591ebc687057ff53b22e
Summary:
Add a config option to use indexedlog as the node map storage instead
of a flat text file.
Reviewed By: quark-zju
Differential Revision: D13062573
fbshipit-source-id: ae14df24a4e36c59fbd9ec82d785aac52a2f8b5f
Summary:
a couple of problems:
* The recent move from tweakdefaults also broke `bin` symbol, rendering
this feature completely broken for non svn backed repos
* The `bgr` tool knows about fewer corpuses than we do, so go directly
to the underlying C++ client binary
* Add configuration options for that binary
Reviewed By: phillco, farnz
Differential Revision: D12849852
fbshipit-source-id: 154d4822d097602505349d3f67b45f19c17a7bf8
Summary:
Normally pushbackup should not take the repo lock. To debug when this does
occur in the background, add a debugstacktrace for lock aquisition when
infinitepushbackup.bgdebuglocks is set.
Reviewed By: quark-zju
Differential Revision: D10508808
fbshipit-source-id: 3cb2c155bac88751122111d91bdc1f51b381f7ff
Summary:
Adds the `grep.biggrepcorpus` option that links the repo to the appropriate
biggrep corpus to each of the repo specific configs that we maintain.
This allows us to know when a repo can be used with biggrep.
Use this option to automatically enable the use of biggrep when in an eden repo.
Reviewed By: quark-zju
Differential Revision: D10434568
fbshipit-source-id: a2c77d8c58b611f818d08725a6298dec1263bb8a
Summary:
Per D8985248, this code isn't necessary anymore under clindex. clindex is out everywhere now, so let's nuke it.
Shortens this file by 30%!
Reviewed By: quark-zju
Differential Revision: D10414815
fbshipit-source-id: 795c625afd128d94f09d0f7c1b294a7f7d8eea14
Summary:
Note that this is more aggressive than the tweakdefaults version; it doesn't allow named branches to be created or listed at all with the config set. As we want to delete named branches entirely, this seems reasonable.
I ported the `--new` flag, but made it optional, to make it easier to support both the core and tweakdefaults callers.
Saurabh Singh and I chatted with the releng team (Craig).
**tldr:** we can remove named branches from the UI but should keep the innards for now
Today the releng team doesn't make any branches (#continuous), but in the near-term future, they might either:
- create remote bookmarks in hg (similar to fbsource)
- create branches in SVN (which are then synced as hg branches by hgsubversion)
- We'll leave all the hgsubversion code to do this alone, just remove user-facing entry points
- They'll also need to be able to query for a commit's branch (`hg log -T '{branch}'` will still work)
- Once www is on hg, this possibility goes away and we can kill named branches for good
Reviewed By: ikostia
Differential Revision: D10401485
fbshipit-source-id: 6f2f3ae28af249bae1fdf782eb14fe7bfc472bb7
Summary:
When entering the critical section (creating a transaction), hgsql might wait
for SQL lock for a long time (minutes). And it currently does nothing during
that wait time. If the lock was acquired after a long time, the work of
catching up with what the database already has might be too much. That is a
waste within the precious hgsql lock held critical section.
Instead of waiting for the lock for N seconds and doing nothing, wait for k
(k < N) seconds and also try to sync with SQL periodically. This makes the
repo closer to what the database has and can reduce work needed to pull from
the database in the critical section.
This adds CPU pressure to the MySQL tier. I added a config option to control
it. Worse case, we can change `hgsql.syncinterval` to make the sync less
frequent.
As we're here, try document the configs.
Reviewed By: phillco
Differential Revision: D10002578
fbshipit-source-id: bd72d8225c919aa2bc62743de1e1d3f27cba606a
Summary:
With hgsql.verbose turned on, print useful messages like how many commits the
server is syncing, how long the lock was held or waited, to stderr so they are
visible to the client. This is useful for the client to understand why the
server "hangs" for a couple of seconds, and is useful in tests.
As we're here, drop no-flake8 comment.
Enable the flag for the hgsql-sync test to verify it.
Reviewed By: phillco
Differential Revision: D10019937
fbshipit-source-id: 8d304ce5208dbc5b92ed20f69daba02e9040c73f
Summary:
For some arcrc formats, hg commands prefer a deprecated blob in the .arcrc file which does not contain an oauth token, and only contains a phabricator cert (which is deprecated in favor of oauth)
This updates the logic to look for the phabricator.intern.facebook.com blob in the .arcrc to get an oauth token.
Reviewed By: mitrandir77
Differential Revision: D9351851
fbshipit-source-id: 11efe30b077480caa7b147c45a8d0eb78189277e
Summary: This cuts the speed on my laptop from 1.3 min (78s) to 8.9s.
Reviewed By: quark-zju
Differential Revision: D8738675
fbshipit-source-id: f6bd0857c949ad96b0eea54d8ddbd9a0427e8895
Summary: The recent reformat caused a few test failures, this should fix them.
Differential Revision: D8207359
fbshipit-source-id: ced37aff3038715e5876fe27b58a60ebe45e4097
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
Summary:
We want to allow blocking full repo streaming clones in certain
repositories (since they can take the lock and take a very long time) unless the
client has explicitly asked for it. The existing stream_out wire protocol has no
way of passing an option, so let's create a new endpoint.
Reviewed By: quark-zju
Differential Revision: D7763717
fbshipit-source-id: eace47143f8fdcc4c6e302b5c26678ccf56ca5d4
Summary:
The hg servers were having trouble bootstrapping the .hg/git directory
because hggit was creating a bare repo and then later git fetches were super
slow. Those .hg/git directories should be managed by infra outside of hg, so
let's add a config knob to prevent hggit from creating a .hg/git repo.
Reviewed By: quark-zju
Differential Revision: D7586441
fbshipit-source-id: 7c2578123428aadaa6a94484e45dda6bd6b5a42d
Summary:
Pushrebase previously defaulted to bundlev1. Now that the server can
support v2, let's default to v2.
Reviewed By: quark-zju
Differential Revision: D7591174
fbshipit-source-id: bc5448490ff319543baedf3f5a1aab160a73ed27
Summary: This allows people to silence the hint.
Reviewed By: markbt
Differential Revision: D7392127
fbshipit-source-id: ac16f952a178d567ce13e22946127456972ebe85
Summary:
Added passing BatchMode option to SSH call only when puchbackup runs in background.
Also fixed dummyssh in skipping options before hostname, added unittest.
Reviewed By: quark-zju
Differential Revision: D7119123
fbshipit-source-id: 2c8e66fee44cca5b23389cba8e21e3a0b237268e
Summary:
Previously pushrebase would only send changegroups using the cg1
format. remotefilelog will soon require cg2 (and it results in better deltas
anyway), so let's change pushrebase to allow using cg2.
Initially it is off by default. We will change it to be on by default once the
server has been upgrade to to handle the received part.
Reviewed By: mjpieters
Differential Revision: D7108732
fbshipit-source-id: ff4ad3a3fc2801aec4876db30c8130ce743b2e6a
Summary:
- Add support for RocksDB engine (developed as a drop in replacement for innodb) to hgsql to allow new xdb.hgsql.1-10 shards to host hg repos
- Prefer MySQL test DBs in same region
- Run all hgsql unit tests also for RocksDB engine
- Allow for nested ifs to make that possible (downside if you switch off rockdb tests, innodb tests are run twice)
Reviewed By: quark-zju
Differential Revision: D7014064
fbshipit-source-id: 073c36176aa7eaf74252ef33c3f47da594920b28
Summary:
This extension can be a bit noisy when defaulted-on. Let's disable its debug output in
tests (except fot its own tests).
Differential Revision: D6777765
fbshipit-source-id: e2a6c6b7527336a1fac24078d2b49eb1e75ecabf
Summary:
Some consumers still rely on enabling 'sparse=' so let's add a module
that just redirects to the real fbsparse.py.
Also updates configerator to use the newer name.
Reviewed By: markbt, quark-zju
Differential Revision: D6755971
fbshipit-source-id: 3a67f029045dacf927742a616a714fe632b97fea
Summary:
The isgooddelta tweak was introduced in D2693043 (perftweaks: change revlog
delta heuristic, 2015-11-24). Comparing with the existing version, the only
change is that we removed `dist > maxdist` check.
Note that the upstream commit 895ecec31 (revlog: add an experimental option
to mitigated delta issues (issue5480), 2017-06-23) also introduces a config
option to override `maxdist` to make the condition fail, which basically does
a same thing.
Instead of introducing new config options or adding more "if"s to the
codebase to make it more obscure, let's just simplify it by disabling the
check entirely, and removing the `dist` concept, removing two config
options: `experimental.maxdeltachainspan` and `perftweaks.preferdeltas`.
The `chainlen > self._maxchainlen` check should be enough for keeping
delta chain length bounded.
Reviewed By: DurhamG
Differential Revision: D6752529
fbshipit-source-id: e8fd8ec39240191db5fb274190fc661e97087a78
Summary: To be fixed later, now I need to test the hgbuild.
Differential Revision: D6748921
fbshipit-source-id: 519555286a43ef7852e37b699e30197c1ad85424
# skip-blame because this was mechanically rewritten the following script. I
ran it on both *.t and *.py, but none of the *.py changes were proper. All *.t
ones appear to be, and they run without addition failures on both Windows and
Linux.
import argparse
import os
import re
ap = argparse.ArgumentParser()
ap.add_argument('path', nargs='+')
opts = ap.parse_args()
globre = re.compile(r'^(.*) \(glob\)(.*)$')
for p in opts.path:
tmp = p + '.tmp'
with open(p, 'rb') as src, open(tmp, 'wb') as dst:
for line in src:
m = globre.match(line)
if not m or '$LOCALIP' in line or '*' in line:
dst.write(line)
continue
if '?' in line[:-3] or ('?' in line[:-3] and line[-3:] != '(?)'):
dst.write(line)
continue
dst.write(m.group(1) + m.group(2) + '\n')
os.unlink(p)
os.rename(tmp, p)
I used this to more quickly track down a failing test-check-config.t issue
in another repo. I thought it might be useful more generally, so I'm sending
it out in case others think it's a worthwhile change.
The ignore regular expression has been updated to detect
"inconsistent config." If present, we track which configs have
that set and we suppress the conflicting defaults error for those
options.
I also added named groups to the regexp to aid readability.
A comment was added to profiling.py to make a desired inconsistent
value error go away.
Most test scripts use "hg" to interact with a temporary test repository.
However a few tests also want to run hg commands to interact with the local
repository containing the mercurial source code. Notably, many of the
test-check-* tests want to check local files and commit messages.
These tests were previously using the version of hg being tested to query the
source repository. However, this will fail if the source repository requires
extensions or other settings not supported by the version of mercurial being
tested. The source repository was typically initially cloned using the system
hg installation, so we should use the system hg installation to query it.
There was already a helpers-testrepo.sh script designed to help cope with
different requirements for the source repository versus the test repositories.
However, it only handled the evolve extension. This new behavior works with
any extensions that are different between the system installation and the test
installation.
statprof.display_hotpath() accepts a "limit" function to choose the
minimum threshold for samples to display. The default is 0.05, which
means you don't need individual items contributing less than 5%.
I had a need to adjust this threshold. We already have a config
option for it. So let's reuse it.
check-config.py doesn't like having multiple defaults for the
ui.configwith() calls. The behavior is obviously correct. I'm
not sure if it is worth teaching check-config.py how to ignore
this. So I've just accepted the new output.
We previously weren't looking for this config helper. And, surprise,
profiling.py references config options without docs.
If I tried hard enough, I could have combined the regexps using a
positive lookbehind assertion or something. But I didn't want to make
my brain explode.
At some point, we should probably do this linting at the tokenizer or
ast layer. I'm not willing to open that can of worms right now.
Before this patch, test-check-config.t fails on Solaris, because
"xargs" doesn't invoke check-config.py with all filenames at once.
"xargs" may invoke specified command multiple times with part of
arguments given from stdin: according to "xargs(1)" man page, this
dividing arguments is system-dependent.
For portability of test-check-config.t, this patch adds "xargs" like
mode to check-config.py and executes it in test-check-config.t without
"xargs".
We had them on 'test-check-code-hg.t' to avoid collision with the test checking
'check-code' itself. Now that this one have been rename, we can safely remove
this suffix for all of them. This get them in line with 'check-pyflakes.t'.