Summary:
This is unnecessary after the recent modifications in the namespace
code.
Reviewed By: quark-zju
Differential Revision: D9815973
fbshipit-source-id: d7c6b07226be35ebbbe5354d5c520c9807cc0aa6
Summary:
Previously, removing stale POSIX lock is done at different places:
- Stale flock lock is removed by `posix.makelock`. The "read + check + unlink"
is done atomically (taking the directory lock).
- Stale (legacy) symlink lock is removed by `lock.py`.
The "read + check + unlink" is not done atomically, which seems like a scary
bug [since 2016](d314a89fa4).
This patch cleans the logic up so removing stale POSIX lock is done
properly at `util.makelock`.
Note that Windows logic is not changed yet, so it's in theory broken.
Ideally we'd like to use flock-ish locking on Windows too. But that's
outside the scope of this diff.
Reviewed By: ikostia
Differential Revision: D9770935
fbshipit-source-id: fc58f30348f8a9aba22df415c6d2b2dfeb6a87d9
Summary:
On POSIX, a lock is created in these steps:
1. Take the directory lock to make "test stale lock & unlink" atomic.
2. Test and remove stale lock.
3. Create a legacy symlink (placeholder) lock.
4. Create a temporary file, flock it.
5. Rename the temporary file to override the symlink lock.
Step 3 is to be compatible with legacy hg programs. However, the error handling
is not great - in case 4 or 5 errored out (ex. Ctrl+C), the symlink file is left
untouched. This diff removes it.
Reviewed By: ikostia
Differential Revision: D9770936
fbshipit-source-id: a004d6326381e304552088bcf322c052254321dd
Summary:
This makes sure that we can instruct `setup.py` to make every Rust binary
long-paths aware on Windows. Yay.
Reviewed By: markbt
Differential Revision: D9809915
fbshipit-source-id: e58787d41c7e8bc5698283fd67dfb3cc6d21321d
Summary:
`treemanifest._generatepackstream` truncates the list of possible base
manifests to length 2, because `cstore.treemanifest.walksubdirtrees` can only
support two base trees to compare against. However, it does this too soon, we
can select a different 2 base manifests for each comparison out of the possible
pool.
Reviewed By: quark-zju
Differential Revision: D9788298
fbshipit-source-id: 4ba8753ccc34e09563bebdf90602211a999a02ea
Summary:
This test arranges it so that `treemanifest._generatepackstream` is called with
three possible base manifests, each of which is appropriate for a different
commit. Because this function truncates the list of candidate base manifests
too soon, we end up with too much data in the bundle.
Reviewed By: quark-zju
Differential Revision: D9788299
fbshipit-source-id: 2147a63108e11750083862abc73c386867549ba2
Summary:
Add a debugbundle handler for treemanifest tree parts. At the moment this just
prints out the size of the treepart. In the future it can be extended to show
more information.
Reviewed By: quark-zju
Differential Revision: D9788297
fbshipit-source-id: c298548062ab48e6589b1c0f3e502590168eddbf
Summary: Otherwise `./entrypoint.py` will be executed as a shell script.
Reviewed By: singhsrb
Differential Revision: D9800130
fbshipit-source-id: 656fb37aae97f1d5757a3f5bba0af701abc84d03
Summary:
Broken by D7563731. Running `exec` in a test will replace the test process.
That means the tests are basically doing nothing. Wrap `exec` in a subshell to
fix it.
Reviewed By: singhsrb
Differential Revision: D9800124
fbshipit-source-id: d4f7f29502dcd2db542f5dd03827a06cc9abd783
Summary:
We need to look up commits in the repository based on the strictly
increasing global revision numbers. This commit adds the following way of doing
so:
- `m<number>`: `hg log -r m<number>` would match the commit having `number` as
the strictly increasing global revision number.
Reviewed By: quark-zju
Differential Revision: D9764819
fbshipit-source-id: b9d3a80852c7acebd0c939a4d4f43c3c9a6cb0ab
Summary:
Buck dev build uses symlinks. Following them would insert wrong libdir in
`sys.path`.
Reviewed By: simpkins
Differential Revision: D9794884
fbshipit-source-id: 14111d0d61ded33f500e2a5d5474a29b175d861c
Summary:
This fix uses mercurial's encoding.tolocal on the Perforce changelist description
instead of trying to decode it as ascii and ignoring/throwing away non-ascii
characters.
Reviewed By: singhsrb
Differential Revision: D9777614
fbshipit-source-id: 713bd63831c965687b7328de06016b4f351b0469
Summary:
Migrate builtin namespace definition to use the registrar. So we can deprecate
`addnamespace`.
`after` is not expressive enough in certain cases, although it's easier to
understand. Add a `priority` parameter for more explicit ordering handling.
Reviewed By: singhsrb
Differential Revision: D9780363
fbshipit-source-id: 85b9cfb91a7c937eea95f2a279a43c11e561b5fc
Summary:
Be more permissive about spaces. Namely:
- Spaces after a section name like `[foo] ` are allowed.
- Spaces in config names are allowed.
- Spaces at trailing lines are ignored and no longer insert an `\n` to the previous config.
This makes it closer to the older config parser behavior. But it's still
different on some cases, like `[foo]]`, `[foo] # bar`, `[foo]]` still do not
parse.
Benchmark shows no obvious (within 10%) slowdown. So this is probably fine.
Reviewed By: strager
Differential Revision: D9620253
fbshipit-source-id: 8489ef8e83606d0557db56e8da0a017d55ff1514
Summary:
This makes it easier and cleaner for other extensions to specify namespace
orders. Eventually we'd make the old way to add namespaces private.
Reviewed By: singhsrb
Differential Revision: D9780361
fbshipit-source-id: 25b96d87a1372a4bb177224804423cc423c86c20
Summary:
Previously, there is no easy API to add a namespace with desired order. There
is `extensions.afterloaded`, but that only works for `ui/extsetup`, not for
`reposetup`.
This diff add a namespace predicate so registering a namespace is similar to
registering a revset. It also provides an optional `after` parameter to define
the desired order, like:
from mercurial import registrar, namespaces
namespacepredicate = registrar.namespacepredicate()
namespacepredicate("svn", after=["remotenames"])
def svn(repo):
return namespaces.namespace("svn", ...)
Existing namespaces will be migrated by upcoming changes.
This is useful for things wanting to resolve custom names, like globalrevs and
commit cloud.
Reviewed By: singhsrb
Differential Revision: D9775703
fbshipit-source-id: f15827b042d605b9cebc1ee1af6479903f47ee9c
Summary:
The Python hook got "kwargs" without "HG_" prefix. Add an explicit test for it.
Also remove unnecessary configs and change the shell to bash, since we don't
run tests using vanilla "sh".
check-code is updated so it no longer complains about the use of `bash`.
Reviewed By: markbt
Differential Revision: D9561962
fbshipit-source-id: 76a190dde1b0aeb0032a65c173ce6368a28e8cf6
Summary:
I got paths like:
c:\\....\\c:\\....\
I guess that comes from a simple `os.path.join` with repo root.
Escape the path so it works on Windows.
Reviewed By: markbt
Differential Revision: D9550577
fbshipit-source-id: 0a2624ec0686df10ecf9cda8d50965354f04cecf
Summary:
Refresh happens if `time.time() > lastrefresh + REFRESHRATE (0.1)`.
`lastrefresh` is previously set before the actual refresh logic. That is
problematic if the refresh itself took more than 0.1 seconds, in which case the
refresh logic will be performed too frequently.
Set `lastrefresh` after the actual refresh logic to make it better.
Reviewed By: phillco
Differential Revision: D9770954
fbshipit-source-id: 06d7aa653406be53f88daeda7ec2239251e54845
Summary:
This function often takes a long time on my laptop. Therefore add a progress
bar.
Reviewed By: phillco
Differential Revision: D9770952
fbshipit-source-id: fbc2a2847b813e973afc56b935476de55fb48ba3
Summary:
Fixed by Yuya at upstream. See https://www.mercurial-scm.org/repo/hg/rev/ad76032d27da.
We probably want to use Cython or Rust to write native extensions.
Reviewed By: phillco
Differential Revision: D9770953
fbshipit-source-id: f7856ca516cf37677fddfbb42c094440c6349572
Summary:
We need to look up commits in the repository based on the strictly
increasing global revision numbers. This commit adds two ways of doing so:
- `globalrev(number)` revset: `hg log -r 'globalrev(number)'` would match the
commit having `number` as the strictly increasing global revision number.
- `m<number>` stringset: `hg log -r m<number>` would also match the commit
having `number` as the strictly increasing global revision number.
Reviewed By: quark-zju
Differential Revision: D9756146
fbshipit-source-id: db9ecf8f8244634d71957ddb645dfda675cab662
Summary:
D9363420 added code to perform extra logging of exceptions. Unfortunately if
an error occurs inside the logging code (for instance, if it tries to log
binary data that cannot be json-encoded) it masks the original exception
message and backtrace with the logging exception.
This issue is specific to python 2.x. Python 3.x reports both exception
backtraces in this case. However, in python 2.x putting a `try...except`
block inside an `except` block ends up overwriting the original exception
info, even when the inner exception is handled.
Moving the inner `try...except` block to its own function is sufficient to
avoid this problem in python 2.x
Reviewed By: singhsrb
Differential Revision: D9764805
fbshipit-source-id: c5a9e65fcd7505ccb478500d62c06c2c94308332
Summary:
Maybe useful as a backup for the regular path and also for syncing speed up.
Scm daemon know new and removed heads, so if for example 1 new and 1 removed head - it is the most probably just an amend, so scm daemon can try the fast path first depends on information in the notification, and if it fails try the slow path.
So users can have better experience before Mononoke, it is much much faster and scm daemon makes 2 attempts anyway!
Reviewed By: quark-zju
Differential Revision: D9309856
fbshipit-source-id: d59f498160a45fab11760b5c1397b48470feb7f8
Summary: This would provide information about performance changes.
Reviewed By: singhsrb
Differential Revision: D9620252
fbshipit-source-id: 51d243b50b349c63e552bd1c43db17497025f73a
Summary:
This is just a refactor to use absolute imports instead of relative
imports.
Reviewed By: quark-zju
Differential Revision: D9728876
fbshipit-source-id: 88853a7015965bc4e20ea4c6d9cab30864435eaf
Summary:
See diff note for the summary. This unblocks the roll out of
`globalrevs` on the clients due to `hgsql` requiring `mysql.connector`. See
https://fb.facebook.com/groups/scm/permalink/1808577559191856/ for how the
error manifests as a client side issue.
Reviewed By: quark-zju
Differential Revision: D9728885
fbshipit-source-id: 62a6f3b64362e2f821ca1ba0d7add0d2eedca2bd
Summary: It's still been failing and we probably don't need to worry about maxrss failing any time soon.
Reviewed By: singhsrb
Differential Revision: D9671400
fbshipit-source-id: e847a240285c2aa2fa15d49ec4fb010483278e9d
Summary:
use correct source for user name when backup token into the secret tool
the change will trigger transparent 1 time rebackup for all users but it is worth to fix
Reviewed By: singhsrb
Differential Revision: D9656508
fbshipit-source-id: 1b0674aa8f5490563d6beb5d429be2533018a0bf
Summary:
this is usefull when a user have a lot of other users commits in the default workspace
the set of commits includes all the stacks where at least one commit is user's own
the option is for the default workspace only, it doesn't apply for other workspaces a user may collaborate in
the idea is that the client code will never offer other user's commits to sync, so cloud sync will perfectly work with some extra commit might present on any machine.
this allow users to continue have other users commits in their workspace for review or other purposes even if it is very very large, but provides reliable backup and sync solution for user's own work.
the option works for all commands `hg cloud join` `hg cloud rejoin` `hg cloud sync` and `hg cloud recover`
Reviewed By: simpkins
Differential Revision: D9645571
fbshipit-source-id: 9adb68568083c87c22bb86c19fb7a5bb0f639265
Summary: This test fails from time to time. Let's widen the upper bound for maxrss a bit. If it continues to fail we can just delete the test.
Reviewed By: quark-zju
Differential Revision: D9659411
fbshipit-source-id: ef65d96919ddc809a29c5cd811dfa0bec780e722
Summary:
Lock ordering is important. When working with shared repos, when we
aquire a shared wlock, we must also acquire the master repo's wlock to ensure
that any operations that might later grab the lock are properly ordered.
Reviewed By: quark-zju, simpkins
Differential Revision: D9613877
fbshipit-source-id: 2c3eeeea534cdf49f3bd7e94db441c3a1479a959
Summary:
This is cleaner and makes more sense. We can do further cleanups here
as well, but moving it as-is to ensure correctness for now.
Reviewed By: quark-zju
Differential Revision: D9613876
fbshipit-source-id: 9c589d9338ffaf3d92fb155c235a447fa8a4a24f
Summary:
Improve the lock handling when fork / exec happens:
- Unlock explicitly so forked process won't cause surprises.
- Set O_CLOEXEC to also reduce surprises.
Reviewed By: wez
Differential Revision: D9646559
fbshipit-source-id: 7b192635594761eee6ddc8c86ab2107c0d95d221
Summary:
This test has been flaking out and breaking the build when the shelve was made >1s ago.
cc @[1266426728:Kostia Balytskyi]
Reviewed By: singhsrb
Differential Revision: D9645529
fbshipit-source-id: 575cb4272710d60c27e61aa519da32f4d2291eff
Summary:
Handle an empty .rowcount file. This is a common crash in the
field. The root cause is likely the non-atomic write to the .rowcount
file in fileproperty's setter.
The `rowcount` value is only used to accelerate calculating how many
commits pulled for display purpose. So the correctness of this value
does not matter.
Reviewed By: quark-zju
Differential Revision: D9631942
fbshipit-source-id: 2493304bf1cc82b6639100434bae667c2fc1eb70
Summary:
Before this commit, `treemanifest` registered a post-transaction hook to
prefetch data for the parents of new commits. Shelve produces a hidden
commit, so this logic failed, since it used a filtered repo instance.
Let change it to use unfiltered repo and remove all of the hidden commits
from the revset to avoid surprises.
Reviewed By: markbt
Differential Revision: D9613454
fbshipit-source-id: 5caddd2948e5d2e982462bebdb950b359995baa1
Summary:
Previously, we would show root-relative paths. Now we show cwd-relative
paths unless the --root-relative flag is passed.
Reviewed By: phillco, farnz
Differential Revision: D9607987
fbshipit-source-id: 6c24c05317c165f88d2a83d6921d1520044a9b6e
Summary:
When I first rolled this feature out, many people liked it but enough
people complained that it would prompt after EVERY file when sometimes they
wanted to always continue (although they were fine with the initial prompt).
This adds the new "always" option to the prompt.
Reviewed By: quark-zju
Differential Revision: D9607680
fbshipit-source-id: 76989b346ed9dd88048a37bcad6dcddd8325f314
Summary:
test-lfs-test-server.t is failing because of spurious output from Bash:
```
======================================================================
FAIL: test_lfs_test_server_t (scm.hg.tests.unittestify.hgtests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/data/users/strager/fbsource/fbcode/buck-out/dev/gen/scm/hg/tests/hg_run_tests#binary,link-tree/scm/hg/tests/unittestify.py", line 96, in runsingletest
raise self.failureException(err + out)
AssertionError:
--- test-lfs-test-server.t
+++ test-lfs-test-server.t.err
@@ -158,6 +158,7 @@
(Restart the server in a different location so it no longer has the content)
$ $PYTHON $RUNTESTDIR/killdaemons.py $DAEMON_PIDS
+ $TESTTMP.sh: line 116: 2128544 Terminated lfs-test-server &>lfs-server.log
$ rm $DAEMON_PIDS
$ mkdir $TESTTMP/lfs-server2
$ cd $TESTTMP/lfs-server2
@@ -187,3 +188,4 @@
Clean up:
$ $PYTHON $RUNTESTDIR/killdaemons.py $DAEMON_PIDS
+ $TESTTMP.sh: line 140: 2130121 Terminated lfs-test-server &>lfs-server.log (wd: ~/lfs-server2)
ERROR: test-lfs-test-server.t output changed
!
Failed test-lfs-test-server.t: output changed
```
If this this test wasn't broken, it would have caught a bug introduced by D9478744 (fixed by D9619464). Fix the failures to make the test pass again.
Reviewed By: quark-zju
Differential Revision: D9619647
fbshipit-source-id: 989db431a28ab2d491ced70dc9113998b0ddb5f5
Summary:
D9478744 broke ad-hoc uploading to LFS. In the `memlocal` class, `self.vfs` is already defined as a property method, so assigning `self.vfs` to `None` shadows the property. This causes publish_watchman_for_windows_to_chocolatey to crash with a TypeError when uploading `watchman.zip`:
```
Traceback (most recent call last):
File "<string>", line 44, in <module>
File "<string>", line 42, in __run
File "/usr/local/fbcode/gcc-5-glibc-2.23/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/local/fbcode/gcc-5-glibc-2.23/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/mnt/btrfs/trunk-hg-fbcode-fbsource-6-1535742021/fbcode/buck-out/dev/gen/watchman/facebook/windows/publish_watchman_for_windows_to_chocolatey#link-tree/watchman/facebook/windows/publish_watchman_for_windows_to_chocolatey.py", line 160, in <module>
main(sys.argv)
File "/mnt/btrfs/trunk-hg-fbcode-fbsource-6-1535742021/fbcode/buck-out/dev/gen/watchman/facebook/windows/publish_watchman_for_windows_to_chocolatey#link-tree/watchman/facebook/windows/publish_watchman_for_windows_to_chocolatey.py", line 52, in main
package=package, archive_uploader=archive_uploader, credentials=credentials
File "/mnt/btrfs/trunk-hg-fbcode-fbsource-6-1535742021/fbcode/buck-out/dev/gen/watchman/facebook/windows/publish_watchman_for_windows_to_chocolatey#link-tree/fb_choco/corp_chocolatey.py", line 80, in publish_package_from_template
package.upload_archives(uploader=archive_uploader)
File "/mnt/btrfs/trunk-hg-fbcode-fbsource-6-1535742021/fbcode/buck-out/dev/gen/watchman/facebook/windows/publish_watchman_for_windows_to_chocolatey#link-tree/watchman/facebook/windows/watchman_windows_package.py", line 60, in upload_archives
archive_url = uploader.upload_file(file=archive_file)
File "/mnt/btrfs/trunk-hg-fbcode-fbsource-6-1535742021/fbcode/buck-out/dev/gen/watchman/facebook/windows/publish_watchman_for_windows_to_chocolatey#link-tree/fb_choco/upload_archive.py", line 33, in upload_file
oid, _size = hg_lfs.upload_only(content)
File "/mnt/btrfs/trunk-hg-fbcode-fbsource-6-1535742021/fbcode/buck-out/dev/gen/watchman/facebook/windows/publish_watchman_for_windows_to_chocolatey#link-tree/tools/lfs/lfs.py", line 145, in upload_only
out, err = runhgcmd("debuglfssend", LFS_URL, stdin=data)
File "/mnt/btrfs/trunk-hg-fbcode-fbsource-6-1535742021/fbcode/buck-out/dev/gen/watchman/facebook/windows/publish_watchman_for_windows_to_chocolatey#link-tree/tools/lfs/lfs.py", line 123, in runhgcmd
"%r exited with %s:\nstdout:\n%s\nstderr:\n%s" % (cmd, exitcode, out, err)
RuntimeError: ['hg', '--traceback', '--config=extensions.lfs=', 'debuglfssend', 'https://dewey-lfs.vip.facebook.com/lfs'] exited with 255:
stdout:
b''
stderr:
b'Traceback (most recent call last):\n File "/usr/lib64/python2.7/site-packages/mercurial/scmutil.py", line 160, in callcatch\n return func()\n[snip]\n File "/usr/lib64/python2.7/site-packages/hgext/lfs/__init__.py", line 240, in debuglfssend\n remote.writebatch(pointers, local)\n File "/usr/lib64/python2.7/site-packages/hgext/lfs/blobstore.py", line 182, in writebatch\n self._batch(pointers, fromstore, "upload")\n File "/usr/lib64/python2.7/site-packages/hgext/lfs/blobstore.py", line 344, in _batch\n for _one, oid in oids:\n File "/usr/lib64/python2.7/site-packages/hgext/lfs/blobstore.py", line 322, in transfer\n self._basictransfer(obj, action, localstore)\n File "/usr/lib64/python2.7/site-packages/hgext/lfs/blobstore.py", line 270, in _basictransfer\n request.data = filewithprogress(localstore.vfs(oid), None)\nTypeError: \'NoneType\' object is not callable\n** unknown exception encountered, please report by visiting\n** Source Control @ FB group (https://fb.facebook.com/groups/scm/)\n** Python 2.7.5 (default, Jul 13 2018, 13:06:57) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]\n** Mercurial Distributed SCM (version 4.4.2_20180830_022112_16a83f70aeda)\n[snip] File "/usr/lib64/python2.7/site-packages/hgext/lfs/blobstore.py", line 270, in _basictransfer\n request.data = filewithprogress(localstore.vfs(oid), None)\nTypeError: \'NoneType\' object is not callable\n'
```
Remove the bogus assignment to fix the crash.
Note: The test-lfs-test-server.t test would have caught the crash. Unfortunately, that test has two issues: it requires manual setup, and it was failing before D9478744. D9619647 fixes the other failure, but the manual setup issue remains.
Reviewed By: quark-zju
Differential Revision: D9619464
fbshipit-source-id: bc11d805cf34eb441988aa43270fae1ce8c2c8fe
Summary: We'd like the lock file to be group-writable.
Reviewed By: simpkins
Differential Revision: D9623516
fbshipit-source-id: 5aa13ed3a5550751f300cdaf80441921f6fff50c
