Commit Graph

43388 Commits

Author SHA1 Message Date
Mark Thomas
d1b5faa3ac hint: allow hints to be defined and triggered in config
Summary:
Add a new config section, `[hint-definitions]` that describes additional hint
messages.  This can also be used to override built-in hint definitions.

Add a new template function `triggerhint(name)` that triggers the given hint.

Reviewed By: quark-zju

Differential Revision: D8056847

fbshipit-source-id: 5ffc945343133eb635ae0820190ecad9f16bc731
2018-05-18 11:19:35 -07:00
Durham Goode
5e95b0e32e treemanifest: change heuristic for sending local trees
Summary:
Previously if we were pulling between peers, we would not send trees
for public commits, since we assumed the receiving end could fetch them from the
server.

It turns out there are situations where public commits are actually local only
to the client (like when pulling directly from subversion before the server
does), so let's change our heuristic to send trees if they exist in the source
client's local data store.

This also required updating the server side tree pack generation logic to handle
the case when the client sends a base manifest node that the server doesn't know
how to handle. We now ignore those bad base manifests.

Reviewed By: phillco

Differential Revision: D7992502

fbshipit-source-id: 8ddd1836e6f325363ad2bb6eae34ddbbc84c2d11
2018-05-18 10:46:59 -07:00
Jun Wu
4c2fadcbba blackbox: add a way to label logs with a "request id"
Summary: This would help labeling interesting commands from callers.

Reviewed By: DurhamG

Differential Revision: D8057148

fbshipit-source-id: 647cc91be3baeaa0362799603a66dab0c6c570b8
2018-05-18 10:46:59 -07:00
Jun Wu
84cf12c3e7 logging: add logging about fsmonitor notefiles, clock and treedirstate root
Summary: Help investigate issues.

Reviewed By: DurhamG

Differential Revision: D8047436

fbshipit-source-id: de21b84330b3b90b06d81621ec362a20b5b747c7
2018-05-17 20:09:16 -07:00
Jun Wu
7049301036 fsmonitor: fix a potential issue with status calculation
Summary:
From the code context, if p2 is `context.overlayworkingctx`, we should avoid
update fsmonitor state.

Reviewed By: phillco, singhsrb

Differential Revision: D8044903

fbshipit-source-id: 82b750cc73a75a2aaa77a630bd73a1a77eedc4e9
2018-05-17 12:21:47 -07:00
Ryan McElroy
2e8e0f6fd0 rage: pretty-print fsmonitor output
Summary:
Previously we would print out all the contents of the file as-is but
this is very user-unfriendly and could produce huge unreadable outputs.

Now we print onyl the first 20 files and label the fields.

Reviewed By: phillco

Differential Revision: D8025914

fbshipit-source-id: c330c34ed4f31c3337bc414cd72c3e43cdd0caf5
2018-05-16 09:42:45 -07:00
Durham Goode
8a6a929876 lz4: add rust lz4 bindings
Summary:
The crates.io lz4 bindings only support the lz4 framed format, while
our python lz4 library produces custom framed compressed blobs. Let's add a new
wrapper around lz4-sys that handles are special framing. We can migrate to the
standard framing later.

Reviewed By: quark-zju

Differential Revision: D7855502

fbshipit-source-id: 04abb1bc784c6be7f22bcd80645d1b50debc93bd
2018-05-16 09:13:18 -07:00
Liubov Dmitrieva
86630f2155 infinitepush: add missing required escaping for mysql LIKE queries
Summary: we should escape \ as \\ according to the doc

Reviewed By: markbt

Differential Revision: D8024433

fbshipit-source-id: 14fb84741dfbd066d8278146bc6fef0cd76846c6
2018-05-16 08:50:41 -07:00
Saurabh Singh
a4f5e8aefc add tests for empty union date store
Summary:
These tests just test the expected outcome when the union data store
is empty.

Reviewed By: quark-zju

Differential Revision: D8018975

fbshipit-source-id: a2cc4c87509b857dbf5f6af506f165ea62080db8
2018-05-15 18:37:47 -07:00
Saurabh Singh
cab75fbb90 add common traits for Key
Summary:
This commit derives the common traits for the Key type just as we did
for the Node type in D7872300.

Reviewed By: quark-zju

Differential Revision: D8018973

fbshipit-source-id: 566a69be16d74529c6eb5f157b84de25835f780f
2018-05-15 18:37:47 -07:00
Saurabh Singh
88a2d8ff6b implement quickcheck::Arbitrary for Key in revisionstore
Summary:
We need to implement `quickcheck::Arbitrary` for Key so that it can be
used for the quickcheck tests.

Reviewed By: quark-zju

Differential Revision: D8018977

fbshipit-source-id: dbdbb34fbd7eaeb18321eafec4604d752f496a4d
2018-05-15 18:37:47 -07:00
Saurabh Singh
0b5b994973 implement quickcheck::Arbitrary for Node type in revisionstore
Summary:
We need to implement `quickcheck::Arbitrary` for Node so that it can
be used for quickcheck tests.

Reviewed By: quark-zju

Differential Revision: D8018978

fbshipit-source-id: ceda99622370bee6e9d05b839f9856c0526f553c
2018-05-15 18:37:47 -07:00
Saurabh Singh
411736dc7c add 'quickcheck' crate to revisionstore
Summary:
I am planning to use the `quickcheck` crate for testing the union data
store. This commit just adds the crate to the revisionstore.

Reviewed By: quark-zju

Differential Revision: D8018974

fbshipit-source-id: d390deeb01aa7d1bf1e66bb5bc948d48bd3f269e
2018-05-15 18:37:47 -07:00
Saurabh Singh
8688e1cc5e union data store: union data store implementation in Rust
Summary:
This commit just introduces the `UnionDataStore` and implements the
`DataStore` trait for it.

Reviewed By: quark-zju

Differential Revision: D7801615

fbshipit-source-id: 14eabd2aa1b1e085de94aec126a7108231ec6e8d
2018-05-15 18:37:47 -07:00
Saurabh Singh
d2b9c6c6ac union store: introduce common type for the union store implmentation
Summary:
We will be implmenting multiple union stores and therefore, it makes
sense to encapsulate the common logic in its own type. This also abstracts the
usage of `RefCell` within the union store.

Reviewed By: jsgf

Differential Revision: D7884651

fbshipit-source-id: a74b6d9df5ee0d7d04359219e276fd5713b3a00b
2018-05-15 18:37:47 -07:00
Saurabh Singh
bcb7ac0b32 node: add common traits for the node type
Summary:
Based on the review comments for D7801615, this commit adds the common traits
for the `Node` type

Reviewed By: jsgf

Differential Revision: D7872300

fbshipit-source-id: 44dedfc3ec0e18ac0dee0dcfc5f7dfc4aff2511d
2018-05-15 18:37:47 -07:00
Michael Bolin
36eb4d26f8 mv fsmonitor/watchmanclient.py to extlib/watchmanclient/__init__.py
Summary:
There is some logic in `hgext/fsmonitor/watchmanclient.py` that needs
to be shared between fsmonitor and the new extension we are trying to
split out of it as part of T29379329.

Reviewed By: quark-zju

Differential Revision: D8003255

fbshipit-source-id: de01c5ba1460d7dde22969980b296afb3a942dd7
2018-05-15 12:06:47 -07:00
Michael Bolin
54b487b301 Move hgext/fsmonitor/pywatchman/ to hgext/extlib/pywatchman/
Summary:
This is a precursor to splitting the fsmonitor extension, as both
it and the new extension will use pywatchman.

Reviewed By: quark-zju

Differential Revision: D8002713

fbshipit-source-id: 37983fe2898d23223d1178eb3f15685f17ff8868
2018-05-15 12:06:47 -07:00
Liubov Dmitrieva
76c5b37722 rebase: do not rely that every node in obsutil.allsuccessors present in the repo or changelog node map
Summary:
This is regulary not the case if Commit Cloud enabled, if a lot of work is done on one
machine 1, then cloud sync is done on another machine (machine 2), we sync draft not obsolete
heads and all history of obsmarkers, but there is no guarantee we download all
the obsoleted commits to the machine 2.

So, there can be a sequence of obsmarkers from commit A to commit B, when some commit in between hasn't been fetched to the machine 2.

Function obsutil.allsuccessors returns all successors (not only final successors) and it is intentionally do not check presence in the repo.

Reviewed By: DurhamG

Differential Revision: D8009823

fbshipit-source-id: f6b90011acaa518f71a2033eb2b885b27a7f06e7
2018-05-15 11:39:40 -07:00
Liubov Dmitrieva
eb93a4ce74 commit cloud: if option to show requests is on, also show the responses
Summary:
[hg] commit cloud: if option to show requests is on also show the responses

this is useful for investigating issues with Commit Cloud

the option is

```
[commitcloud]
debugrequests = true
```

only works with --debug

Reviewed By: markbt

Differential Revision: D7950549

fbshipit-source-id: ab7f07ad2a61f6bee9c627dfa3d3b43d6380e9cf
2018-05-15 08:17:35 -07:00
Durham Goode
8c6e5fd964 mpatch: add rust bindings
Summary:
Adds rust bindings around the existing mpatch c library.

Also fixes a bug in mpatch where it could reference uninitialized memory.

Reviewed By: quark-zju

Differential Revision: D7769299

fbshipit-source-id: bcc21df85c97ef6f5537ebff8fbf1b350ee64fc3
2018-05-14 16:06:32 -07:00
Alexandre Marin
5e3b491c6f importer - lowercase p4 path in p4->hg path mapping
Summary: This change makes all p4 paths lowercase when dealing with the mapping from p4 to hg, thus making the importer work fine with different case paths from Perforce.

Differential Revision: D7978382

fbshipit-source-id: 134705ac27d889e80e5de589ab165e8acfd52346
2018-05-14 14:46:48 -07:00
Jun Wu
e3010aca6b tests: restore simple error output for buck .t tests
Summary:
Partially revert D7584511 so buck test failures for `.t` tests would show
the `.t` output change instead of a long, less meaningful environment
variables.

Reviewed By: singhsrb

Differential Revision: D7992713

fbshipit-source-id: 64de5e19589277018e1e37968ea69a2329d67852
2018-05-14 13:00:29 -07:00
Mark Thomas
741ae1d88e smartlog: refactor smartlog templates
Reviewed By: ryanmce

Differential Revision: D7988363

fbshipit-source-id: 8f38f06360876af7f9b88a3e97c293e16dbd0f32
2018-05-14 12:05:13 -07:00
Durham Goode
0efe4fb7db infinitepush: add test for excess treemanifest downloads
Summary:
This adds a test that verifies that if we pull infinitepush commits and
received some normal-public commits as part of that pull, we will only receive
the tree data for the infinitepush commits. The commit cloud team had some
concerns here, since this will be much more common in that future.

Reviewed By: singhsrb

Differential Revision: D7993229

fbshipit-source-id: c91752e0da86a81142503ed97ac104668d1af874
2018-05-14 12:05:13 -07:00
Durham Goode
18697e3fb3 hg: implement getmissing() for Rust hgstore
Summary:
Initial implementation of getmissing for a simple Rust pass through
data store. Future diffs will extend this to implement the union data store
completely in Rust.

Reviewed By: quark-zju

Differential Revision: D7632405

fbshipit-source-id: e660d33f8231410805cfaba6d77c56f27b002f8e
2018-05-14 12:05:13 -07:00
Durham Goode
f2b4d7f2e0 hg: implement getmeta() for Rust hgstore
Summary:
An initial implementation of getmeta for the Rust hg data store. Future
diffs will add more functionality.

Reviewed By: quark-zju

Differential Revision: D7632404

fbshipit-source-id: 53bd3b96b777bc3c5aef2b4d07ce1a9d9a5a52ed
2018-05-14 12:05:13 -07:00
Durham Goode
39dde8552d hg: implement getdeltachain() for Rust hgstore
Summary:
An initial implementation of getdeltachain for a simple pass through
data store. Future diffs will add additional functionality.

Reviewed By: quark-zju

Differential Revision: D7632407

fbshipit-source-id: 1a38089ba8ea70f8772af95afd871ee493082d80
2018-05-14 12:05:13 -07:00
Durham Goode
880ff5d0a9 hg: implement datastore.get() for Rust hgstore
Summary:
Implements the get function for a simple pass through rust data store
layer. Future diffs will implement more functions, and then later we will
implement the entire union data store in Rust.

Reviewed By: quark-zju

Differential Revision: D7632403

fbshipit-source-id: 3a1d0a8500e3110213d70dc1cff637cf8eadd809
2018-05-14 12:05:13 -07:00
Durham Goode
a97e97e413 hg: initial boiler plate for new hgstore crate
Summary:
This will contain all the Python centric hg store code that will let
Python call into the Rust storage layer.

Reviewed By: quark-zju

Differential Revision: D7632406

fbshipit-source-id: 6b7bcc8f47a23e9c0121e1f92de1137369bf584e
2018-05-14 12:05:12 -07:00
Durham Goode
507677e8a0 infinitepush: add test for excess file downloads during pullbackups
Summary:
If a user runs hg pullbackup while their repository doesn't have the
latest commits, it's possible that pullbackups will deliver some public commits
to the user in order to then deliver the necessary draft commits on top. Let's
add a test that verifies the public commits are not delivered with remotefilelog
data.

Reviewed By: quark-zju

Differential Revision: D7927966

fbshipit-source-id: 8661a291c8b3a67359c9975e130f23fec643307d
2018-05-14 11:21:43 -07:00
Jun Wu
410cb5dcef xdiff: fix types on 32 bit systems
Summary:
Similar to upstream patches [1] [2].

[1]: 1b9f644050
[2]: https://phab.mercurial-scm.org/D3538

Reviewed By: DurhamG

Differential Revision: D7989782

fbshipit-source-id: fc11569d92f778c0a3509204fc9fa1690c078dcc
2018-05-14 11:14:38 -07:00
Jun Wu
f085bc4510 templatekw: define 'username' as current username
Summary:
This could be useful for implementing logic like "if this commit is by the
current user, show something differently".

Reviewed By: markbt

Differential Revision: D7964292

fbshipit-source-id: de1ac0b5edde838dbaae646a88ebf636b4925b22
2018-05-14 11:14:38 -07:00
Liubov Dmitrieva
77e1cb22e8 commitcloud: do not use filtered set of heads when update infinitepush state
Summary:
commitcloud: do not use filtered set of heads when update infinitepush
state

hg cloud sync command can be interrupted when pulling

some of commits will be pulled, but not written to the commit cloud and
infinitepush states

the next run of cloud sync is smart, it will not pull what has been already pulled in but will fix the
commit cloud state

but because we used the filtered list of heads for infinitepush state, it will not fix the infinitepush
state

So, we shouldn't use the filtered set

Reviewed By: markbt

Differential Revision: D7972200

fbshipit-source-id: 94c01694d4ac77beeed647f77cbdb30fe3f7a404
2018-05-14 06:42:19 -07:00
Jun Wu
5851c126a5 tests: fix helpers-usechg.sh
Summary:
I noticed some tests were running slower without chg. Upon investigation,
`alias hg=...` seems stop working. Similar to D7563731, let's define
functions explicitly instead.

Reviewed By: singhsrb

Differential Revision: D7964291

fbshipit-source-id: 08a69c865ffef6be7e84dd66a7cece9284b94e60
2018-05-11 16:20:55 -07:00
Jun Wu
cef17b6c33 chg: revert D7845335
Summary:
`pkill -f chgunix` is a convenient, but not a safe way to invalidate chg
servers. We're going to do the server invalidation in a better way. So just
revert the change.

Reviewed By: singhsrb

Differential Revision: D7978130

fbshipit-source-id: c13b75204ae1097ffe992b2e26d80d028022ff0d
2018-05-11 16:05:07 -07:00
Kostia Balytskyi
b955fc974e extutil: make background run truly background
Summary:
`DETACHED_PROCESS` simply means that child process won't inherit
current process' console (it does get one allocated by the OS though). `CREATE_NO_WINDOW` means that child
process does not get an allocated console altogether.

For details: https://msdn.microsoft.com/en-us/library/windows/desktop/ms684863(v=vs.85).aspx

Reviewed By: quark-zju

Differential Revision: D7969621

fbshipit-source-id: 58d5e69808fb1064bcc1101c971f52088a7fd2de
2018-05-11 10:11:22 -07:00
Jun Wu
a2e7d1cfe3 treestate: implement Rand for FileStateV2
Summary: This will be used in tests.

Differential Revision: D7769655

fbshipit-source-id: 27647685848c03f56740f49361dd286abdef8e33
2018-05-10 16:40:25 -07:00
Mark Thomas
6b20072672 commitcloud: use source repo when writing infinitepushbackupstate
Summary:
The infinitepushbackupstate file must be written using the vfs of the source
repo when the share extension is in use.

Reviewed By: mitrandir77

Differential Revision: D7936097

fbshipit-source-id: a57e241ca969632ced65029cb5ccf61373bd8aeb
2018-05-10 11:32:30 -07:00
Mark Thomas
37a68060fa commitcloud: prevent race between creating and syncing obsmarkers
Summary:
In order to prevent newly added obsmarkers from being lost while we are syncing
them, we separate out the obsmarkers that we are in the process of syncing into
a separate `syncing` file.  New obsmarkers that are created during sync go into
the `pending` file and are synced on the next sync, which means it's safe to
delete the `syncing` file when sync completes.

Differential Revision: D7932299

fbshipit-source-id: bd86e836ded10e75790b87ca6734a29f068f3571
2018-05-10 03:06:03 -07:00
Mark Thomas
eef982accb commitcloud: add test demonstrating obsmarker race
Summary:
A command that creates obsmarkers while they are being synced can cause the
obsmarkers to be dropped.

In this test, the obsmarker for the rebase restack is lost.

Differential Revision: D7932300

fbshipit-source-id: 11988f2a2c77eed9f9fab258a6623abd8c50e1cd
2018-05-10 03:06:03 -07:00
Jun Wu
4d4a8d26ae treedirstate: do not print debug messages during cleanup by default
Summary:
They got randomly printed out and broke tests. Let's limit it to
"debugtreedirstate" command.

Reviewed By: singhsrb

Differential Revision: D7904376

fbshipit-source-id: 4f0b3708fdc55ed207c3c3baa2926f4b2374730f
2018-05-09 19:32:15 -07:00
Jun Wu
2b34292f89 tests: remove test-fb-hgext-fastmanifest-concurrency.t
Summary:
It's flaky and fastmanifest is no longer used in production. So let's just
remove the test.

Reviewed By: singhsrb

Differential Revision: D7945558

fbshipit-source-id: a6a4b34d40f0994223f9eebc41a34ffdcb260032
2018-05-09 19:32:15 -07:00
Jun Wu
599c63b56f tests: fix test-gendoc-ro.t
Summary:
D7845334 seems to "fix" the tests - the warnings are gone like other
`test-gendoc-*.t`. So let's update the test.

Reviewed By: DurhamG

Differential Revision: D7937819

fbshipit-source-id: 00ec52f5860c9ad7a5ab3b42a3d6db87124b7fbe
2018-05-09 13:28:29 -07:00
Jun Wu
8a679289b2 tests: fix test-fb-hgext-scm-prompt-hg.t
Summary: Follow up of D7874269.

Reviewed By: singhsrb

Differential Revision: D7921934

fbshipit-source-id: 4aa4fa52e93bf9d21e8369d4c5891d3046af01eb
2018-05-09 11:28:18 -07:00
Durham Goode
0fcd554c57 treemanifest: enable ondemand tree generation when sendtrees is enabled
Summary:
Previously we only enabled on demand tree generation when treeonly was
set, but we also need it when sendtrees is set since we'll need to generate
trees for sending.

Reviewed By: quark-zju

Differential Revision: D7927856

fbshipit-source-id: a69d6c7920a92e4f90bdcd1d04aad9ef59e9c778
2018-05-09 11:28:18 -07:00
Durham Goode
a25ff90a5e clone: disable pull during hgsql streaming clones
Summary:
hgsql depends on repositories being byte for byte identical, but the
current pull after a streaming clone can cause the repository to be different
(like if different delta decisions were made, or the commits were ordered
slightly differently). Let's disable that pull when the repository is an hgsql
repo.

Reviewed By: ryanmce

Differential Revision: D7925300

fbshipit-source-id: 6eba7ad4ccdd37f6d7c5090522867d1a54f722b7
2018-05-09 06:15:20 -07:00
Jun Wu
6596190811 tweakdefaults: avoid crash with "rebase -r foo -d func(SRC)"
Summary:
The destination revset could contain `SRC` that will break tweakdefaults.
Change the ordering so it could work.

Reviewed By: singhsrb

Differential Revision: D7924127

fbshipit-source-id: cb3eb1379425303607e4cb6f57f534133d457dea
2018-05-09 00:26:47 -07:00
Jun Wu
f7228afd10 chg: drop "sensitive config sections"
Summary:
The "sensitive config sections" was used because "hg serve" loading
different extensions are incompatible with each other. Now we neither load
extensions nor run their uisetups, and just use one chg server.  So
sensitive config sections can be removed.

Reviewed By: singhsrb

Differential Revision: D7847149

fbshipit-source-id: 758c1df21d280bf0f88d91432e1201c8417df532
2018-05-08 17:02:22 -07:00
Jun Wu
4d5d548fb7 chg: change process title immediately after fork
Summary:
`pkill -f chgunix` was used to kill chg server processes. That could kill
worker processes if the worker hasn't changed its process name. Let's change
the process name early after forking to reduce the window a chg worker can
be killed by `pkill -f chgunix`.

Reviewed By: singhsrb

Differential Revision: D7845335

fbshipit-source-id: 3b3fc64c4328a058e0c6aad3cb11b0bc4efd9d13
2018-05-08 17:02:22 -07:00