Summary:
Previously, if the rust walker encountered any issues with individual
files it would stop and throw an exception or in some cases panic. To replace
the python walker we need to handle errors more gracefully, so let's add logic
for reporting which files had issues and what those issues were.
This also contains a minor one-line fix to prevent the walker from traversing
directories that contain a '.hg' directory.
Reviewed By: quark-zju
Differential Revision: D19376540
fbshipit-source-id: ede40f2b31aa1e856c564dd9956b9f593f21cf27
Summary:
The command shows how visibility, bookmarks, remotenames change in the repo
over time, using metalog data.
Also add this command to hg rage report.
Reviewed By: markbt
Differential Revision: D19358668
fbshipit-source-id: 9bddc039143b45c29c0c937ee60906ab93a2defe
Summary: The colors are from the our smartlog template. They're used in the next change.
Reviewed By: markbt
Differential Revision: D19358670
fbshipit-source-id: 7476f95767c5fcf1cccdce2119a2ab7ce45d270e
Summary:
We don't want automigration to block normal execution of the command,
so let's take the lock with a short timeout, then skip automigration if we don't
get it.
This should solve an issue where debugedenimporthelpers were hanging because
they tried to take a lock that a hg process that was accessing eden held.
Reviewed By: quark-zju
Differential Revision: D19375544
fbshipit-source-id: 6c107aac87b5b839cc2d347a3b40ce67da627b71
Summary:
On Windows, select cannot be used for pipes, and thus the doublepipe class
would end up busy spinning trying to read from it. Since stderr is almost
always empty let's move it to a background thread so the main code path
doesn't have to deal with it.
This proves to be a significant win for `hg pull`. For an empty manifest
cache, `hg pull` used to take ~5 minutes, it's now around 1 minute, with
Mononoke taking 40 seconds out of that.
Reviewed By: DurhamG
Differential Revision: D19352144
fbshipit-source-id: 77112c1c18d7dab15f6946a645f73e89908802f2
Summary:
Use `encoding.unitolocal` to convert the Rust-generated unicode strings to the
local encoding.
Reviewed By: quark-zju
Differential Revision: D19371723
fbshipit-source-id: 6b37ef51f37f2212ca2d9ab20f9ed4da29b3d63c
Summary:
Box-drawing characters with curves aren't reliably renderable on Windows. Add
a "square" glyph set that uses right-angle characters. These characters are
available in cp437 and cp850, so should be available on most Windows systems.
For completeness, add a "dec" glyph set that uses DEC line drawing characters,
for use in old terminals like xterm.
Reviewed By: quark-zju
Differential Revision: D19371722
fbshipit-source-id: 35887243cceab66c702e2b5278b572f77946805f
Summary:
D19340538 deleted `eden/scm/edenscm/mercurial/rust/__init__.py`, but
chg's `_preimportmodules` still tries to import it, which breaks Mononoke
```
$ hg init repo-hg
+ Traceback (most recent call last):
+ File "/data/users/torozco/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/__init__.py", line 61, in run
+ dispatch.runchgserver()
+ File "/data/users/torozco/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/mercurial/dispatch.py", line 336, in runchgserver
+ _preimportmodules()
+ File "/data/users/torozco/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/mercurial/dispatch.py", line 319, in _preimportmodules
+ __import__(name)
+ ImportError: No module named rust
+ [1]
```
NOTE: I had a `__init__.pyc` left in the directory removed in D19340538, which
I'm guessing is why was still working on a `make local` build. After removing
that directory, a `make local`-built hg crashes consistently with the same
error for me.
Reviewed By: farnz
Differential Revision: D19371748
fbshipit-source-id: 4f57b5e1f75b2bd6e857303111fc01446124417b
Summary:
This makes it possible to use zstore instead of 00changelog.d to store commit
data. Right now it double writes to zstore and 00changelog.d if turned on.
In the future we can switch to only writing to zstore if 00changelog.i
alternative is available.
Some related configs were added for fallback strategy: revlog or server.
The revlog fallback allows us to do fast migration. The server fallback
(not implemented in this diff) allows us to stop depending on revlog.d.
Reviewed By: DurhamG
Differential Revision: D18737260
fbshipit-source-id: 3c9605f0babd8a215ee74bdf1275cc4e9dbd766c
Summary:
The `00changelog.d` file contains commit data (user, date, message, but no
ordered parents).
By setting the `zstore` attribute on `changelog`, reads from `00changelog.d`
(aka. the `revision` method) are going to use `zstore` instead. Writes will
go to both `00changelog.d` and `zstore`.
Reviewed By: DurhamG, xavierd
Differential Revision: D18737258
fbshipit-source-id: fd1cf52a77fe396df66b77c4df77089326f5d4c2
Summary:
For some reason, our linker flag `-uinitthreading` no longer works in buck opt
build. This affects the `indexes` and `threading` Python extensions. Move them
to `bindings` to solve the problem.
Reviewed By: fanzeyi
Differential Revision: D19340538
fbshipit-source-id: 0c82b472f2916229d3c5e481954b69415b5b0dc2
Summary: I would like to link the Phabricator Diff Number to specific mercurial revisions. After talking to Durham, the least invasive and best method for accomplishing this is to log the Phabricator Diff Number at runtime. Hence, whenever someone modifies the current commit, log the Phabricator Diff Number (when possible). The resulting table will theoretically have a map from "revision" -> "Phabricator diff number".
Reviewed By: DurhamG
Differential Revision: D19334379
fbshipit-source-id: eb04a8af4dbeb9a7e130a268fce8a6c5d26ebfa9
Summary:
In the refactor to the new tracing framework, the threshold comparison
got changed and became a seconds-vs-milliseconds check, which therefore almost
never passes. Let's change it back to seconds-vs-seconds.
Reviewed By: quark-zju
Differential Revision: D19322203
fbshipit-source-id: d9cfb3854d3ee6b8c7ab5309544abd64af51027f
Summary:
D19114546 introduced a new rollback mechanism where it updates to the
original commit if the user quits. Unfortunatley it only did the update if the
user had made a commit, so if they exitted immediately before making a commit,
it left them on the wrong commit and all their changes are pending.
Reviewed By: quark-zju
Differential Revision: D19334201
fbshipit-source-id: c959096adfa3563894578436e8252c037b50e32c
Summary:
The Rust bindings now provide a subset of time parsing features. Replace the
old Python implementation. This has multiple benefits:
- Strong gurarnatee that Rust and Python behave the same.
- Parse relative time (ex. `5 minutes ago`)
- Parse date beyound i32 range (ex. >= year 2038)
Reviewed By: DurhamG
Differential Revision: D18946332
fbshipit-source-id: 721f47bc5b2835d7ca0a05ab34ea4faa1a411a4e
Summary:
This is causing a deadlock when an hg process takes the lock, accesses
an edenfs file, then eden launches debugedenimporthelper which also tries to
take the lock. Let's back it out for now.
Reviewed By: krallin
Differential Revision: D19331038
fbshipit-source-id: b8480c2e07079c3babeae8de48c02affa43188fc
Summary:
The Rust Contentstore code assumes that the prefetch request doesn't contain duplicated
filenodes so let's do it on the Python side.
Reviewed By: quark-zju
Differential Revision: D19318847
fbshipit-source-id: 73375de60f4fb81021eba16506037984eebdf048
Summary:
We had eden-backing repos that were using formats from years ago
because they never ran pull to trigger the automigrate. Let's automigrate as
part of debugedenimporthelper to keep these repos up-to-date.
Reviewed By: quark-zju
Differential Revision: D19320510
fbshipit-source-id: 475a6acf8748677de3183dfb5d4c6df7caca606d
Summary:
If users or automation invoke a bunch of really long commands, it can
occupy a lot of space in our logs. The full command for really long commands is
unlikely to be useful, so let's truncate them to 256 bytes.
Reviewed By: ikostia
Differential Revision: D19317932
fbshipit-source-id: 428479a18dd7e5a7ae4d8f862cf26c02c15b1fcb
Summary:
It turns out some old repos are still using the legacy treedirstate
format, which broke when I made this a hard requirement. Let's roll this back
for now.
Reviewed By: singhsrb
Differential Revision: D19313773
fbshipit-source-id: ded78be033cb04323a2e1d237059947dee509df9
Summary:
Interactive smartlog history relies on the order from the server, which has
changed, and so is now incorrect. Sort the versions from the server so
we can be sure they're always in the right order.
Interactive smartlog history also ignores the workspace parameters to specify
the workspace. Fix that.
Reviewed By: farnz
Differential Revision: D19309253
fbshipit-source-id: 9fc1b443e72557331e22d0987c73286f5c64e16d
Summary:
We no longer user tags in any of our repositories, so let's delete all
the code and remove some tech debt.
Previously drawdag was our last remaining use of tags, but in a previous diff I
switched that to use bookmarks.
Reviewed By: quark-zju
Differential Revision: D19042226
fbshipit-source-id: d7871c71c1026599aa70fdd806418347ea214a8a
Summary:
We want to delete all the non-treestate dirstate implementations. Let's
start throwing an exception if treestate is not enabled. We temporarily have a
bypass in case we break an important usecase in the process.
This also sets the standard new repo to be created in treestate mode, but adding
treestate to newreporequirements.
Reviewed By: quark-zju
Differential Revision: D19204621
fbshipit-source-id: 61c0c69ce6e559615ef570d2343bea78c1b5d5ee
Summary:
Tweakdefaults adds the ability for `hg rebase -d` to simply fastforward
the user if the destination is a descendant of the current working copy parent.
If the working copy is dirty, it merged the files, potentially leaving the
working copy with merge conflicts that are hard to recover from.
A normal hg update disallows merges that would result in potential conflicts.
Let's update the rebase code to use the same merge logic, so fast forward
rebases simply abort if there are potential conflicts.
Reviewed By: quark-zju
Differential Revision: D19204198
fbshipit-source-id: 1b0c2c0fdab909eebe74d1a211dbfb2d780058cd
Summary:
Allow rendering of cloud smartlogs using renderdag. Use the ability of renderdag
to reserve columns to provide the smartlog-like indentation of draft stacks.
Reviewed By: quark-zju
Differential Revision: D19272577
fbshipit-source-id: 3311d83511b0d5347b45f33c282ce26b8133d047
Summary:
Allow rendering of graphs using renderdag by setting `experimental.graph.renderer` to `ascii`, `ascii-large` or `lines`.
The minimum height of each row can be set using `experimental.graph.min-row-height`.
Reviewed By: quark-zju
Differential Revision: D19272580
fbshipit-source-id: ac5eff8446bd3c09d07624a193195b4224055ffc
Summary:
This can happen for "Reserved" diffs.
Reported By: hyr
Differential Revision: D19270860
fbshipit-source-id: 24e9b8c08fc16e3089e0dc3f8064a5e9671d9ace
Summary:
Sometimes GraphQL does not return JSON result. Do not crash on the client-side.
Context:
https://fb.workplace.com/groups/scm/permalink/2565497036833234/
Reviewed By: farnz
Differential Revision: D19254348
fbshipit-source-id: 75ca93b1dc8648d5c59915ecf450b9366df13c01
Summary:
Make cpython-ext free from business logic. This adds a bit overhead.
But it should be fine given the error cases are considered rare.
Reviewed By: markbt
Differential Revision: D19186692
fbshipit-source-id: daaffc1369a36c781d1badea822bf62a144eb54e
Summary:
We no longer use the `failure` crate. Rename the module.
`cpython-failure` is dropped since it was merged into `cpython-ext`.
Reviewed By: markbt
Differential Revision: D19186693
fbshipit-source-id: 410d1491bcadd8d3272c7e2df08ecbe66fc0fef2
Summary:
The `map_pyerr<PE>` API is kind of hard to use correctly since the
code that calls `map_pyerr` does not really know the best suitable error type.
The existing code also "encourage"s the use of stdlib error types like
RuntimeError, ValueError, etc since the code is shorter, which is not great.
Error types from `mercurial.error` are better choices.
With previous diffs in this stack, we now decide the Python error type based
on the Rust error type. It's no longer necessary to specify an error type
with `map_pyerr<PE>`. So let's just drop the type parameter.
A `RustError` was added for error types that `cpython-ext` does not know how
to convert. We should avoid leaking it to the Python land by just implementing
how to convert `anyhow::Error` to `PyErr` in `cpython-ext`.
Reviewed By: markbt
Differential Revision: D19169527
fbshipit-source-id: f4563c36174cd51201b526bbc92a3f1c8a3da864
Summary:
Since we now have the indexedlog error type. Match it and print helpful
messages.
Reviewed By: markbt
Differential Revision: D19169207
fbshipit-source-id: cd874f651b681d2dbbb5aea613c56721c4ab6181
Summary:
The hgpython interpreter was used to run Python scripts in tests that might
rely on Mercurial modules.
The previous hgpython implementation is Python PAR based, which does not have
access to native Rust modules like bindings. Change it to use the native
"hg debugpython" implementation that is more compatible.
Reviewed By: markbt
Differential Revision: D19190496
fbshipit-source-id: 9791dbf9ba0ed92de702291faa9145f01b05ec40
Summary:
The current error message looks like fatal while it is not. Make it clear that
extension failures are not fatal.
Reviewed By: markbt
Differential Revision: D19166545
fbshipit-source-id: 7ce0231c90b0c5f50add15a12cb69b60b76dfcb1
Summary:
There are 3 extensions that can provide the `{gitnode}` template: "gitrevset",
"fbscmquery", "hggit". The last can read from commit extras. Fix the first two
to also read commit extras. At some point we need to unify the implementations.
Reviewed By: farnz
Differential Revision: D19198979
fbshipit-source-id: 24a0df0df4ceb7a9af9236a7f70babfd54e9802b
Summary:
D17099990 changed the way status is computed and added a loop over the
nonnormal files to find any added, removed, or merged files. Unfortunately the
nonnormal file list can contain untracked ignored files in some cases, and if
that list is large (like if someone disabled a sparse profile) this loop can
take many seconds.
Let's updated the loop to filter out ignored files. This matches a similar loop
in fsmonitor, and we may want to think about removing nonnormal handling from
the fsmonitor code entirely now that it's handled in dirstate.py.
Reviewed By: quark-zju
Differential Revision: D19190259
fbshipit-source-id: 4ba4150507fdd72439bc4e5eb731a951c6100f5f
Summary:
I encountered a 19s `hg sl` run where 13s were spent on
`remotenames.updateaccessedbookmarks` (8 times).
{F225347899}
The filesystem is unusually slow since the it's nearly full. However, the
updates are not really necessary since I'm not accessing more remote names.
Accessing one bookmark via smartlog rewrites the entire accessed file, which is
O(N^2) and undesirable.
Improve it by skipping writing if nothing has changed. In the future we might
want to optimize it further by:
- Only update (and do ui.log) "accessedbookmarks" at most once per command.
- Potentially drop the "accessedbookmarks" tracking by using "remotenames"
directly - everything in "remotenames" is accessed.
Reviewed By: markbt
Differential Revision: D19191528
fbshipit-source-id: 46635d4e5c9d0034ace9cdafc1f42a4512aa8774
Summary:
Currently, hg's lfs extension used bytes concatenation to download a response
from the server-side. However, concatenating bytes in Python can be
inefficient, since it'll re-allocate the whole thing and copy it every time you
concatenate.
This makes repeated concatenation O(N^2) on the number of things you
concatenate, which is undesirable. This diff updates hg's lfs extension to not
do that.
Reviewed By: farnz
Differential Revision: D19194388
fbshipit-source-id: f4a38afd8ba41bc1c2e64e9127035675ecda6651
Summary:
HTTP makes no provision to tell your client that you failed halfway through
producing your response and won't have the answer they're looking for. So, if a
LFS server fails while producing a response, then we'll report an OID mismatch.
We can do a little better and disambiguate between "the server sent us the
wrong blob" (very scary) and "the server crashed" (merely annoying) by looking
at the content length of the response we got back. If it's not what was
advertised, we can reasonably safely assume the server crashed.
Reviewed By: farnz
Differential Revision: D19194389
fbshipit-source-id: 569bd20cffe2901e2801261ce783e99bcf8358e2
Summary:
We don't support any of these VCS's, and we're not running any of the
tests. The code is just bit rotting. Let's delete them.
Reviewed By: quark-zju
Differential Revision: D19043710
fbshipit-source-id: 6e0d625c755cbc875755dc09b394bc730186db1d
Summary:
On Windows, subprocess.Popen caused the current process to wait until
the background process was complete, which meant commit, amend, rebase, etc had extra long waits on the end. We have utility functions that correctly set up the background process, so let's use that instead.
Unfortunately we can't redirect stdin, stdout, stderr when using close_fds on
Windows, so we lose the ability to do logging.
Reviewed By: quark-zju
Differential Revision: D19149682
fbshipit-source-id: 6880ec7833e286b79b34c49c339aff3318f07295
Summary:
This is part of the logic of remotenames. Unfortunately our production setup
has some really annoying legacy code in multiple places that disables
remotenames and do `rm .hg/bookmarks` manually. They should really enable
remotenames and get rid of `rm .hg/bookmarks`. However that has complications.
With recent changes, `.hg/bookmarks` is moved to `.hg/store/bookmarks` and
later moved to `metalog`. That caused 10k+ bookmarks left in local bookmarks
after clone (or pull).
Writing remote bookmarks as local bookmarks makes no sense in modern setup.
Therefore this diff adds a config option to disable such feature.
The ideal state is the bookmark logic in exchange writes remotenames directly,
instead of silently ignoring it. But that's a risky and larger change. My
expectation is, the remotenames extension will be enabled and take care of the
rest.
The idea of not writing local bookmarks come from markbt.
Reviewed By: DurhamG
Differential Revision: D19148278
fbshipit-source-id: a03eac68378908586d6e6848ed43532c0aab06c3
Summary:
A future diff will remove the tags feature. Convert uses tags heavily
and it breaks a number of tests, so let's remove tags from the convert extension
before we remove tags entirely.
Reviewed By: quark-zju
Differential Revision: D19043330
fbshipit-source-id: 628d27fea1601931da45d1280eff981c5d12f93c
Summary:
In a future diff we'll be removing tags. The most prevalent tag is
'tip', which shows up in a ton of test output. Let's drop that tag first, so we
can safely update the tests before we drop tags entirely.
Reviewed By: xavierd
Differential Revision: D18995058
fbshipit-source-id: 8c63710cd4ed567ea24e32724b8660f9006a61f1
Summary:
A future diff will remove tags entirely, so let's move drawdag to be
bookmark based.
Reviewed By: quark-zju
Differential Revision: D18995059
fbshipit-source-id: 70ef67259b37ef9821009d0145aa1e03c09b1309
Summary:
Rolling back the entire transaction can causes Eden to get out of sync
with Mercurial and cause hangs. It can also causes users to lose the work
they've done splitting the commit.
Let's change split to commit the transaction after every commit. If a user exits
mid-split they will now be left in a state where the splits they've done so far
are still present, and we print a message telling them how to roll all the way
back if they want to.
Reviewed By: quark-zju
Differential Revision: D19114546
fbshipit-source-id: b67d5543f9c23a6299f4164c73662759661f59a9
Summary:
The current code crashes with `phaseroots` being an unknown attribute.
Reported By: DurhamG
Reviewed By: DurhamG
Differential Revision: D19048762
fbshipit-source-id: f59764ec04284cd643fbb5a3c319868df06c4912
Summary: There are some modules missed in the preimport list. Add them.
Reviewed By: xavierd
Differential Revision: D18957117
fbshipit-source-id: 254384f0bcafd52421fb22dc696b3a6a1fb98537
Summary:
This is mostly result of:
```
find . -type f -exec sed -i "s/fbconduit/fbscmquery/g" {} \;
```
`fbconduit` extension doesn't use conduit anymore so the name is just misleading.
Reviewed By: ikostia
Differential Revision: D18748843
fbshipit-source-id: 0d59e61ba7a96d86d9d1333d81255108cc3141bc
Summary:
This is showing in profile for about 10% of the time, let's just disable it as
getting files should be fairly quick.
Reviewed By: quark-zju
Differential Revision: D17841315
fbshipit-source-id: 08a61e19351fb5217431a3ca8993b8074f6d6c74
Summary:
There is no need to duplicate the code.
The test change are due to commitpending being called 3 times, vs 1 before. The
total amount of times {fileslog,manifestlog}.commitpending is called is unchanged.
Reviewed By: quark-zju
Differential Revision: D17841316
fbshipit-source-id: be33a126f913af8824a6d5ade17351707d39ed2f
Summary: The smart traceback is generally more useful.
Reviewed By: markbt
Differential Revision: D18666054
fbshipit-source-id: a96bb3e2919ed6692c4a7b965ad74cd2c8a66241
Summary: Add type annotations to all functions in this file.
Reviewed By: xavierd
Differential Revision: D18949804
fbshipit-source-id: 4ac1789aa086e9e5f2397bc005a33cfb2f6c0b56
Summary:
D18263067 updated the `CMD_FETCH_TREE` handler to fetch tree with a depth
of 1, rather than using fetching the entire recursive tree structure.
I think this has contributed to some of the recent user-reported slowness for
fetch operations, as fetching a large directory structure now triggers many
more individual requests for each separate tree.
This diff adds a configuration parameter to control the tree fetch depth. I
have set the default value at 3, which should hopefully provide a reasonable
level of tree prefetching without completely fetching all contents of deep
trees.
Reviewed By: wez
Differential Revision: D18942585
fbshipit-source-id: 6a8d749434520baee25a4277712c44b916adcb3f
Summary:
I made a mistake when editing the last version of the code. It uses
indexedlogdatastore to repair indexedloghistorystore, which is incorrect.
Fix it.
Reviewed By: xavierd
Differential Revision: D18945952
fbshipit-source-id: addec020c761e6c11d2fe485eb408f5b200afb08
Summary:
Make util.smarttraceback:
- Support `traceback` object.
- Look more similar to normal traceback (headers, and indentation).
- Hide boring content.
- Show binary hashes in hex form.
Reviewed By: markbt
Differential Revision: D18666055
fbshipit-source-id: d61446604c673ec5e1a96912fa29935d4238ecc9
Summary:
See also D5271419. Decorators in `bundle2` and `exchange` are not side-effect
free. Move them to `uisetup` for better chg compatibility.
Reviewed By: StanislavGlebik
Differential Revision: D18926786
fbshipit-source-id: 2a3fa6ed4cef42c9bad4666ff75496d483991f88
Summary:
See also D5271419. Decorators in `bundle2` and `exchange` are not side-effect
free. Move them to `uisetup` for better chg compatibility.
Reviewed By: StanislavGlebik
Differential Revision: D18926782
fbshipit-source-id: b99e2882cb6530a648a065c6d739ee9a7aebb851
Summary:
See also D5271419. Decorators in `bundle2` and `exchange` are not side-effect
free. Move them to `uisetup` for better chg compatibility.
Reviewed By: StanislavGlebik
Differential Revision: D18926783
fbshipit-source-id: 9855b8debe3f809139a9fc3c6eea2775a1562cfd
Summary:
See also D5271419. Decorators in `bundle2` and `exchange` are not side-effect
free. Move them to `uisetup` for better chg compatibility.
Reviewed By: StanislavGlebik
Differential Revision: D18926787
fbshipit-source-id: cfbf8de6d8c841478d7aa3de180a0816349fa4e4
Summary:
The `reset` extension does its own obsmarker creation, rather than going
through `scmutil.cleanupnodes`. This means it doesn't support new-style
visibility tracking.
Fix this by making it use `scmutil.cleanupnodes`. This isn't completely
straightforward:
* The revset it uses to work out what to prune might accidentally include
public commits (e.g. when you reset to a different public branch). Make sure
these are filtered.
* The tests originally had strip and obsmarker based tests. The strip tests,
when converted to obsmarkers and using `scmutil.cleanupnodes` stop working
because they strip and revive the same commit over and over, which is an edge
case that obsmarkers can't handle well. Fix this by restoring the strip tests
as strip tests. A separate test handles the new-style visibility.
* Reset's behaviour is still a bit wonky. If an ancestor of the source commit
has other (non-bookmarked) descendants, then reset will try to prune those
commits. New-style visibility will ignore this, and this is tested in the
new test.
Reviewed By: farnz
Differential Revision: D18912817
fbshipit-source-id: cc115333407cf67d339c24fcd0807ddedce2660d
Summary:
This solve the following issue when running on devserver with demandimport enabled:
In [1]: ui.config?
abort: No module named windll!
Reviewed By: xavierd
Differential Revision: D18898009
fbshipit-source-id: bf4f3b12ac6f0a1628ed8ef805f6843868363542
Summary:
Since we got `repair` APIs in the Python land, use them to attempt to repair
things.
This would hopefully be able to fix some user-reported issues, usually
caused by hard reboots.
Reviewed By: xavierd
Differential Revision: D18737907
fbshipit-source-id: ef3deccbf4a1c959cff7e3eb78771acd9ae9bd01
Summary:
From Dulwich changelog:
0.19.0 2018-03-10
API CHANGES
GitClient.send_pack now accepts a generate_pack_data rather than a
generate_pack_contents function for performance reasons.
(Note: this ignores all push blocking failures!)
Reviewed By: quark-zju
Differential Revision: D18901269
fbshipit-source-id: 9510b3a516cb2fcbeae9f1f2072d8445e442a9ed
Summary:
Previously, it's fine to migrate up to metalog, and fine to migrate down once
since we double write data. However, re-enabling metalog can be problematic
since there is no code path to do "migrate up" again.
This diff fixes the issue by tracking what files (or "keys") are using metalog
as the source of truth in metalog. So we can accurately figure out whether to
migrate svfs files to metalog on demand.
Reviewed By: xavierd
Differential Revision: D18864424
fbshipit-source-id: e61e1790c231f9c88de869f413f27bb954a29920
Summary:
We only support the "C" policy, therefore let's inline it and simplify the code
significantly.
Reviewed By: quark-zju
Differential Revision: D18819680
fbshipit-source-id: 0bc0a12b85ddc42af007d3a20c0f4e8497bedb21
Summary:
This is not available on Python3, instead stat.ST_MTIME and friends should be
used to get an integer time. This is achieved by wrapping the output of the
stat function and overriding the st_[acm]time properties.
Reviewed By: quark-zju
Differential Revision: D18819679
fbshipit-source-id: 2911f3d47506456e927872d57f69253d903618e2
Summary:
Some users were reporting hg gc failures because it couldn't access a
hidden commit. It looks like there's a `repo.unfiltered().revs("draft()")`
line to get the list of draft commits, so we need to also use an unfiltered repo
when extracting all the pointers.
Reviewed By: quark-zju
Differential Revision: D18863471
fbshipit-source-id: 61220c1ec3dcbf55879211bb271b4d9b686d66a1
Summary:
D18732257 changed our fastlog integration from using conduit to using
scmquery. This changed the result type from a hash to a dictionary of {"hash":
"xxx"}, but the consumer code wasn't updated appropriately. This fixes it.
Reviewed By: mitrandir77
Differential Revision: D18833566
fbshipit-source-id: 2573cbf4f93c5e4aeb2515d42696526ceb51428e
Summary:
It was used to highlight "more interesting" (aka. extension) code. Nowadays the
non-extension part of code is also interesting. Therefore drop the extension.
Reviewed By: simpkins
Differential Revision: D18666053
fbshipit-source-id: 027190f742015bdb9e7dba109e7a28987af3cf68
Summary:
A previous change to the BFS iterator fixed it to return the root of the repository.
This influenced the `debugmanifestdirs` command which was manually
handling the root of the repository. This diff updates the command
to match the old behavior. Basically after the fix to BFS we are
not printing any characters for the path of the root. Forcing the debug command
to print of a '/'.
(Note: this ignores all push blocking failures!)
Reviewed By: quark-zju
Differential Revision: D18862167
fbshipit-source-id: 9ca6da707ae8ab037e88a2bbce3b3993b5d219e3
Summary:
In most cases nowadays people run into worse experiences when their operations
run through the fsmonitor path. Watchman is reliable enough that we can rely
on it to work better than the default crawling algorithms.
Reviewed By: quark-zju
Differential Revision: D18835322
fbshipit-source-id: 30873c7d4b1ec306e59b6152c20833cc0027cad5
Summary:
For now pushrebase updates working parent, bookmarks. This diff makes it also
update remote bookmarks.
This resolves a race condtion in remotenames where the pulled remote bookmarks
have unknown nodes and cause update to silently fail.
Reviewed By: sfilipco
Differential Revision: D18601035
fbshipit-source-id: 6f2c318cbf6b29a6427eeac6d374c1fb13e6155c
Summary:
Move it to configuration.
This gives better control over how grep is executed. Previously the assumption
was that xargs was present in PATH. This updates removes that assumption.
Reviewed By: quark-zju
Differential Revision: D18715471
fbshipit-source-id: 09c3098984588cd7478172db0bcfb8d5cb8243fd
Summary:
`reachableroots(roots, heads, includepath=False)` returns a subset of `roots`.
I implemented `reachableroots` using the documented revset using segmented
changelog APIs and noticed smartlog graph is not correct. This diff fixes
the documentation.
This diff was also sent upstream as https://phab.mercurial-scm.org/D7518.
Reviewed By: ikostia
Differential Revision: D18723363
fbshipit-source-id: ed56ae85345dcf102159c4df0eb19c02cef2bf27
Summary:
The `update` hook gets called even for in-memory updates that does not affect
real filesystem (or edenfs). That hook is set to `edenfsctl redirect fixup`,
which adds 0.35s overhead per commit when doing in-memory rebase.
Define a edenfs-only update hook to remove the overhead.
This is part of a stack of optimization that makes a 20s rebase to 5s (0.6s per
commit to 0.07s per commit).
Reviewed By: xavierd
Differential Revision: D18838470
fbshipit-source-id: cbe81f9dd89bff6d7c4bec3063d6014211f9aac6
Summary:
With narrow-heads before the new dag structure, reading phases is expensive.
Avoid it.
This is part of a stack of optimization that makes a 20s rebase to 5s (0.6s per
commit to 0.07s per commit).
Reviewed By: xavierd
Differential Revision: D18838468
fbshipit-source-id: 0847d2508114203f1d16a7240a43375d325103a5
Summary:
With narrow-heads, rebase operation does not need to read existing phases to
decide new phases. Therefore remove it.
This is part of a stack of optimization that makes a 20s rebase to 5s (0.6s per
commit to 0.07s per commit).
Reviewed By: xavierd
Differential Revision: D18838469
fbshipit-source-id: 1298d4f4b0d72a764a286478e76c90b42b3fc569
Summary:
sysplatform is defined as a byte string, and thus a byte string needs to be
provided.
Reviewed By: ikostia
Differential Revision: D18816931
fbshipit-source-id: c653f1319f160bc00d7a1c90b06ff3b4f4d035ad
Summary:
In Python3, unicode doesn't exist, instead of re-inventing the wheel, just use
unicode from the pycompat module.
Reviewed By: ikostia
Differential Revision: D18816929
fbshipit-source-id: d3ec9ae4f2190db64866568b1945b26e05b9c2a5
Summary:
Somehow, pyre isn't able to inline the value of ispy3 and propagate it properly
and thus complains about lots of code paths. Let's inline it everywhere.
Reviewed By: ikostia
Differential Revision: D18816333
fbshipit-source-id: 80f81cb91bef4c6831952f9469401d87040335e2
Summary:
The None value has an Optional type, but was overwritten by a string in the
subclass, let's just change the base value to an empty string.
Reviewed By: ikostia
Differential Revision: D18805103
fbshipit-source-id: 30113d5b4394163554b340917f3cf32178e4fe9e
Summary:
The copy method is inherited from list and pyre requires proper type
annotation.
Reviewed By: ikostia
Differential Revision: D18805104
fbshipit-source-id: efe828c6033af48754c0ef28ee52340616582e23
Summary:
The run method is inherited from Thread, and pyre requires proper type
annotation.
Reviewed By: ikostia
Differential Revision: D18805026
fbshipit-source-id: 4f9addc5d1d9655f3796342d99366df9f0cfe102
Summary:
The branchmap type inherit from dict and override the copy method, pyre
requires types for it.
Reviewed By: ikostia
Differential Revision: D18805028
fbshipit-source-id: 43de36df6f40e5a3485e83e8056b2a70cdebcfac
Summary:
These methods are inherited from Thread, and pyre wants proper type annotation
in this case.
Reviewed By: ikostia
Differential Revision: D18805027
fbshipit-source-id: 1d479c7af4da74220be3e175c8d1c0803da6d9e4
Summary: This updates the hg and telemetry wrapper callsites of getScmStatus to first try running getScmStatusV2() with fallback option. This does not retry `hg status` while a checkout is in progress.
Reviewed By: simpkins
Differential Revision: D18209899
fbshipit-source-id: e7a77b902f5a0ee624e4ea3185a1901bdac090e6
Summary:
On the LFS Server, we've noted that LFS batch requests from Mercurial have
about ~40ms latency, but that the same request sent through curl doesn't.
Those requests, when sent through curl, complete in < 1ms, so thats a lot of overhead.
After adding more logging in the LFS server (D18636703), it turned out that that is
because the body of the request shows up 40ms after the request itself.
After capturing the traffic on the backend (D18689109), it turned out that
Mercurial is sending the headers and the response body in 2 separate TCP
packets.
The reason for that is delayed TCP acks. What happens is that when we call
`h.endheaders()`, that calls `send(2)`, which in turns send the packet with the
headers. Later, when we call `h.send(data)`, we make another `send(2)` syscall,
but that doesn't go over the wire immediately: because of Nagle's algorithm, we
wait for an ACK for the first packet we send before sending a new one, and
because of delayed ACKs, the ACK takes 40ms for this ACK to show up (which is
over 40 times the roundtrip latency!).
This diff passes the data to `h.endheaders()`, which in Python's httplib will
concatenate the headers and the data into a single `send(2)` syscall if the
data isn't a file-like object (which it isn't for batch requests), and
otherwise fall back to doing exactly what we're doing right now.
This will result in everything in a single packet if we're fetching a single
file (and AFAICT, without further delay if we're fetching enough files that the
request doesn't fit in a single packet).
An alternative approach would be to set `TCP_NODELAY` on our socket here, but
for now this seems less intrusive and just as effective, so I've opted for that
approach.
Reviewed By: farnz
Differential Revision: D18811419
fbshipit-source-id: 4cf5719e5eed90e5dd994e6c8861aceb69373d89
Summary:
The Mercurial codebase contains over 500 errors, let's ignore them for now, we
can go back to them later to fix them.
Besides the manual change to .pyre_configuration.local, the changes were
generated with:
pyre --output=json check | pyre-upgrade fixme
Reviewed By: singhsrb
Differential Revision: D18803908
fbshipit-source-id: 724db7bd864c0de47a97ef2092bdee9f2cda531f
Summary:
The line causes pyre to enter an infinite loop, for now, let's simply ignore it
when type checking is enabled.
Reviewed By: singhsrb
Differential Revision: D18803909
fbshipit-source-id: d89b4cd0311a4a5416dd31197a8c69f4a6b65944
Summary:
When user types Dxxx as a revset locally they usually mean the latest version
of the commit - not neccesarily the one in phabricator. This usecase was
usually handled by doing local lookup which can be very slow in case of slow
commits: see for example those user complaints:
https://fb.workplace.com/groups/scm/permalink/2487795837936688
Reviewed By: farnz
Differential Revision: D18809252
fbshipit-source-id: b3442d6fa2ef9c9c0dff4909c874689810fbfa88
Summary: Log files that required manual merge during a rebase to dev command timers so we have an idea of how often they happen and on which files
Reviewed By: simpkins
Differential Revision: D18648175
fbshipit-source-id: 83ffe6d9177ca00b8fd1095745c585186bc2c8e9
Summary:
Since it's not calling conduit anymore we can remove all conduit related
functions.
Reviewed By: quark-zju
Differential Revision: D18733121
fbshipit-source-id: 20a6022d9ac7ae6e9afea2791a8daac57cefb22c
Summary:
conduit is going away, phabricator team wants us to stop using
it in hg.
Reviewed By: quark-zju
Differential Revision: D18732257
fbshipit-source-id: c4faf83e36af80fc616e91adede141ba12de5446
Summary: conduit is going away, phabricator team wants us to stop using it in hg
Reviewed By: quark-zju
Differential Revision: D18727399
fbshipit-source-id: c208f5fec5fefd83aa37629b56d2bbaa2532d30e
Summary:
In NFS v4.x, flock ends up inheriting some semantics of fcntl,
including that write locks can only be created against file handles
opened for O_WRONLY or O_RDWR.
This diff changes the mode of 'lockpath' to O_RDWR to solve this.
Reviewed By: simpkins
Differential Revision: D18714734
fbshipit-source-id: 84ba4a6a5de034a4942b1ca3aa8b0c92f882ce38
Summary:
Since bookmarks are stored in svfs, track it in metalog for better transaction
support.
This affects some transaction / rollback behaviors. Since our plan is to remove
`hg rollback` eventually, the compatibility is intentionally broken.
Reviewed By: xavierd
Differential Revision: D18538147
fbshipit-source-id: 6070772a4bce88d3808547207f5df194696b1b52
Summary:
Existing tools (ex. shell complete script) still read from
`.hg/store/{remotenames,bookmarks}`. Be compatible with them by double writing
content to both metalog and filesystem. Metalog is the source of truth for hg.
Reviewed By: xavierd
Differential Revision: D18524106
fbshipit-source-id: c37bc86b7443bba1b65735f29243767650b22ecd
Summary:
Similar to D17199834, by moving 'bookmarks' from localvfs and sharedvfs to
storevfs, we can make it part of metalog and get better transaction support.
Reviewed By: xavierd
Differential Revision: D18524104
fbshipit-source-id: ae148c1f02fc83b5c2d73102ecab39ff223ea5df
Summary:
Right now `bookmarks` is stored at both "local" vfs and the "shared" vfs.
In an upcoming change I'd like to move bookmarks to the "metalog" to get
better transaction. Therefore make it always shared.
The shared bookmark feature is turned on by default. Assuming nobody manually
turns it off, production hosts all have shared bookmark turned on.
Reviewed By: simpkins
Differential Revision: D18524103
fbshipit-source-id: 3160146f357dd01f2d78f5d2f4a14838f8f12a94
Summary:
The only valid place that metalog can be changed is inside a transaction, since
it's transaction.{close,abort} that writes or discards metalog changes.
In other words, metalog should not be changed during A, B, or C:
|<- A ->|<----------- repo lock --------->|
|<- B ->|<- transaction ->|<- C ->|
Add detection for them.
Reviewed By: xavierd
Differential Revision: D18538143
fbshipit-source-id: 036286ed32a897fe3ce0a91c1e3c848cc6167b1d
Summary:
Before this diff, metalog has a same lifetime as repo.svfs and never gets
reloaded or dropped. That is problematic in case external processes also
make changes, ex. running `hg amend` in `histedit exec`.
pid 1> histedit action foo (a single transaction, metalog loaded here)
pid 1> histedit action exec ... (a single transaction)
pid 2> hg amend (changes metalog)
pid 1> histedit action bar (cannot perform this action because metalog has
conflicts!)
This is why test-mutation.t didn't work with metalog.
Fix it by discarding metalog state at transaction/lock boundary (enter/exit).
The next two diffs add checks so we wouldn't discard uncommitted data silently.
Reviewed By: xavierd
Differential Revision: D18519995
fbshipit-source-id: ceb030362d66ad4be142e81accb82a4afa67f305
Summary: This resolves ProgrammingErrors detected by a later change.
Reviewed By: xavierd
Differential Revision: D18538145
fbshipit-source-id: 650e956707af6024457cdc7dabf516d087ad03cd
Summary:
"outstanding merge conflicts" is not a helpful message to inexperienced users.
Print out commands to get more context.
Reviewed By: simpkins
Differential Revision: D18535010
fbshipit-source-id: 035ec9f3d79bf04a997ee907469f2e3d749a1d0e
Summary: Some commits in AOSP have empty fields for author/committer email addresses.
Reviewed By: tchebb
Differential Revision: D18661778
fbshipit-source-id: e975392da677879d598eb9fc77558251a55c2f23
Summary:
Now that the fb-mercurial sources are available in the Eden repository, update
our CMake build files to always build them. This moves the build logic from
the centralized `FBMercurialFeatures.cmake` file into `CMakeLists.txt` files
in the appropriate subdirectories.
Reviewed By: chadaustin
Differential Revision: D18588011
fbshipit-source-id: ded9decde5c2ec766aae0bb0f4f5b021d1044a98
Summary:
D8997967 changed `hg rage` to pull from a path that uses the machine's unix user. But The write in the background backup for commit cloud used `util.shortuser(ui.username())`. This means that in On Demand, it would write to `$LOGDIR/samfu/$REPO$DATE` but `hg rage` would look in `$LOGDIR/svcscm/$REPO$DATE`. Changing this to the same `util.getuser()` should fix this.
Caveat: I didn't actually test this. I just know that my `hg rage` says "log directory does not exist: /tmp/svcscm" (P122851697)
Reviewed By: markbt
Differential Revision: D18654245
fbshipit-source-id: ccb78eba778ddedb23c7947a3eeb16e9f8fd2021
Summary:
This diff replaces eden's dependencies on failure::Error with anyhow::Error.
Failure's error type requires all errors to have an implementation of failure's own failure::Fail trait in order for cause chains and backtraces to work. The necessary methods for this functionality have made their way into the standard library error trait, so modern error libraries build directly on std::error::Error rather than something like failure::Fail. Once we are no longer tied to failure 0.1's Fail trait, different parts of the codebase will be free to use any std::error::Error-based libraries they like while still working nicely together.
Reviewed By: xavierd
Differential Revision: D18576093
fbshipit-source-id: e2d862b659450f2969520d9b74877913fabb2e5d
Summary:
[svcscm@hg022.prn2 /data/scm/www]$ hg log -r master
transaction abort!
rollback completed
abort: File name too long: '/data/scm/www-merge-generaldelta/.hg/store/data/flib/intern/____generated____/_graph_q_l_meerkat_step/flib/intern/entschema/generated/entity/bizapps/dcp__insight/application__config/_ent_d_c_p_insights_application_config_action.php/._g_q_l_g___intern_____set_permission_manager_domain_dcp_insights_application_config_data_____ent_d_c_p_insights_application_config_action____gen_perform_graph_q_l_set_permission_manager_domain_dcp_insights_application_config_mutation_coercer.php.i-U0sMqX~'
On hg022.prn2 www currently failes under certain circumstances because filenames longer than
255 characters are currently created. I traced this back to mktempcopy and made sure that
we truncate filenames when they are created too long.
Reviewed By: quark-zju
Differential Revision: D18639433
fbshipit-source-id: 20e4b086ca5c775b6054b642ecedfb05efd00fb9
Summary: Migrate callsites to use the new pattern matching filtering APIs.
Differential Revision: D18610738
fbshipit-source-id: 242182f4e6a58bbff5dddbb177287921c85519bc
Summary:
Similar to D2756217. But skip sync directly if the repo has been synced
recently.
This will help reducing CPU load on the DB tier.
Reviewed By: ikostia
Differential Revision: D18621013
fbshipit-source-id: 83370669ea61eb693d06495aaa13664648b97aca
Summary:
It seems an emtpy GraphQL query always times out. So let's avoid such queries
from the client side.
Reviewed By: xavierd
Differential Revision: D18626286
fbshipit-source-id: 70f0061f7f7ad8c7d8be4e15fc419039dc07d61e
Summary:
Many people hit TApplicationException recently. Most of them can be fixed
by `eden doctor`. Instead of crashing and showing an unhelpful backtrace, show
a shorter error message and prompt the user to run `eden doctor`.
Reviewed By: fanzeyi
Differential Revision: D18609445
fbshipit-source-id: 5492fa92008cb33e1122b72b48d0858db0d5f994
Summary:
The restackonce function warns about not using it and using `rebase.rebase`
directly, so let's just do that.
Reviewed By: quark-zju
Differential Revision: D18574767
fbshipit-source-id: 56829bccbba7f727867ba2b9bd0a81530ba29bf3
Summary: Eden sets the "portablefilenames" to ignore and the current Mercurial code doesn't consider the setting on Windows, which triggers iterate on the Eden clone. Iterate is an unsupported function on Eden clone.
Reviewed By: quark-zju
Differential Revision: D18195607
fbshipit-source-id: a21e1fcec3ee74398f22eaf56c70116468a45b45
Summary:
The caller already pass in a node, so there is no need to convert the rev
number to a node. This broke `hg next -B`.
Reviewed By: quark-zju
Differential Revision: D18539038
fbshipit-source-id: 44afdd184bc7c949f8efb863702af8a301bbcfb7
Summary: The commit timestamps in Git have second granularity. As a result, sometimes multiple succeeding commits have numerically equivalent timestamps. This means that sorting by timestamp may not result in the correct ordering of commits. This is particularly true because the sort we use is a stable sort, which means that commits with the same timestamp must go into the sort in ascending commit history order to have the correct order coming out of the sort. This commit updates the log that generates the list to use ascending commit order rather than descending.
Reviewed By: tchebb
Differential Revision: D18486839
fbshipit-source-id: 7a0ff099375a250d92b0f1a846e1105bf7bb9c86
Summary: The existing logic for parsing rev-list data relies very heavily on split, which creates many intermediate strings that take up a lot of memory. This commit updates the logic to use indices into the original output to extract individual strings. It also includes a new unit test for the list parsing. Big thanks to Tom Hebb for identifying the excessive memory usage.
Reviewed By: tchebb
Differential Revision: D18513949
fbshipit-source-id: c3b4e420ae2635904b42b84f2973e83c81c21dd7
Summary:
Revision numbers are deprecated, let's only show the short hash of the commit
instead.
Reviewed By: quark-zju
Differential Revision: D18519655
fbshipit-source-id: df277cc7e99ad747899d1fae2d92cd88eebea0f1
Summary:
This diff replaces code of the form:
```
use failure::Fail;
#[derive(Fail, Debug)]
pub enum ErrorKind {
#[fail(display = "something failed {} times", _0)]
Failed(usize),
}
```
with:
```
use thiserror::Error;
#[derive(Error, Debug)]
pub enum ErrorKind {
#[error("something failed {0} times")]
Failed(usize),
}
```
The former emits an implementation of failure 0.1's `Fail` trait while the latter emits an impl of `std::error::Error`. Failure provides a blanket impl of `Fail` for any type that implements `Error`, so these `Error` impls are strictly more general. Each of these error types will continue to have exactly the same `Fail` impl that it did before this change, but now also has the appropriate `std::error::Error` impl which sets us up for dropping our various dependencies on `Fail` throughout the codebase.
Reviewed By: Imxset21
Differential Revision: D18523700
fbshipit-source-id: 0e43b10d5dfa79820663212391ecbf4aeaac2d41
Summary:
The finddate function is used when using `hg update --date`, and since it
returned a revision number, the user would see the revision number deprecation
warning. Using nodes solves this.
Reviewed By: quark-zju
Differential Revision: D18486760
fbshipit-source-id: e1a51f624d8e7133fbf334f9b8b4b4c3d5aff2a0
Summary:
This diff is preparation for migrating off of failure::Fail / failure::Error for errors in favor of errors that implement std::error::Error. The Fallible terminology is unique to failure and in non-failure code we should be using Result<T>. To minimize the size of the eventual diff that removes failure, this codemod replaces all use of Fallible with Result by:
- In modules that do not use Result<T, E>, we import `failure::Fallible as Result`;
- In modules that use a mix of Result<T, E> and Fallible<T> (only 5) we define `type Result<T, E = failure::Error> = std::result::Result<T, E>` to allow both Result<T> and Result<T, E> to work simultaneously.
Reviewed By: Imxset21
Differential Revision: D18499758
fbshipit-source-id: 9f5a54c47f81fdeedbc6003cef42a1194eee55bf
Summary:
simpkins encountered a case where the treematcher does not work with many
patterns. It turns out `globset` has a hard-coded regex size limit (10MB).
Implement proper error handling so we can detect such issues and fallback to
slower paths.
Reviewed By: sfilipco
Differential Revision: D18500299
fbshipit-source-id: 0122ba9b0246c1536b2069a40e13261ee47f8bba
Summary: Add `hg debuglfsgc` command to reclaim disk space. Make it part of `hg gc`.
Reviewed By: xavierd
Differential Revision: D18417035
fbshipit-source-id: 7c0a445a0d4405df5cff960c0e28c4fc5a1f2c31
Summary:
Even keep was being passed in, rebase still added mutation markers to the old
commits, which would then make them hidden in the smartlog output. Not adding
the mutation markers in this case solves the issue.
Reviewed By: quark-zju
Differential Revision: D18496840
fbshipit-source-id: f5d0f8920bd00041598841a65356b88df00708d7
Summary:
In preparation for merging fb-mercurial sources to the Eden repository,
move everything from the top-level directory into an `eden/scm`
subdirectory.
