Summary:
The server is expected to provide head (of all segs), parents (of each seg),
roots (of all segs). We checked roots and parents but only check head in debug
build. Let's check head in release build too.
Reviewed By: andll
Differential Revision: D29405816
fbshipit-source-id: 1a97eb52a9a0d1d444ae5dabd1a01f0786be9fa9
Summary:
The external grep command (originally from fb-hgext's tweaks of grep command)
was run without redirecting its stdout. That could be problematic for
streampager, because writing directly to stdout will mess up with streampager
interface.
For example, the following command will show "nothing", while it should print
something:
fbcode/antlir/fbpkg/build % lhg grep -l scuba --config pager.pager=internal:streampager --config grep.usebiggrep=False --config pager.interface=full
Fix it by using a PIPE and forward the external command's output to streampager
buffer properly.
Reviewed By: andll
Differential Revision: D29408821
fbshipit-source-id: 4b2a0a6bbd64aa00d09f921d830b173cc56ae630
Summary:
Found by xavierd. Recent `os_info` bump now detects CentOS as OracleLinux.
Workaround it to keep our repo functional.
Reviewed By: xavierd
Differential Revision: D29410415
fbshipit-source-id: 1bd8183f46e3c2265aef119e9f96d9d05a5dbae6
Summary: Previously, hg would always try to fetch data via EdenAPI, even when doing so wouldn't make sense (for example, a local test repo). The only time it makes sense to try to use EdenAPI at all is when the repo in question is supported by Mononoke. As a quick heuristic, let's disable EdenAPI if `paths.default` is not set to a `mononoke://` URL.
Differential Revision: D29372739
fbshipit-source-id: b3601e529ed44c355cb192aedf4332317f4e3132
Summary: Add an option to allow manually forcing EdenAPI to be enabled or disabled. This is useful in a variety of cases, such as bypassing the normal EdenAPI activation logic in tests, or to forcibly disable EdenAPI in cases where it isn't working correctly.
Differential Revision: D29377923
fbshipit-source-id: f408efe2a46ef3f1bd2914669310c3445c7d4121
Summary:
Users get used to hints at the end of the output, not in a separate panel in
streampager. Make it so.
Reviewed By: andll
Differential Revision: D29400532
fbshipit-source-id: 41dc5d20f44a315bc0235ca3cb7857ae8955d3ad
Summary: This will be used by the next change.
Reviewed By: andll
Differential Revision: D29400533
fbshipit-source-id: e6b90bedd8d8a6cf9452dfb5c5f14f9980e12f62
Summary:
More straightforward way of D29404055 (e6ea02372c). Return the non-lazy set directly from
Rust. This avoids some overheads.
Note: ignoring whitespace will make reviewing easier.
Reviewed By: andll
Differential Revision: D29404901
fbshipit-source-id: 02e4766256863fe3fe258bcb318473355cd1efe4
Summary: This was used to narrow down issues.
Reviewed By: andll
Differential Revision: D29404054
fbshipit-source-id: 3bfdac332d63bdb13f40d5cf23dacec242b46d52
Summary:
Running `hasnode` in a loop is inefficient with lazy changelog.
Update it so it does batch filtering and does not ask remote service.
With the change, BackupState.heads will only contain locally known heads.
So the extra filtering is removed.
Reviewed By: andll
Differential Revision: D29404056
fbshipit-source-id: 327bd4c292ad75fd14a47587482440b6ece1d2d5
Summary:
Previously, the obsolete() set (from repo.dageval) is a "meta" (lazy) set that
might trigger frequent remote lookups. Calculate the obsolete() set into a
static set form so it does not trigger remote lookups.
This avoids expensive remote lookups in:
File "/data/users/quark/fbsource/fbcode/een/scm/edenscm/hgext/smartlog.py", line 432, in <lambda>
hidenodes = repo.dageval(lambda: obsnodes - heads(obsnodes) - roots(obsnodes))d
This issue was found out by setting env vars:
EDENSCM_LOG=dag::protocol=debug
to find the commit hashes, and:
EDENSCM_DISABLE_REMOTE_RESOLVE=a8209cbe13bcdc42d4999ffaa15dc9881660d5d,76d91d23d583f92b6c909b6747db1b75229d5be5
to find the traceback.
Reviewed By: andll
Differential Revision: D29404055
fbshipit-source-id: cc8bdd5347b38c1681565455874ed22f1a87321e
Summary:
With lazy changelog, for every commit hash that is unknown to the repo, it
needs to be resolved remotely. For commit hash that is also unknown to the
remote server, it can be bad because we don't have cross-process negative
caching and will trigger the remote resolution over and over. Practically,
the solution is to avoid remote lookup if the "minor" correctness issue
is acceptable (ex. D29111710 (ac6c6cf3fa), D29114049 (880b5c3cd8)).
That has been tricky to debug - the remote fetching happens in Rust, and
cannot be easily inserted something like `import ipdb; ipdb.set_trace()`
like Python. I have been inserting sleeps and use gdb to understand the
call stack when writing the above 2 diffs.
This diff makes debugging easier by supporting "break point" setting by
environ variables. So one can do something like:
$ EDENSCM_LOG=dag::protocol=debug hg ...
DEBUG dag::protocol: resolve names [435c235d65ccc4f95595d74478a617450c96c2e] remotely
$ EDENSCM_DISABLE_REMOTE_RESOLVE=435c235d65cc4f95595d74478a617450c96c2ec hg ... --traceback --debugger
Reviewed By: andll
Differential Revision: D29404057
fbshipit-source-id: d8a631f279f32e2ee88f097796cdc85d8ca27b77
Summary:
The logic was wrong. If local is True, vertexes is a filtered list, while nodes
isn't. They cannot be mix-used.
Reviewed By: andll
Differential Revision: D29403466
fbshipit-source-id: 06f0853282b0f284d5b2970bfa37589ef59f6169
Summary:
We don't need the full precision of execution time in the telemetry, thus
reduce the precision before logging it.
Reviewed By: kulshrax
Differential Revision: D29384354
fbshipit-source-id: cab02ab4db6b260525e56e0395ca1071fdb4b442
Summary:
Straightforward conversion of native checkout implementation from old scmstore API to new (non-async, batched) scmstore API.
We'll meet again someday soon, async.
Reviewed By: andll
Differential Revision: D29321512
fbshipit-source-id: 1e3e0d92c95730a5c2df610061f6faf5b1eb9068
Summary: The returned value now includes roots. Rename the function to clarify.
Reviewed By: kulshrax
Differential Revision: D29383072
fbshipit-source-id: 02a255ce20d9797f482f6fe1c716f2d79a12d4e0
Summary: There is a regression in 1.7.0 (which we're on at the moment) so we might as well update.
Reviewed By: zertosh, farnz
Differential Revision: D29358047
fbshipit-source-id: 226393d79c165455d27f7a09b14b40c6a30d96d3
Summary:
Use segmented changelog fast path during master branch pull
This change is gated by a config
Reviewed By: quark-zju
Differential Revision: D29347804
fbshipit-source-id: f0cc7633e67fb04266ce15506375873d58ea77fc
Summary:
Make it possible for Python to detect the error and fallback to slow path
accordingly.
Reviewed By: andll
Differential Revision: D29369067
fbshipit-source-id: 1de6162532b145469c09070bdf843ec96d38e431
Summary:
An alternative to D29363808 (e396cab669). The benefit is that parents_and_head is used by
both the client and the server. So we don't need to duplicate D29363808 (e396cab669) in
Mononoke code.
Reviewed By: andll
Differential Revision: D29365079
fbshipit-source-id: bca60ba2b3df477929d8e72b2363e5a0f744b35d
Summary:
Pull fast path uses `reload` which drops pending changes.
To avoid misuse, raise an error if pending changes are present.
Reviewed By: andll
Differential Revision: D29363799
fbshipit-source-id: 8f520d2c5553432abc452bc7b2b59d7af80e0a99
Summary:
The import pull data logic used low-level locking, persisting APIs, it does not
write cached idmap to disk. So we need to manually insert the idmap remote
lookup result to the actual local idmap explicitly.
This addressed an issue that verify_missing fails in the pull fast path.
Reviewed By: andll
Differential Revision: D29363813
fbshipit-source-id: 2749855a6c8c924bd1b772173de066d400f73764
Summary:
For a NameDag, `IdConvert` on `self.map` cannot resolve names remotely, but
`IdConvert` on `self` can. Use the latter. This is similar to D27547584 (af3c3b3fd0) where
some `self.map` are updated to `self`.
This addressed an issue found in the pull fast path test. Note there is another
issues to solve.
Reviewed By: andll
Differential Revision: D29363810
fbshipit-source-id: 28ba583ed14bbc5d52af81d4128d965f24eef011
Summary: The test pulls when the client has a lazy graph, and the server has a few merges.
Reviewed By: andll
Differential Revision: D29363806
fbshipit-source-id: 09bc3c4c3d21924f500ca86e8d86f58a15159169
Summary:
`fmt::Debug` for a NameDag is too verbose. Separate part of it so we can debug
print segments for a given (group, level). This will be used by upcoming
changes.
Reviewed By: andll
Differential Revision: D29363805
fbshipit-source-id: e1c6713be10b8b64fc7a42178117e724e0d691d0
Summary:
The client TestDag might have outdated server Dag as the remote protocol,
because it is a static "snapshot". Ensure the remote Dag is updated when
using the pull API.
This is an issue solved by tracking down issues in tests added in upcoming
diffs.
Reviewed By: andll
Differential Revision: D29363807
fbshipit-source-id: a560b2e91999873338604907a6d83cc7d2ff5c58
Summary: It will be used by the next change.
Reviewed By: andll
Differential Revision: D29363802
fbshipit-source-id: 842735ac05ea5fea4ea0c3625a68d06d27bc37d5
Summary:
It is useful when drawdag itself triggers remote fetches.
This was used but is not used after some refactoring. I think it might be useful
in the future so kept it.
Reviewed By: andll
Differential Revision: D29363803
fbshipit-source-id: fa178ac9783d1dc1b73525eeb8cd3d766cf46a0f
Summary: The test will be used to verify upcoming changes.
Reviewed By: andll
Differential Revision: D29363809
fbshipit-source-id: d34d13123914cfabb5c82dee3873b6e0c4979ee2
Summary: Make it easier to write more tests around pull.
Reviewed By: andll
Differential Revision: D29363804
fbshipit-source-id: 5b2cf8675343898fabc1d8845228e240e463edf8
Summary:
The roots data will be useful for the client to check if the pulled commits are
going to overlap with its existed DAG.
Reviewed By: andll
Differential Revision: D29363808
fbshipit-source-id: e09d924d65537f59fd4ea209b568265d07a80e46
Summary: Minor change to make the code a little bit more straightforward.
Reviewed By: andll
Differential Revision: D29363801
fbshipit-source-id: 2c4bd6ece07282f044622227a3c077cb31db6d17
Summary: Make the docstring a bit more consistent.
Reviewed By: andll
Differential Revision: D29363798
fbshipit-source-id: 1b4e2a7a1af4c4cffe3693e437a831bab1b43fd7
Summary:
Sometimes we need repo internal states for deeper investigation. Right now
the process is quite adhoc - telling people to figure out where the "backing"
repo is, excluding unwanted files like 00changelog.d, etc. Let's just make
it easier so we have a dedicated command to tar up selected internal states.
Reviewed By: andll
Differential Revision: D29242716
fbshipit-source-id: 85604d0df5b3a12f1aed29e4f31fcd5c166f9066
Summary: Verify we actually have pending keys to fetch before attempting a remote request in scmstore TreeStore.
Reviewed By: kulshrax
Differential Revision: D29345214
fbshipit-source-id: 328bdcbc41429e59de6ceb488533bafa97518fcc
Summary: Previously, it was not possible to interrupt `hg` during EdenAPI fetch operations. This made it impossible to interrupt long-running fetches, which is very frustating to users. This can be simply fixed by using `block_unless_interrupted` in place of `block_on`.
Reviewed By: quark-zju
Differential Revision: D29344670
fbshipit-source-id: 3b0d36dda28f5f7cc812a07981f295f8d0fbdd8a
Summary:
This is simple command mostly to be used by testing before we fully integrate with hg pull
This command does not perform discovery and requires from/to revision to be passed in cmd line
Reviewed By: quark-zju
Differential Revision: D29315647
fbshipit-source-id: 26d67031e566b7c99af1e2a5ab287f02b52f7db0
Summary: This is breaking the Windows release, reverting.
Reviewed By: fanzeyi
Differential Revision: D29339787
fbshipit-source-id: 22d8ff5db5619194e4597754dc37343cf0bc3286
Summary: `|&` apparently fails on mactest, so I've replaced it with `2>/dev/stdout | ` which works on my devserver and macbook.
Reviewed By: andll
Differential Revision: D29337621
fbshipit-source-id: eaac2592f4c7bfda6696c2500f3b08441b596c39
Summary:
Introduce basic contentstore fallback tracking to help monitor the scmstore shim rollout.
This will be expanded to a general fetch metrics system for scmstore in a future change.
Reviewed By: kulshrax
Differential Revision: D29305839
fbshipit-source-id: c6cc3ea15a3bb7b90f4ec298febc911ec4e2af91
