Summary:
I initially saw the incremental build as something that would be run in places
that had IdMap and IdDag stored side by side in process. I am reconsidering
to use incremental build in the tailing process to keeps Segmented Changelog
artifacts up to date.
Since we update the IdMap before we update the IdDag, it is likely that we
will have runs that only update the IdMap and fail to update IdDags. This diff
adds a mechanism for the IdDag to catch up.
Reviewed By: krallin
Differential Revision: D24516440
fbshipit-source-id: 3a99248451d806ae20a0ba96199a34a8a35edaa4
Summary:
mitrandir77 pointed me to this field which is exactly what I want.
Leave alias handling in place for configerator, which has a different name.
Reviewed By: mitrandir77
Differential Revision: D24628323
fbshipit-source-id: ccd01e6314feefd8ab75e8052a0a8a98b3119691
Summary: This diff removes reading the token from the "cert" property in the `~/.arcrc` and forces the caller to use either an OAuth token or CATs.
Reviewed By: quark-zju
Differential Revision: D24242614
fbshipit-source-id: 18538270102b7aa28731e82c8dd21f5da9e2f2d6
Summary:
We have automation that wants to use this to hold the lock while doing
some maintenance. They want the ability to wait for the lock so they don't have
to busy loop.
Reviewed By: snarkmaster
Differential Revision: D24604466
fbshipit-source-id: be02539908655e183f334865718b68b633b069a5
Summary:
I am wondering whether we should customize the serialization format for the
InProcessStore. I want to have a basis for the comparison before I proceed.
Reviewed By: quark-zju
Differential Revision: D24580273
fbshipit-source-id: d3ddfdc029dbdd84f60acace06fddc80b4d005f4
Summary: We had some aliases for fbsource, and `fbcode` is still in use. Teach phrevset to recognise all aliases via config for ease of patching.
Reviewed By: markbt
Differential Revision: D24589906
fbshipit-source-id: bd61e86135d63ae07fa62d741e16cea4882f691b
Summary:
D24293498 (ea03d62698) is causing 'hg log scm/metagit' to fail with weird revision
number lookup errors. Let's back it out for now.
Reviewed By: singhsrb
Differential Revision: D24549796
fbshipit-source-id: b10c4caf8fdadd2955ae1f829381c66d788042e6
Summary: This is essentially a backout of D24365328 (9f664a8b30).
Reviewed By: DurhamG
Differential Revision: D24544495
fbshipit-source-id: 08f598df0892a8479fac563096f9782038e18dfe
Summary: This change adds wire types for the history API in the most straightforward way possible. In a future change, I'll move the `WireHistoryEntry` / `HistoryResponseChunk` conversion logic into the `ToApi` implementation. This implementation also doesn't add per-item errors, or standardize `history` to match the protocol evolution standards used by `trees`.
Reviewed By: kulshrax
Differential Revision: D24342046
fbshipit-source-id: d46403a823f2a1e89ad9d6d2074241d8bfe4810e
Summary: This change introduces custo `Serialize` and `Deserialize` implementations for wire types containing a fixed-length byte array. This change is more verbose than should be necessary. Because `TryFrom` is implemented to convert `&[T]` to `[T; N]` and `AsRef` to convert `[T; N]` to `&[T]`, we should be able to use a generic serde `with = ` attribute on the byte array fields of the appropriate structs. Unfortunately, I'm getting a lifetime issue that I haven't been able to resolve, and const generics aren't available on stable to implement the trait bounds without an explicit lifetime.
Reviewed By: kulshrax
Differential Revision: D24460527
fbshipit-source-id: d3a4179b833a523ce164d3df934e4cbdc2202546
Summary:
Removes top-level metadata from `TreeEntry` and add a new, specialized type for carrying child entry metadata, `TreeChildEntry`. This change also fully removes the `revisionstore_types::Metadata` from EdenAPI trees, which is only used on files.
In a follow up change I'll optimize handlings of paths / path segments in `TreeChildEntry` keys. Right now they need to be joined manually.
Reviewed By: kulshrax
Differential Revision: D24434716
fbshipit-source-id: d0739471b1f6cef58b435e10b5fb774bfb08f7f6
Summary: Rather than silently dropping entries which cannot be fetched, this change has the `WireTreeEntry` type carry optional error information, allowing it to be (de)serialized to / from `Result<TreeEntry, EdenApiServerError>` instead of a bare `TreeEntry`. Currently, handling of these failures is up to the individual application code, but it might be useful to introduce utility functions to drop failed entries and log errors.
Reviewed By: kulshrax
Differential Revision: D24315399
fbshipit-source-id: 94e4593b77cf2dc12d0dcc93d174c8a4eda95344
Summary:
`message` is not longer an attribute of an exception object in Python 3. Avoid
using it. This avoids turning rendering `{mirrornode}` template into a crash like:
Traceback (most recent call last):
File "edenscm/hgext/fbscmquery.py", line 78, in mirrornode
return client.getmirroredrev(reponame, "hg", torepo, totype, node)
File "edenscm/hgext/extlib/phabricator/graphql.py", line 399, in getmirroredrev
self._raise_errors(ret)
File "edenscm/hgext/extlib/phabricator/graphql.py", line 504, in _raise_errors
raise ClientError(None, errormsg)
edenscm.hgext.extlib.phabricator.graphql.ClientError: The provided crypto auth token(s) are expired. Check to make sure you have specified a long enough expiration time for your crypto auth token(s).
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "edenscm/mercurial/commands/__init__.py", line 4225, in log
displayer.show(
....
File "edenscm/mercurial/templatefilters.py", line 422, in byteify
return b"".join([byteify(t) for t in thing if t is not None])
....
File "edenscm/mercurial/templater.py", line 391, in evalstring
return stringify(evalrawexp(context, mapping, arg))
File "edenscm/mercurial/templater.py", line 350, in evalrawexp
return func(context, mapping, data)
File "edenscm/hgext/fbscmquery.py", line 83, in mirrornode
mapping["repo"].ui.warn(_x(str(e.message) + "\n"))
AttributeError: 'ClientError' object has no attribute 'message'
Reported by: arnabde03
Reviewed By: mofarrell
Differential Revision: D24528362
fbshipit-source-id: 57d093811986f3a65d1201de1f9aed3770db2ad1
Summary:
Users get confused and try to pull from the wrong repo - e.g. `hg checkout D23665606` in `fbsource` - and find the resulting output about missing hashes scary.
Give them a nice clear error instead.
Reviewed By: wez
Differential Revision: D24485097
fbshipit-source-id: e9aeb7ed6aaf494f828d736ca5203100807fb321
Summary:
indexedlogdatastore is supposed to use remotefilelog.cachelimit to set
the max size, but instead it was setting the max per-log size, which means the
max size was N times bigger. Let's fix that.
Reviewed By: xavierd
Differential Revision: D24483181
fbshipit-source-id: f33cedbfdbb318e9d5eb9fda497645050b93e9fe
Summary:
In the next diff in this stack, I'm changing the Rust thrift library as well as
the codegen, and this is causing quite a bit of pain with regard to this
codegen that is checked in to the hg build:
- Regenerating the codegen isn't super obvious.
- The Sandcastle build fails because it uses the current codegen with the old
library.
According to lukaspiatkowski, this has also been a problem in the past when Eden got
migrated to Tokio 0.2, so I'd like to save myself and others some pain by just
not generating the server side codegen and not checking it in, since it's
unused. This reduces the surface of stuff that might go out of sync.
#forcetdhashing
Reviewed By: markbt
Differential Revision: D23649604
fbshipit-source-id: d684bec427431a366de42c88e53072caa98d5b2f
Summary:
When the new rust store is enabled, it can throw exceptions if the rust
code tries to fetch multiple trees when the server doesn't support bfs fetching.
The python code path ignores these requests, letting the eventual single tree
lookup path (i.e. not the prefetch path) perform the download. Let's let the
rust code do the same for rust stores.
Eventually every repo will support bfs fetching and we can delete this code path
entirely.
Reviewed By: quark-zju
Differential Revision: D24459106
fbshipit-source-id: d9a931364bd3235113b0dbcf09cba35a16e16e8e
Summary:
Use the new API `smartset.prefetch` for the prefetching logic for `log`
commands.
Prefetching is only enabled if the template contains related symbols.
Reviewed By: DurhamG
Differential Revision: D24289052
fbshipit-source-id: 810da3b249d45a7516aaf39a0b74f6965b87f96f
Summary: This makes `log` command more efficient using the `hybrid` backend.
Reviewed By: DurhamG
Differential Revision: D24293498
fbshipit-source-id: db3bac28a0b04ba6af449357bc08b285cd62780d
Summary:
Add prefetch APIs on smartset to make it easier to specify what need to be
prefetched.
This is intended to be used by `{phabstatus}`, log with lazy commit text backend,
and revsets like `author(x)`, etc.
Reviewed By: sfilipco
Differential Revision: D24321292
fbshipit-source-id: 7e645ccbe39feeae1e2da7bfb115623945bf9cef
Summary:
This makes it possible to "fork" sets and set different prefetch properties on
them.
Reviewed By: sfilipco
Differential Revision: D24365329
fbshipit-source-id: c4bdd6ab21e6a891a02d450f9e6edab3a3781eca
Summary:
The generatorset has a pure Python implementation for rewindable generator.
However it is not thread-safe, and we want thread-safety since the set
iteration will be driven by async Rust in multiple threads.
One of the issues is, during thread 1 `next(gen)`, thread 2 might call
`next(gen)`, and that's not allowed by Python.
Fix them by switching to the Rust RGenerator.
Reviewed By: DurhamG
Differential Revision: D24365328
fbshipit-source-id: 2785e80c7c460a7f754ed23e3af99f4a5c9fbcdf
Summary:
Similarly to the indexedlogdatastore, the LFS stores can become corrupted on
power loss. This is happening fairly frequently on Windows Sandcastle due to
the OS being virtualized and power being cut abruptly.
For now this only attempts to repair the shared stores, in theory we could also
try to repair the local stores but haven't looked into it.
Reviewed By: DurhamG
Differential Revision: D24449202
fbshipit-source-id: 605a7943a0850b625bf00c514879b3da1ab2b406
Summary:
The ContentStore/Metadatastore are made of several different stores, attempting
to expose all of them to Python to drive the repair logic from there would leak
implementation detail of how the stores are implemented.
Instead, let's simply expose a single `repair` function out of the
pyrevisionstore crate that takes care of repairing all of the underlying
stores. For now, this is just moving code around, but a future diff will
integrate the LFS stores.
Reviewed By: DurhamG
Differential Revision: D24449203
fbshipit-source-id: 1631ced9068716453cb404bf7e65cefbf2db5247
Summary:
This will be used to avoid 1-by-1 fetching for the changelog backend with
commit text stored remotely.
Reviewed By: sfilipco
Differential Revision: D24321293
fbshipit-source-id: 9695c72166cadc0b167e2ce7fde822cdf6b1cea8
Summary: This will be used to test if a keyword is in an AST.
Reviewed By: DurhamG
Differential Revision: D24293499
fbshipit-source-id: c4c238e09fae6728d9d20ffe94e04194ac20bbb8
Summary:
There were 2 recent reports about "visibleheads" not being enabled.
Previously, to enable visibility, one has to run a `pull` command with
`visibility.automirate=start` and `visibility.enabled=true`, and
`pull.automirate=true`, and our current config does not have
`visibility.automigrate`.
There are other issues with pull-based auto migration. For example, the edenfs
backing repo does not run pull and won't be migrated.
Move the visibility and dirstate automigrate logic to `repo.__init__` so they
always run. Also simplify the configuration so a single config
`visibility.enabled=true` (or `format.dirstate`) is enough to trigger the
migration.
Reviewed By: DurhamG
Differential Revision: D24340586
fbshipit-source-id: 2b8ed8da51c2345c9e6e680519dbc4f10eab1ac0
Summary:
We expect `.hg/store` to exist (ex. `storerequirements`). Drop support for repo
layout without a store.
Reviewed By: DurhamG
Differential Revision: D24340585
fbshipit-source-id: 757dc8633db9549a6bea082281326ebdefcbfcba
Summary:
Treestate is the only supported dirstate format. Remove test cases about legacy
dirstates.
Reviewed By: DurhamG
Differential Revision: D24340584
fbshipit-source-id: 2ebaa7d5aa9e8639e3ce4caa67c4f93155b52558
Summary:
Turn on rust changelog (changelog2) for all hosts (except hgsql).
Turn on doublewrite backend for hg-dev hosts, triggered by pull.
Tests are mostly working, and I have been using it for weeks.
Reviewed By: singhsrb
Differential Revision: D24259759
fbshipit-source-id: b89a27f98a6d3d1e4ea187bf7b29f875d0e96e2e
Summary: It will be used to test whether `{phabstatus}` is in template.
Reviewed By: sfilipco
Differential Revision: D24293496
fbshipit-source-id: 1b59236e07280c0cefb07be9da7a3fd18d9940a1
Summary:
There are 2 issues:
- len(repo) is no longer the "max rev" + 1. Use 1<<63 instead.
- If invalidatelinkrev is a repo requirement, linkrev can no longer be trusted.
Reviewed By: DurhamG
Differential Revision: D24270423
fbshipit-source-id: 4e5c999772fad6ef2b1dabd2d6363d4226242c4d
Summary:
If linkrevs are known unreliable, do not use them. This fixes some cases with
segmented changelog, file revisions are incorrectly filtered out and not sent
to client.
Reviewed By: DurhamG
Differential Revision: D24233228
fbshipit-source-id: 268a95b44c0217658b849317cb0a680450b91d23
Summary: This makes the output less verbose.
Reviewed By: DurhamG
Differential Revision: D24411259
fbshipit-source-id: 50b175f2e31ffda62e78dd226775edaf53893bce
Summary:
With segmented changelog, the maximum revision is not always the "tip".
Therefore, track "tip" explicitly in svfs / metalog. This improves
compatibility.
Reviewed By: DurhamG
Differential Revision: D24214711
fbshipit-source-id: 7b22de7ac684a90965a695924bdb9114758aae43
Summary: Avoid HashSet or HashMap order to preserve the order of inserting commits.
Reviewed By: DurhamG
Differential Revision: D24214460
fbshipit-source-id: 66df2e0aba1820e6585f8da66897078f38abf82f
Summary:
Previously bundlerepo only supports revlog changelog. Detect repo configuration
and support non-revlog changelog. This is done by adding commits from bundle in
memory, and updating visibleheads.
Reviewed By: DurhamG
Differential Revision: D24214461
fbshipit-source-id: 63f9af49a0fe0ed2581b611f3cf7002d1b89d2fd
Summary:
The nullid and wdirid are special hg hashes that do not respect SHA1. They were
handled at the bindings layer. However the bindings layer cannot handle them
in a stream. Therefore move it in hgcommits.
Reviewed By: DurhamG
Differential Revision: D24365330
fbshipit-source-id: e8dc6205351ec1a2304252b9ec446dda010e6295
Summary:
Pure Python generator has a few limitations:
- Can only be iterated once.
- Cannot be iterated by multiple threads concurrently.
By converting revset iterators to Rust Stream, they run in different threads
and can context switch in `next()`, breaking the program.
Provide a native reentrant generator implementation to address the issue.
This is sound because Rust methods cannot be interrupted by Python.
Reviewed By: DurhamG
Differential Revision: D24365331
fbshipit-source-id: 885dade922b7863a73203b206a96b492d55bccd0
Summary:
Update the return type of streamcommitrawtext from
`{"vertex": v, "raw_text": t}` to `(v, t)`. This makes it easier to use
in Python, as Python supports `for v, t in ...` but does not support
`for {"vertex": v, "raw_text": t} in ...`.
Reviewed By: DurhamG
Differential Revision: D24295457
fbshipit-source-id: 284a29b9deae2d8509d3afea0fcbcaadbfebbae8
Summary:
In case the server does not respect the input contract and missed
some items without returning errors. The current logic would retry
forever. Change it to detect the issue and raise an error.
Reviewed By: DurhamG
Differential Revision: D24293497
fbshipit-source-id: 09421c7743078a488a9c81ce66fd92c12b39543c
Summary: The store stores sorted(p1,p2)+text to match SHA1 hashes. It's not just `text`.
Reviewed By: DurhamG
Differential Revision: D24325554
fbshipit-source-id: 8a91970f60fb535ca1a5a2d30c7d27f2714f28de
Summary: It's no longer useful as the new abstract interface does not need it.
Reviewed By: sfilipco
Differential Revision: D24399516
fbshipit-source-id: 2b6735d2a26706c6a3e6b592d2f3ecfc874c94cb
Summary:
This verifies the abstraction and simplifies the code.
The new code will use non-master segments for add_heads. Therefore the test
changes.
Reviewed By: sfilipco
Differential Revision: D24399496
fbshipit-source-id: 39067ad88ade79b4f7758bcdaafc03e5f34ced91
Summary: This makes the main namedag.rs cleaner. The next step is to move MemNameDag.
Reviewed By: sfilipco
Differential Revision: D24399495
fbshipit-source-id: c1e79a60edd8597fe7264f04548e5312414241a7
Summary: This is the last non-abstract interface of NameDag.
Reviewed By: sfilipco
Differential Revision: D24399514
fbshipit-source-id: f39bb84a1851a4fe4d1f29e6b0961e6a153c943d
Summary:
There is a need to open AbstractNameDag cleanly from a path.
Abstract that.
Reviewed By: sfilipco
Differential Revision: D24399498
fbshipit-source-id: ca242cd929e8f5580120c01eeaa928f630c21ed7
Summary:
I copied the code since it's hard to implement using the macros.
In the future I plan to merge MemNameDag into AbstractNameDag
and remove the macros.
Reviewed By: sfilipco
Differential Revision: D24399517
fbshipit-source-id: 326e76cd06a6e1ad26b39bcb51ba0ff24106c984
Summary: The `delegate!` is updated to support complex `impl`s.
Reviewed By: sfilipco
Differential Revision: D24399518
fbshipit-source-id: b9ba31174472cce4248e9644611cfc207abc3c1d
Summary: Will be used as bounds for abstraction.
Reviewed By: sfilipco
Differential Revision: D24399497
fbshipit-source-id: 343be12237d4850fbde9ebbe4034469527bd77fc
Summary: The `snapshot` field can be used instead.
Reviewed By: sfilipco
Differential Revision: D24399507
fbshipit-source-id: 67de20d897b8b763f724f3ccbd46618dec7911b9
Summary:
The trait requires an `IdMap` snapshot to be locally ready. That's not easy for
all possible implementations. Drop it to simplify things.
Reviewed By: sfilipco
Differential Revision: D24399501
fbshipit-source-id: 4d85f77c99208cda30b2a543a0bb5b295f49a65c
Summary: There were 2 prepare_filesystem_sync. Unify them into one implementation.
Reviewed By: sfilipco
Differential Revision: D24399513
fbshipit-source-id: 80d009c33b7f23dc2c4225da6fd0fb09589ba061
Summary: More general purposed type for Syncable{IdDag,IdMap}.
Reviewed By: sfilipco
Differential Revision: D24399502
fbshipit-source-id: 0599db6dd07fe3d430458f86a33a9144d850fca1
Summary: This makes it more generic.
Reviewed By: sfilipco
Differential Revision: D24399493
fbshipit-source-id: 8a1d0a13dd29989b17fe3ef1497b10b6fa0629d6
Summary: Similar to IdDag change, move impls to separate files.
Reviewed By: sfilipco
Differential Revision: D24399508
fbshipit-source-id: 575b6e7194677b67b6755b0a30ae7d014d498b10
Summary:
The lock, reload, mutate, persist pattern is general. It can be used for IdMap
too.
Reviewed By: sfilipco
Differential Revision: D24399512
fbshipit-source-id: d25e51ba735061ca101101d75aff95deb88b1d36
Summary:
Now `build_segments_persistent` and `build_segments_volatile` are the same.
Just keep one of them.
Reviewed By: sfilipco
Differential Revision: D24399511
fbshipit-source-id: a9f1ac920cdf5b448bd99bf9b6d4ca4160ba0304
Summary:
Previously, we keep the last high level segment per level in memory, and
drop it on disk. When we cross the memory / disk boundary, we had to
maintain such properties carefully. That was needed because some DAG
algorithms rely on complete high level segments.
Now that no DAG algorithms depend on such properties, let's just drop
the logic adding the last segment back to simplify the code.
This removes the need of building segments after open() and sync().
Reviewed By: sfilipco
Differential Revision: D24399515
fbshipit-source-id: 4c640d9aa03c050fcd97f70ee386e32d3a8ee26d
Summary:
This makes the algorithm a bit more robust. Now none of the DAG algorithms
depend on high-level segments are complete and cover all low-level segments.
This also removes constraints. For example, SyncableIdDag can now just
deref() to the normal IdDag for queries without worrying about correctness.
Reviewed By: sfilipco
Differential Revision: D24399503
fbshipit-source-id: e6a91010cff82264cf423e2f24dee1d372822ef6
Summary:
They depend on high-level segments covering low-level segments, which
adds extra complexities. Remove them to simplify logic.
Reviewed By: sfilipco
Differential Revision: D24399509
fbshipit-source-id: 56a8e06c263107d1da4d6754b884ce51e18e30bf
Summary:
Even if ui.threaded is off, we're seeing sys call interrupt issues. I
think this is because ui.threaded defaults to true, and we check it's value
before loading the repo config (which includes the dynamicconfig to turn it
off). It's already off in dynamicconfig, so let's turn it off by default.
This highlights an issue with dynamicconfig, in that it can't deploy pre-repo
configs at the moment.
As for the actual interrupted syscall, it looks to be caused by SIGCHLD signals
from child processes we spawn (possibly the dynamicconfig child process).
Reviewed By: quark-zju
Differential Revision: D24401521
fbshipit-source-id: 0372ffc7297fa6fe6a7e4f0ae69299b252c3632e
Summary:
Move logic to select a changelog backend to a separate method so it can be used
by bundlerepo.
Reviewed By: DurhamG
Differential Revision: D24214463
fbshipit-source-id: 6667317e9d51076b4a5d4a07779792f27cbd6d5e
Summary:
This is for cases like:
repo1 = hg.repository(path)
repo2 = hg.repository(path) # same path
with repo1.lock():
repo2.lock() # deadlock
Previously it simply deadlocks with confusing message:
waiting for lock held by process X # X is the current process
With this change it will raise a ProgrammingError about deadlock.
Reviewed By: DurhamG
Differential Revision: D24214458
fbshipit-source-id: 57faddaed6d2f1d36fab067c2e1f63b3cda0a1fe
Summary:
Replace the legacy `displaygraph` with `rustdisplaygraph`. The `edegfn`
parameter is no longer used and is dropped.
Reviewed By: DurhamG
Differential Revision: D24299592
fbshipit-source-id: 8336da1c8831b3352a950544d72cbdcf0203858d
Summary: Only keep the new Rust renderer.
Reviewed By: DurhamG
Differential Revision: D24299593
fbshipit-source-id: a0705ce477a5e1166a71fbadac051ae093b4598d
Summary:
`onlyboth` returns a lazy set for revlog backend.
`torevs` does not work well with such lazy set.
Use `torevset` instead.
Reviewed By: DurhamG
Differential Revision: D24397872
fbshipit-source-id: e7f5c6412b4120305b67103ceff1b1922a67ec7d
Summary: This preserves the `--noninteractive` flag used by some tools.
Reviewed By: DurhamG
Differential Revision: D24040789
fbshipit-source-id: 8d50f3f3ce6b2015f0ef6c3bd1b4fbb874d0ea7d
Summary:
This restores the compatibility of setting up merge tools using the `ui.merge`
config while still limiting the default `editmerge` tool to interactive
sessions.
Reviewed By: sfilipco
Differential Revision: D24377259
fbshipit-source-id: 3d2befba412b824fc985ddffa131e339644178c2
Summary: Make it testable by allowing specifying paths to load as user hgrc.
Reviewed By: sfilipco
Differential Revision: D24377258
fbshipit-source-id: 969028df64d55ad1f1304e35675d84595ed6a2bf
Summary:
Include a `User-Agent` header in EdenAPI requests from Mercurial. This will allow us to see the version in Scuba, and in the future, will allow us to distinguish between requests send by Mercurial and those sent directly by EdenFS.
Keeping with the current output of `hg version`, the application is specified as "EdenSCM" rather than "Mercurial".
Reviewed By: singhsrb
Differential Revision: D24347021
fbshipit-source-id: e323cfc945c9d95d8b2a0490e22c2b2505a620dc
Summary: This was competely undocumented, and thus undiscoverable. Add some documentation, so that it's at least clear the feature exists
Reviewed By: singhsrb
Differential Revision: D24332961
fbshipit-source-id: 9e73163a9314ceb7f953a3b1ac0f58c9a6e6d4d9
Summary:
The hide-before config was added to encourage people to actively hide unused
drafts for repo performance, instead of keeping unused draft forever, since a
lot of code paths assume `len(draft)` is small. See D13993584 (28b4dfbb38) for more context.
Now our hide-before data is set to 2.5 years ago (2018-2-25), this change
probably only affects a very small number of users.
Reviewed By: DurhamG
Differential Revision: D24298198
fbshipit-source-id: 938aca1222b55e09fdb058ff01bc063733f201dc
Summary:
Rust tests run in multiple threads. Setting environment variables affects other
tests running in other threads and causes random test failures.
Protect env vars using a lock.
Reviewed By: DurhamG
Differential Revision: D24296639
fbshipit-source-id: db0bee85625a7b63e07b95ea76d96029487881d4
Summary:
The shell-script cargo tests seem very flaky. Use a dedicated Python script to
run the tests, with a more concise output that only includes failures, and run
tests in parallel.
Reviewed By: DurhamG
Differential Revision: D24296433
fbshipit-source-id: 1d63146c6c84f1035dded24fcd3d79f116c2e740
Summary:
Ideally we'd just delete the p4 convert functionality, but I'm too lazy
to go through and extract it right now.
This was recently enabled when I enabled all the convert tests. We don't use the p4 logic, so it's safe to just turn back off to get a release out.
Reviewed By: quark-zju
Differential Revision: D24352068
fbshipit-source-id: 6f3a1f88739b2e2348aff00e8cae333473bbe71a
Summary: My recent change accidentally returned early when reading the prompt input, which skipped the \r truncation needed for Windows.
Reviewed By: sfilipco
Differential Revision: D24350672
fbshipit-source-id: 4a589d76bf41cda7fda2518003ef272f9a6ead48
Summary:
We're seeing slow pack fetches in some cases. Let's add some extra
debug output to get more information.
Reviewed By: quark-zju
Differential Revision: D24295593
fbshipit-source-id: b5a5bdf169a8c05a3143da09d69646a7a742ef08
Summary:
We're seeing history fetching take quite a while during checkout and
rebase, but it's not really necessary for a checkout. In addition to it being
slow, if memcache doesn't have the history entry we'll fall back to a full
revision fetch from the server, which includes data. Let's disable prefetching
history during checkout.
Reviewed By: quark-zju
Differential Revision: D24295594
fbshipit-source-id: 70aa6e1925074b6546626a5192a7562d6da31f2b
Summary:
My recent diff changed these lines from `bytes(value)` to
`pycompat.decodeutf8(value)`, since we want these to be strings. Unfortunately,
on python 2 decodeutf8() just passes the value straight through, so in cases
where we're handed bytearray, we pass that through instead of converting it to
bytes. Some down stream consumers requires it to be bytes.
Let's conditionally turn it to bytes.
Reviewed By: krallin
Differential Revision: D24307818
fbshipit-source-id: a0cc64b7e2cf7645586e633e7a4a382b69390e15
Summary:
When the server returns a 429, the intention is that the client will wait for a
little bit then try again later (there is no harm in that, as we haven't really
used many server resources for this). However, it turned out that right now we
just abort. Let's fix it!
Note that this changes the behavior a bit for the error cases, in the sense
that we no longer return `Ok(None)` but instead return an `Err`. Xavier noted
this should make sense here.
I've also had the client send its retry attempt via a header, because who
knows, that might be useful.
Reviewed By: kulshrax
Differential Revision: D24308127
fbshipit-source-id: 35639956f36342dfb0056b0d348dc4ad56bd576c
Summary:
This is Ruby's `iter.each_slice(n)` or Rust's `slice.chunks(n)`.
It will be used by phabstatus to batch requests.
Reviewed By: singhsrb
Differential Revision: D24289050
fbshipit-source-id: 528b35d43874222f1b6783738e965d9a3b159deb
Summary: Make it easier to iterate contexes instead of revs.
Reviewed By: simpkins
Differential Revision: D24289053
fbshipit-source-id: deffdc4a5d4e7e3644a98f8112f5c8b8bdbd154d
Summary:
Previously `--master` falls back to `null` if `-r` is provided for
compatibility. That could be surprising as smartlog's sorting does not work
well without a master branch.
Change `--master` to a consistent value regardless of `-r` is provided or not.
Reviewed By: simpkins
Differential Revision: D24297586
fbshipit-source-id: c6e695333ae17db11dcb98d9cdb57cc2f298ff64
Summary: Introduces fetching of child entry IDs, and child file metadata for a specified tree manifest ID. The aux data lookup will only be performed if `with_file_metadata` is set, which is actually kind of wrong. Instead `with_children` from the wire type should be exposed in the API request type, and `with_*_metadata` should be hidden or used for data other than the child entry `Key`s.
Reviewed By: kulshrax
Differential Revision: D23886678
fbshipit-source-id: 0cba72cea7be47ae3348a406d407a19b60976c0c
Summary: Some of these tests were already passing. Some required minor fixes to pass.
Reviewed By: quark-zju
Differential Revision: D24290635
fbshipit-source-id: 789d3033142466c868057da8f9f23d27aede8ea4
Summary:
They now pass. Didn't test the p4 tests since I don't have p4
available, but I don't think we use it anymore.
Reviewed By: quark-zju
Differential Revision: D24284643
fbshipit-source-id: 184697052c0b7000ad9765b4033964b4d382b811
Summary:
Template filters are expecting the output of the template to be a unicode
string, not a byte string. Thus we need to ensure that diff return a unicode
string.
Reviewed By: DurhamG
Differential Revision: D24290908
fbshipit-source-id: 70899249c3e26edd1714fd901fa6af405b61e544
Summary:
This reverts D24066918 (a684e53a31). That diff changed the phabstatus code to use the last
created revset iterator rather than the first for peekahead purposes.
The first revset iterator does appear to be the correct one to wrap. After
D24066918 (a684e53a31) I consistently see `hg log -r 'draft()' -T'{phabstatus}\n'` make
Phabricator queries for only one diff at a time. Reverting the changes from
that diff fixes the issue.
Note that the revset argument passed to `hg log` appears to make a difference
in the behavior here. When using simple revsets including just specific
commit ranges then the peekahead behavior works as expected both before and
after D24066918 (a684e53a31). However, when using `draft()` in the revset specifier then
the behavior works before D24066918 (a684e53a31), but not after it.
Reviewed By: singhsrb
Differential Revision: D24269910
fbshipit-source-id: c2c2abf6360fe3129a1fc73b5b433b763bce1fd2
Summary:
The infinitepush sqlindex was returning bytes instead of strings, which
caused various formatters to serialize as "b'sadf'" instead of "sadf". Let's fix
the type converter to produce strings on python 3. Note, this is the opposite to
what we do in the hgsql converter. hgsql works with binary data, so we default
to bytes there. The sqlindex doesn't deal with any binary data, so defaulting to
strings is better.
Also updates a comment on the hgsql converter to be accurate. I confirmed there
are no other customer converters in the code base that need updating.
Reviewed By: krallin
Differential Revision: D24282899
fbshipit-source-id: 708732c17626f6ebc2c3cbe7f00afa5771f31d7b
Summary:
Include the client correlator string from the `clienttelemetry` extension in each EdenAPI HTTP request via the `X-Client-Correlator` header.
The `ClientIdentityMiddleware` in `gotham_ext` already understands this header (as it is already used by the LFS server), and `gotham_ext`'s `ScubaMiddleware` will automatically include the provided correlator in the server's Scuba samples.
Reviewed By: farnz
Differential Revision: D24282244
fbshipit-source-id: 13d04e706eda38893cff6e740bd1d7bf104e43dd
Summary:
The logic inside `eden/scm/tests/features.py` script enables certain
features of HG on tests based on the name of the test. Mononoke's integration
tests suit reuses eden/scm's tests suit and as a consequence it triggers the
same `features.py` logic. It was fine until D24201934 (f5988c415c) introduced a feature for
test that is named in the same way as one of the Mononoke tests - the
`test-clienttelemetry.t`.
In order to fix this problem Mononoke will pass a `--nofeatures` flag to the
`eden/scm/run-tests.py` invocations that will turn off the usage of
`features.py`.
Reviewed By: farnz
Differential Revision: D24276294
fbshipit-source-id: eb28ed55a05de3b012e37407603c2370adaaad16
Summary:
If Phabricator takes more than 10 seconds to respond, a `socket.timeout`
exception may be thrown. Treat this like other networking errors, and simply
report the diff phabricator status as `Error`. Previously this exception was
unhandled, causing the entire command to abort.
Reviewed By: singhsrb
Differential Revision: D24272274
fbshipit-source-id: f646d111a91f901e09d9f94a1e0102d6dd4d0952
Summary:
This change introduces two new metadata types, Category and Transience, and a mechanism for Category to provide a default Fault and Transience, which can be overriden by the user.
Also introduces a mechanism for attempting to log exceptions which occur during exception logging, falling back to the previous behavior of just swallowing the exception on failure.
Reviewed By: DurhamG
Differential Revision: D22677565
fbshipit-source-id: 1cf75ca1e2a65964a0ede1f072439378a46bd391
Summary:
It only has benchmark code that led to the use of mincode. Now hgcommits is the
main crate for commit storage. `commitstore` without `hg` in its name was
initially planned to support other kinds of commits including git and bonsai.
However we don't have immediate goal for that at present. So let's just remove
the commitstore directory.
Reviewed By: singhsrb
Differential Revision: D24263618
fbshipit-source-id: 84b4861ae490817377e69d8c2006c63331e3db1f
Summary:
On the hg servers we're seeing crashes due to signals during syscalls.
Rolling back to prior to the signal changes seems to have fixed it, though we
haven't bisected enough to be sure this is the cause.
The ui.threaded option is already used to opt-out of running on a background
thread, let's also use it to opt-out of the custom signal registering in hopes
that it fixes the server issue, until they can be deprecated.
Reviewed By: quark-zju
Differential Revision: D24254804
fbshipit-source-id: 50e4fc8c7e3a88b5baa49394f6f1edffc946187d
Summary: Need to add new quickcheck tests, verify that remove `Serialize` from `TreeEntry` is okay.
Reviewed By: kulshrax
Differential Revision: D23457777
fbshipit-source-id: aa94ed7aa81b41924eba4a8bd1bdc2c737365b77
Summary:
Change
abort: repository repo: timed out waiting for lock held by <lockinfo ...>
to:
abort: repository repo: timed out waiting for lock held by process '842210' on host 'hostname'
Reviewed By: singhsrb
Differential Revision: D24214462
fbshipit-source-id: 65056ebb9764651b2f0126061fafdfdefaa4e9c6
Summary: The rev numbers are almost gone, remove them from the test.
Reviewed By: sfilipco
Differential Revision: D24255156
fbshipit-source-id: 5cbc4a71c2d3f773c5b474d1edce84ceceb50bf9
Summary:
This will allow Eden to control what data is in Mercurial's memory and
what is on disk. This will let it persist the hg_import_helper process longer,
and prevent slow startup times when needing to restart it.
Reviewed By: xavierd
Differential Revision: D24231131
fbshipit-source-id: a4f743740b44957e8d2dd93f07e9831eadfee7ab
Summary:
This was added in D16078908 (1bc6ecf8fe). It turns out fsync followed by a hardlink
on btrfs results in very slow hardlink performance (100-600ms). Since fewer and
fewer of our files use this atomic write code path and since this affects almost
every hg write command, let's roll this back.
This will increase the chance of data loss during a hard reboot, but commit
cloud is good enough to allow recovering from that in almost every situation.
Reviewed By: sfilipco
Differential Revision: D24230056
fbshipit-source-id: aae1a5612eda5f62bb5ec1442b1312ed45c42786
Summary:
With invalidatelinkrev, linkrev stored in revlog shouldn't be used.
This makes some tests pass with segmented changelog.
Reviewed By: singhsrb
Differential Revision: D24201944
fbshipit-source-id: 6473c30266c79aa97a955b1a6c867411cc67de2b
Summary:
The old code assumes `0..len(repo)` are valid revs, which is no longer true
with segmented changelog.
Reviewed By: singhsrb
Differential Revision: D24201948
fbshipit-source-id: b882a215701c57ccdf4af340c889586b040772da
Summary:
Copy segmented changelog, metalog and mutation store for local clones.
This mainly affects tests.
Reviewed By: singhsrb
Differential Revision: D24201941
fbshipit-source-id: c60da9e2bf982a6f66004415e45178749157745e
Summary: This is needed to move our hg servers to python 3.
Reviewed By: quark-zju
Differential Revision: D24204056
fbshipit-source-id: cbaf97893f8f77b535952ac290766f0fd5e14f0c
Summary: Do not show revnum or "changeset" after commit (with --debug or --vrebose).
Reviewed By: singhsrb
Differential Revision: D24201942
fbshipit-source-id: 2f0d15711df67070e50d4bf30f0b1b4401d85524
Summary:
With segmented changelog, linkrev can exceed i32 range and cause "pack" (aka.
bundle) to fail. Workaround it by packing nullrev instead.
~30 tests now pass with segmented changelog with this change.
Reviewed By: singhsrb
Differential Revision: D24201940
fbshipit-source-id: 5f27c185837cd3c1fbe9b65d21ef3cd641eec2e5
Summary:
debugbuilddag assumes revs created start from 0, 1, 2..., which is
no longer true with segmented changelog.
Change it to resolve revs using the local `nodeids` array so it's
compatible with segmented changelog.
Reviewed By: singhsrb
Differential Revision: D24191790
fbshipit-source-id: ca7d1cccbba664128c227d66071b166c799cdf49
Summary:
A previous change updated all tests to remove the use of rev numbers. The
update to the list of results missed this tests case.
Reviewed By: quark-zju
Differential Revision: D24208236
fbshipit-source-id: 289136f31e66eb74b106d7ea9401419fc369d59f
Summary:
Print a message for each EdenAPI method call to stderr if the user has `edenapi.debug` set.
These messages are already logged to `tracing`, but also printing them out when `edenapi.debug` is set makes the debug output more useful, since it provides context for the download stats. This is especially useful when reading through EdenFS logs.
Reviewed By: quark-zju
Differential Revision: D24204381
fbshipit-source-id: 37b47eed8b89438cdf510443e917a5c8660eb43b
Summary: Use a `HashMap` to store user-specified additional HTTP headers. This allows headers to be set in multiple places (whereas previously, setting new headers would replace all previously set headers).
Reviewed By: quark-zju
Differential Revision: D24200833
fbshipit-source-id: 93147cf334a849c4d2fc4f29849018a4c7565143
Summary: The panics can happen when the input sets are out of range.
Reviewed By: kulshrax
Differential Revision: D24191789
fbshipit-source-id: efbcbd7f6f69bd262aa979afa4f44acf9681d11e
Summary: Change fold and metaedit to not show revnum in editor message.
Reviewed By: kulshrax
Differential Revision: D24191787
fbshipit-source-id: 140ec58c8eb00c067c6e40e1a18187f7801246e9
Summary:
Some sort of serialization for the Dag is useful for saving the IdDag produced
by offline jobs load that when a mononoke server starts.
Reviewed By: quark-zju
Differential Revision: D24096964
fbshipit-source-id: 5fac40f9c10a5815fbf5dc5e2d9855cd7ec88973
Summary:
We're seeing an issue on the hg servers where the filecache assertion
that if a value is in obj.__dict__ it's also in obj._filecache is broken. This
occurred about 10% of the time in sandcastle jobs. The diff that caused this
went in in April (D21148446 (73c5cb89de)), so it's unclear why it's only cropping up now.
This is caused by the following steps:
1. repo._bookmarks is accessed while _bookmarks is in the _filecache but not in
the __dict__
2. This causes construction of _bookmarks, before it can set it to __dict__
3. Construction of _bookmarks calls repo.invalidate(clearfilecache=True), which
deletes _bookmarks from _filecache.
4. _bookmarks construction completes, and gets set to __dict__ (but now it's
missing from _filecache, so the invariant will fail next time someone checks).
5. Someone accesses _bookmarks later, and the assertion fires.
The fix is to just not clear the filecache during bookmark construction. The
main purpose of this invalidate was to let the changelog be reloaded, and I
think that will still happen since, if there are any new commits in the
changelog, the file size and time will change, triggering a reload next time the
_filecache entry is checked.
Reviewed By: quark-zju
Differential Revision: D24182914
fbshipit-source-id: fb49137e28d9224c6617d9c84faaf2f9de363aaf
Summary:
Our stdin/stdout bytes/str manipulations caused input() to print
warnings about buffered not being supported. The only reason we do those
manipulations to handle the case in tests where the prompt answer doesn't come
from stdin, so let's just handle that case via readline instead of prompt.
This is what upstream Mercurial does.
Reviewed By: quark-zju
Differential Revision: D24122909
fbshipit-source-id: ab9d989a66d39990b688c65a1fae80bd48b0f42e
Summary: Add `--debug` flag to `read_res cat` command for debug printing entire entry rather than just the data blob.
Reviewed By: kulshrax
Differential Revision: D23999804
fbshipit-source-id: 6955854edab2643cffbe5fae484a398716b48055
Summary:
Initial support for a backend using edeanpi.
Note this is just the first step. Most code paths are not updated to the
streaming API to get commit data, so they will error out with commit not found
errors.
Confirm that commit data can be fetched via edenapi:
$ RUST_LOG=debug lhg dbsh --config experimental.lazy-commit-data=1
In [1]: master= repo['master'].node()
In [3]: cl.inner.getcommitrawtext(master) is None
Out[3]: True
In [4]: s=cl.inner.streamcommitrawtext(repo.nodes('master~10::master'))
In [5]: it=iter(s)
...
[2020-09-25T02:09:16.793Z DEBUG hgcommits::hybrid] >> resolve_remote input=[e6c4e203b66f1416e08dc597a2d63b91e93b1466, bfb610989e9dd701e785b4a3a5998e76d9709cab, 68bbfc79602a153895b761089e9479dd8fa33351, 5366fe39ad538463abae6c648eb5150bbb79d4c7, 5ea45d8ab0f8203837ca1736f36ded4a492571b4, 722da0a32eae12de5e85078beea2ae4b7aafe4a4, 4dbe3eab10d13b30697e1762eb7b9ff3ad0cf630, 430ae91aab8028b6572ccef89f8396dafec622c4, 5abd96c5420f0d512c63e768f8cea83f1c6691c9, c84ab3412cebfade730e95a1bc5ebc9b1dd0747b, 790ed2d40e4a0b08fb22fe9b4246fec0165f8a87]
[2020-09-25T02:09:16.793Z DEBUG hgcommits::hybrid] << resolve_remote input=[e6c4e203b66f1416e08dc597a2d63b91e93b1466, bfb610989e9dd701e785b4a3a5998e76d9709cab, 68bbfc79602a153895b761089e9479dd8fa33351, 5366fe39ad538463abae6c648eb5150bbb79d4c7, 5ea45d8ab0f8203837ca1736f36ded4a492571b4, 722da0a32eae12de5e85078beea2ae4b7aafe4a4, 4dbe3eab10d13b30697e1762eb7b9ff3ad0cf630, 430ae91aab8028b6572ccef89f8396dafec622c4, 5abd96c5420f0d512c63e768f8cea83f1c6691c9, c84ab3412cebfade730e95a1bc5ebc9b1dd0747b, 790ed2d40e4a0b08fb22fe9b4246fec0165f8a87]
...
[2020-09-25T02:09:16.958Z DEBUG zstore::zstore] >> Zstore::contains id=3422a85c3703dd0bf0030d5d4c1bb65775adff90
[2020-09-25T02:09:16.958Z DEBUG zstore::zstore] << Zstore::contains id=3422a85c3703dd0bf0030d5d4c1bb65775adff90
[2020-09-25T02:09:16.958Z DEBUG zstore::zstore] >> Zstore::insert data_len=1010 id=3422a85c3703dd0bf0030d5d4c1bb65775adff90
[2020-09-25T02:09:16.958Z DEBUG zstore::zstore] << Zstore::insert data_len=1010 id=3422a85c3703dd0bf0030d5d4c1bb65775adff90
...
[2020-09-25T02:09:16.959Z INFO zstore::zstore] >> Zstore::flush
[2020-09-25T02:09:16.959Z DEBUG indexedlog::log] >> Log::sync dirty_bytes=7048
[2020-09-25T02:09:16.959Z DEBUG indexedlog::log] << Log::sync dirty_bytes=7048
[2020-09-25T02:09:16.959Z INFO zstore::zstore] << Zstore::flush
In [6]: list(it)
Out[6]: [...]
The logs about `hgcommits::hybrid ... resolve_remote` shows the remote fetching
is working. The logs about `Zstore::insert` and `Zstore::flush` shows the
commit data were written to disk.
Reviewed By: sfilipco
Differential Revision: D23924148
fbshipit-source-id: a3d77999e29395ce5c603fe66412936947456534
Summary:
Support constructing the "hybrid" commits backend, which is similar to
"doublewrite" but read commit text from edenapi via the `streamcommitrawtext`
method.
Reviewed By: sfilipco
Differential Revision: D23924149
fbshipit-source-id: cb15ee4be7953af7798d460557ba2ae2d4f24a52
Summary:
The hybrid backend is similar to the doublewrite backend, except that it does
not use revlog to read commit data, but uses EdenAPI instead.
Note:
- The non-stream API will not fetch commit data from EdenAPI.
- The commit hashes are not lazy yet.
Reviewed By: sfilipco
Differential Revision: D23924147
fbshipit-source-id: eb2cf8d3a7e1704b4efb13ad3ad86f8b6a1b31d0
Summary:
This can be used like:
In [1]: s=cl.inner.streamcommitrawtext(repo.nodes('.%%master')) # repo.nodes returns a generator, becomes stream
In [2]: s
Out[2]: <stream at 0x7f5eec742df0>
In [3]: list(s)
Out[3]: [{'vertex': ..., 'raw_text': ...}, ...]
In [4]: s.typename()
Out[4]: 'cpython_ext::convert::Serde<hgcommits::ParentlessHgCommit>'
Reviewed By: sfilipco
Differential Revision: D23911870
fbshipit-source-id: f54959a551d446ed5b8086a2235fe74e47b29e70
Summary:
This makes it convertible to `PyObject` via `cpython_ext::convert::Serde`
without additional code or dependencies.
Reviewed By: sfilipco
Differential Revision: D23966993
fbshipit-source-id: 74d83524a7c0701cde7aa6d61bb930ff4a1c90f5
Summary:
This API allows us to stream the data. If callsites only use this API, we'll
be more confident that there are no 1-by-1 fetches.
Reviewed By: sfilipco
Differential Revision: D23911865
fbshipit-source-id: 4c7dd8c2b5be33be5a55822845d55345797bacdf
Summary:
The API is basically to resolve `input_stream` to `output_stream`, with a
stateful "resolver" that can resolve locally and remotely.
Reviewed By: sfilipco
Differential Revision: D23915775
fbshipit-source-id: 14a3a37fc897c8229514acac5c91c7e46b270896
Summary:
Introduce `FileMetadata` and `DirectoryMetadata` to `Treeentry`, along with corresponding request API.
Move `metadata.flags` to `file_metadata.revisionstore_flags`, as it is never populated for trees. Do not use `metadata.size` on the wire, as it is never currently populated.
Leaving `DirectoryMetadata` commented out temporarily because serde round trips fail for unit struct. Re-introduced with fields in the next change in this stack.
Reviewed By: DurhamG
Differential Revision: D23455274
fbshipit-source-id: 57f440d5167f0b09eef2ea925484c84f739781e2
Summary:
EdenAPI always checks the integrity of filenode hashes before returning file data to the application. In the case of LFS files, this resulted in errors because the filenode hash is computed using the full file content, but the blob from the server only contains an LFS pointer.
Fix the bug by exempting LFS blobs from filenode integrity checks. (If integrity checks for LFS blobs are desired, the LFS code should be able to do this on its own since LFS blobs are content-addressed.)
Reviewed By: quark-zju
Differential Revision: D24145027
fbshipit-source-id: d7d86e2b912f267eba4120d1f5186908c3f4e9e3
Summary:
`cpython_ext` provides utilities to implement From/ToPyObject directly for
serde types. Lets' use it to simplify the code and set up an example.
debugshell:
In [2]: s,f=api.commitdata(repo.name, list(repo.nodes('master')))
In [3]: list(s)
Out[3]:
[{'hgid': (7, 61, 22, ...), 'revlog_data': '...'}]
Note: `HgId` serialization should probably be changed to use `serde_bytes` somehow
so it does not translate to a Python tuple. That will be fixed later.
Reviewed By: kulshrax
Differential Revision: D23966987
fbshipit-source-id: 9278ccae6f543c387eafe401d4ef8d6ce96d370f
Summary:
This can be used to automate Python/Rust conversions for complex structures
like `CommitRevlogData`.
Reviewed By: kulshrax
Differential Revision: D23966988
fbshipit-source-id: 17a19d38270e6ef0952c13a1cd778487e84a94ff
Summary:
The goal is to implement `FromPyObject` and `ToPyObject` more easily.
Today crates have to dependent on `cpython` to implement `From/ToPyObject`,
which is somewhat unwanted for pure Rust crates.
The `ser` module used to ignore the `variant` field for non-unit enum variants.
They have been fixed so the serialized value can be deserialized correctly.
For example, `enum E { A, B(T) }` will be serialized to `"A"` for `E::A`, and
`{"B": T}` for `E::B`.
Reviewed By: kulshrax
Differential Revision: D23966994
fbshipit-source-id: c50d57bf313caeec65a604ed9b05a5729f3b3635
Summary:
Switch from the default tuple deserialization which only understands the tuple
format, to "bytes" deserialization, which understands not only the existing
"tuple" format (therefore compatible with old data), but also "bytes" and "hex"
formats (for CBOR).
This will unblock us from switching to bytes serialization in the future.
Note: This is a breaking change for mincode serialization. Mincode + HgId users
(zsotre, metalog) have switched to explicit tuple serialization so they don't use
the default deserializaiton and remain unaffected.
Reviewed By: kulshrax
Differential Revision: D23966995
fbshipit-source-id: 83dd53f57bd4e6098de054f46a1d47f8b48133d0
Summary: This will unblock us from switching HgId to bytes serialization by default.
Reviewed By: kulshrax
Differential Revision: D24009039
fbshipit-source-id: a277869ec24652af428cda581faffa62c25d32c4
Summary: Similar to D23966992 (2a2971a4c7), add support to serialize Key differently.
Reviewed By: DurhamG
Differential Revision: D24009041
fbshipit-source-id: 2ecf1610b989a04083196d180bc62307b5162c2f
Summary: Similar to D23966992 (2a2971a4c7), add support to serialize Sha256 differently.
Reviewed By: DurhamG
Differential Revision: D24009040
fbshipit-source-id: b77f6732802f95507e1540f0bbde4d5a92d13cac
Summary:
Add a way to specify different merge tools for interactive and non-interactive
mode.
This will be used for the default `editmerge` merge tool, which pops up the
`EDITOR` (vim) regardless of interactive mode, causing various user complains
and hangs, including `arc pull` running rebase triggering editor, or VS Code
running rebase triggering editor, and some other rebase hangs or vim errors.
Reviewed By: DurhamG
Differential Revision: D24069105
fbshipit-source-id: ec16fdc704cab6daeedb0c23d4028b4309d96d3f
Summary:
The `hg unhide` command acquired the repo lock without acquiring the wlock.
This causes locking order problems, as it calls other parts of the code that
will acquire the `wlock` (such as autopull during revset resolution) while it
is already holding the `lock`.
This can cause `hg unhide` to deadlock with other `hg` commands that acquire
`wlock` before `lock`.
Reviewed By: kulshrax
Differential Revision: D24129559
fbshipit-source-id: cf31ec661123df329f1773d2b67deb474d6476f8
Summary: Instead of returning an error upon receiving an empty request, just return a `Fetch` object that does nothing. This prevents Mercurial from crashing in situations where an empty request somehow makes it to the EdenAPI remote store.
Reviewed By: quark-zju
Differential Revision: D24119632
fbshipit-source-id: cf4ec707b4097656c76d7084a55b2d0b3150b679
Summary:
Previously, EdenAPI was using `remotefilelog.debug` to determine whether to print things like download stats. Let's give EdenAPI its own `debug` option that can be configured independently of remotefilelog.
One notable benefit of this change is that download stats will always be printed immediately after the HTTP request completes. This can help rule out network or server issues in situations where Mercurial appears to be hanging during data fetching. (e.g, if hg had downloaded all of the data but was taking a while to process it, the debug output would show this.)
Reviewed By: DurhamG
Differential Revision: D24097942
fbshipit-source-id: bf9b065e7b97fc7ffe50ab74b1b13e2fe364755c
Summary:
Previously phase calculation was done via a simple ancestor check. This
was very slow in cases that required going far back into the graph. Going a year
back could take a number of seconds.
To fix it, let's take the Rust phaseset logic and rework it to make only_both
produce an incremental public nodes set. In a later diff we can switch the
phaseset function to use this as well, but right now phaseset returns IdSet, and
that would need to be changed to Set, which may have consequences. So I'll do it
later.
Reviewed By: quark-zju
Differential Revision: D24096539
fbshipit-source-id: 5730ddd45b08cc985ecd9128c25021b6e7d7bc89
Summary:
A couple of files were moved but test-check-code.t wasn't updated to reflect
this, causing it to fail.
Reviewed By: DurhamG
Differential Revision: D24113079
fbshipit-source-id: 9a0c0b6f07a6532715bf5ee401036ded0a05b16a
Summary:
Pull Request resolved: https://github.com/facebookexperimental/eden/pull/65
Using $LOCALIP will ensure more consistent behavior when setting up the server in ipv4 or ipv6.
The LOCALIP variable was also abused when it was used to override ssh client address, so SSH_IP_OVEERIDE env was created here.
Lastly the result of `curl` call is now printed whenever the test failed to verify that Mononoke is running.
Reviewed By: farnz
Differential Revision: D24108186
fbshipit-source-id: e4b68dd2c5dd368851f0b00064088ffc442e31e0
Summary: D24070707: `[Thrift] Provide sorted fields to read_field_begin` made a change to the generated rust thrift files, so the eden/scm thrift files have to be regenerated to fix the build.
Reviewed By: farnz
Differential Revision: D24109655
fbshipit-source-id: e8575a76642673a11514fdce8e30f13ca28151f0
Summary:
Formats a subset of opted-in Python files in fbsource.
Black formatting was applied first, which is guaranteed
safe as the AST will not have changed during formatting.
Pyfmt was then run, which also includes import sorting.
The changes from isort were manually reviewed, and
some potentially dangerous changes were reverted,
and the directive was added to those
files. A final run of pyfmt shows no more changes to
be applied.
Reviewed By: zertosh
Differential Revision: D24101830
fbshipit-source-id: 0f2616873117a821dbc6cfb6d8e4f64f4420312b
Summary: This can be used by dag::Vertex and minibytes::Bytes.
Reviewed By: kulshrax
Differential Revision: D23966985
fbshipit-source-id: 3b4b29648e038ef49f26ce2b500119e148544d9e
Summary:
The py_stream_class causes the code to be more verbose. It basically enforces
the bindings crate to define new types wrapping pure Rust types, and then
define py_stream_class.
In a future diff, I'm adding FromPyObject/ToPyObject support for types that
implements serde Deserialize/Serialize. py_stream_class gets in the way,
because the blanket type from cpython-ext cannot be used in the py_stream_class
macro. cpython-ext is not the proper place to define business-related stream
types.
Therefore, define a type-erased Python class, and implement
FromPyObject/ToPyObject automatically for TStream<anyhow::Result<T>> where
T implements FromPyObject or ToPyObject.
The FromPyObject now converts a Python iterator back to a stream. It's
no longer zero-cost. However, I'd imagine such usecases can be short-cut
using pure Rust code.
Background: Initially, I added some FromPyObject/ToPyObject impls to pure
Rust crates gated by a "pytypes" feature. While that works fine with cargo
build, buck does not support dynamic features and the fact that we support
both py2 and py3 makes it extremely hard to support cleanly in buck build.
For example, if minibytes::Bytes defines ToPyObject for Bytes, then any
crate using minibytes would have 2 different versions: a py2 version, a
py3 version, and they both depend on python. That seems to be a bad approach.
Reviewed By: sfilipco
Differential Revision: D23966984
fbshipit-source-id: eafb31ad458dcbdd8e970d8e419a10fbbe30595f
Summary:
Per the feedback on D23920367 (318f5683a5), let's make the human-readable download stats shorter. Example:
```
Downloaded 10.59 MiB in 12.35s over 5 requests (7.19 Mb/s, latency: 123ms)
```
The amount downloaded is now reported in binary-prefixed bytes (so that it can be directly compared to file sizes) whereas the transfer rate is reported in decimal-prefixed bits per second (so that it can be directly compared to a user's measured network speed).
Additionally, we now use the default formatting available from `std::time::Duration`, which will automatically choose the appropriate display units.
Reviewed By: quark-zju
Differential Revision: D24096525
fbshipit-source-id: 39c49f1b08135bbae7a7544b1ffe2bdbfe1533a1
Summary:
The bfs fetching path in Rust was broken because it directly called
getdesignatednodes. getdesignatednodes returns False if it wasn't able to
succeed, so we need to be able to fall back in that case. The _prefetchtrees
function is meant for that, so let's just call that.
Reviewed By: singhsrb
Differential Revision: D24090946
fbshipit-source-id: d16c2c8f80f690a22046385f0e95785996a62949
Summary:
Previously, only the first `__iter__` gets wrapped. With D23095468 (34df768136), the first
`__iter__` is used by the "simplify graph" feature, not the main iteration
loop rendering the graph log output, causing the prefetch feature to fail.
File "edenscm/mercurial/commands/__init__.py", line 4196, in log
return cmdutil.graphlog(ui, repo, pats, opts)
# pats = ()
File "edenscm/mercurial/cmdutil.py", line 3250, in graphlog
ui, repo, revdag, displayer, graphmod.asciiedges, getrenamed, filematcher
File "edenscm/mercurial/cmdutil.py", line 3106, in displaygraph
rustdisplaygraph(ui, repo, dag, displayer, getrenamed, filematcher, props)
File "edenscm/mercurial/cmdutil.py", line 3208, in rustdisplaygraph
for (rev, _type, ctx, parents) in dag:
File "edenscm/mercurial/graphmod.py", line 63, in dagwalker
rootnodes = cl.tonodes(revs)
# revs = <baseset- [7408158, 72057594037927936, ...]
File "edenscm/mercurial/changelog2.py", line 196, in tonodes
return self.inner.tonodes(revs)
# revs = <baseset- [7408158, 72057594037927936, ...]
File "edenscm/hgext/phabstatus.py", line 281, in next
return self.__next__()
Lift the "first time" limitation for wrapping `__iter__` to solve the problem.
Reviewed By: simpkins
Differential Revision: D24066918
fbshipit-source-id: 6bbd244e729724e5143147bde60bcb4c8ee4bc80
Summary: Since this is only used in the manifest target, fold it into it.
Reviewed By: DurhamG
Differential Revision: D24062629
fbshipit-source-id: c3241b53bde7abba8a80a2945661d1a24b7e3034
Summary: We now get progress bar output when fetching from memcache!
Reviewed By: kulshrax
Differential Revision: D24060663
fbshipit-source-id: ff5efa08bced2dac12f1e16c4a55fbc37fbc0837
Summary: These aren't included anywhere, we can remove them.
Reviewed By: DurhamG
Differential Revision: D24062627
fbshipit-source-id: 9ff101eb44965ac3502ada3265ffcc8acc09d2e5
Summary:
This is no longer about datapack, but only about parsing manifest entries, thus
renaming.
Reviewed By: DurhamG
Differential Revision: D24062634
fbshipit-source-id: 5c52b784d20437e87012dd4bc6cb13d879da9cb9
Summary: The code doesn't use anything from libmpatch, we don't need to depend on it.
Reviewed By: DurhamG
Differential Revision: D24055084
fbshipit-source-id: 0f7bac73f1b711da4395e25619577a0a2e0ca959
Summary: These are unused, no need to keep the code around.
Reviewed By: DurhamG
Differential Revision: D24055085
fbshipit-source-id: 6246d746983a575c051ddcb51ae02582a764a814
Summary: This is unused, no need to keep it around.
Reviewed By: DurhamG
Differential Revision: D24054164
fbshipit-source-id: 161b294eb952c6b4584aa0d49d8ff46cd63ee30f
Summary:
This code is effectively unused. The only bit still relevant is that EdenFS
still depends on the Manifest class to parse a manifest.
Reviewed By: DurhamG
Differential Revision: D24037723
fbshipit-source-id: 901ae2ffc8960a95ec655a2e14d79afb8d32dcab
Summary: This is unused, let's remove it.
Reviewed By: DurhamG
Differential Revision: D24037722
fbshipit-source-id: bc8a272809cb1f20f54d651a39ee42ff57169534
Summary:
The 'diff()' template needs to be able to output non-utf8 patches.
To enable this, let's change the templater to output bytes instead of strings.
Note, the templater supports json output, which requires utf8 encoded values. In
that situation we escape any non-utf8 binary and clients would have to be aware
and unescape them.
Reviewed By: quark-zju
Differential Revision: D24029105
fbshipit-source-id: aa011696c0d97281b91c6871c116037d76c178e0
Summary:
This is unused code. It broke a test while I was doing python 3 work,
so let's just delete this.
Reviewed By: quark-zju
Differential Revision: D24063953
fbshipit-source-id: 323b323ff2b9717c9c16eed54adac467a321e2ae
