Summary:
We had a bug: if two files were reverted and then we try to amend one of them
mercurial will actually amend both of them.
Looks like the problem was caused by "Prune files reverted by amend" block.
Previously this block was considering every file that was changed in a commit we
are about to amend and comparing with working copy. If a file is the same in a
commit we are about to amend and in the working copy then it will amended as
well.
This diff attempts to fix it by considering only files that were selected for
amending.
Reviewed By: DurhamG
Differential Revision: D24949727
fbshipit-source-id: cf6cb95af3f67ec769e8a58db3b829945133b830
Summary:
We have an edge case - if we reverted two files and then we try to amend only a
single one then both of them will be amended.
This diff adds a test for this and other edge-cases. The next diff will fix it
Reviewed By: DurhamG
Differential Revision: D24949726
fbshipit-source-id: c5c53de1d67f161efa8564f89127e61ac2f28ac9
Summary:
When we streamclone, we snapshot the revlogs under a lock, then we start
sending. That works fine, unless we have a file whose size changes during
the sending phase. This can happen if it's promoted from a single `.i` to a
`.i` and a `.d`.
When that happens, the clone fails (the client reports it received a bunch of
corrupted data because it starts interpreting parts of files as inputs). Since
the breakage is also confusing client side, I updated the server side to also
assert that it's sending what it thinks it's sending.
Reviewed By: DurhamG
Differential Revision: D24958885
fbshipit-source-id: a0349c651b7cb63ab27546adf9944e7fba63a95d
Summary:
The wirepack sending code builds up the entire history blob in memory
before sending it. Previously we did this by appending to the string. In Python
2 this was fast, in Python 3 this is n^2 and n can be 100k+ in cases of long
history.
Let's switch to list+join.
Reviewed By: xavierd
Differential Revision: D24933183
fbshipit-source-id: 5c36d7868e7c64a2292bd68ec2ffb584d85dd98f
Summary:
osxfuse is rebranding as macfuse in 4.x.
That has ripple effects through how the filesystem is mounted and shows up in
the system.
This commit adjusts for the new opaque and undocumented mount procedure and
speculatively updates a couple of other code locations that were sensitive to
looking for "osxfuse" when examining filesystems.
Reviewed By: genevievehelsel
Differential Revision: D24769826
fbshipit-source-id: dab81256a31702587b0683079806558e891bd1d2
Summary:
We got a [report](https://fb.workplace.com/groups/scm/permalink/3379140858802177/) that a new hg build fails with an error because it can't xor None types.
```
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: PyErr {
ptype: <type 'exceptions.TypeError'>, pvalue: Some("unsupported operand type(s)
for ^: 'NoneType' and 'NoneType'"), ptraceback: Some(<traceback object at
0x00000249BB158248>) }',
```
Full stack trace is here
{P149395441}
This seems likely to be related to the diff I landed recently - D24725902 (7b1798be37).
However it's unclear why it was affecting only windows because I couldn't repro
it on linux.
Turned out that we have experimental.treematcher option disabled on windows,
which causes it to use includematcher instead of treematcher. And includematcher
returns either None or BytesMatchObject and they are impossible to xor.
This diff fixes it by converting them value to bool first, and also it adds a
test for it.
Reviewed By: singhsrb
Differential Revision: D24918192
fbshipit-source-id: 1359e8b97d26d3b1a4795b7b3d4cfa3d6d4ae843
Summary:
We've seen http 2 potentially causing hangs for users. Let's make this
configurable for lfs, so we can disable it and see if things get fixed.
Reviewed By: krallin
Differential Revision: D24898322
fbshipit-source-id: dc7842c0247dc6b9590a1f160076b17788aab1b9
Summary:
As discussed in a group thread (see link below), HTTP 2 may be causing
hangs for users. Let's start by making the http-client configurable. In
subsequent diffs we'll make edenapi and lfs configurable as well.
Reviewed By: krallin
Differential Revision: D24898323
fbshipit-source-id: f0035a1b8df3cee626ebe519e9e99358c1b3f043
Summary:
This isn't code that compiles, but the convention in Rust is that code actually
is doctests unless annotated otherwise, so if tested with Cargo, those fail.
This fixes that.
Reviewed By: farnz
Differential Revision: D24917364
fbshipit-source-id: 62fe11700ce561c13dc5498e01d15894b17b5b22
Summary:
Thread Pool fails with py3 hg build. Let's replace with a loop.
Most of the usage for the command will be for a single head anyway.
Reviewed By: krallin
Differential Revision: D24902167
fbshipit-source-id: c7af46d0d63ddd074c98788bf55520ae3f2550b8
Summary:
Transfers iddag flat segments along with the head_id that should be use to
rebuild a full fledged IdDag. It also transfers idmap details. In the current
version it only transfers universal commit mappings.
Reviewed By: krallin
Differential Revision: D24808329
fbshipit-source-id: 4de9edcab56b54b901df1ca4be5985af2539ae05
Summary:
BE: remove old subscription to save resources in IceBreaker. The client code will recreate it anyway if missing but cleaning up will help us to reduce number of unused subscriptions.
Classic example: repo opsfiles or configerator maybe needed once and then a user don't use
Another example: switching workspaces failed and it could be result in subscriptions are not cleaned up properly
Reviewed By: markbt
Differential Revision: D24859931
fbshipit-source-id: 6df6c7e5f95859946726e04bce8bc8f3ac2d03df
Summary:
Those are the tweaks I've made to make `--config devel.bundle2.debug` more
verbose to aid with my investigation. This might help somebody else in the
future so let's comit it:
* added "params" decoding to debugsendbundle
* added "message" to `error:unsupportedcontent` part (we already send it with
some other error parts)
Reviewed By: sfilipco
Differential Revision: D24840405
fbshipit-source-id: b25d5823d05f3d50230c078e8db459dc66256707
Summary:
This function is useful in the mononoke to compute the universal commit idmap
that is required for clone.
Reviewed By: quark-zju
Differential Revision: D24808327
fbshipit-source-id: 0cccd59bd7982dd0bc024d5fc85fb5aa5eafb831
Summary:
`flat_segments` are going to be used to generate CloneData. These segments will
be sent to a client repository and are going to bootstrap the iddag.
Reviewed By: quark-zju
Differential Revision: D24808331
fbshipit-source-id: 00bf9723a43bb159cd98304c2c4c6583988d75aa
Summary: This is the object that will be used to bootstrap a Dag after a clone.
Reviewed By: quark-zju
Differential Revision: D24808328
fbshipit-source-id: 2c7e97c027c84a11e8716f2e288500474990169b
Summary:
The goal is to reused the functionality provided by AssignHeadOutcome for clone
purposes.
Reviewed By: quark-zju
Differential Revision: D24717924
fbshipit-source-id: e88f21ee0d8210e805e9d6896bc8992009bd7975
Summary:
The original problem was a fastlog bug, solved by D24513444 (c3bcc1ab88).
Restores prefetching for phabricator status so `hg ssl` and `hg fssl` become fast again.
Original commit changeset: b10c4caf8fda
Reviewed By: sfilipco
Differential Revision: D24749774
fbshipit-source-id: fa14f7dde9c922733525a7ff014efc32875426fa
Summary:
The original issue was a rust-cpython bug, solved by D24698226, or https://github.com/dgrunwald/rust-cpython/pull/244.
Original commit changeset: 08f598df0892
Reviewed By: sfilipco
Differential Revision: D24759765
fbshipit-source-id: f9a1359cfce68c8754ddd1bcb8bfc54bf75af7ff
Summary:
On Windows terminal, with light color schemes, crecord text was barely visible
(sometimes invisible) due to low contrast on either the background, or the
foreground. Making the text bold makes it brighter and thus more readable.
As a bonus, I've also made the hunk lines magenta to mimic what `hg diff` does.
Reviewed By: DurhamG
Differential Revision: D24718598
fbshipit-source-id: 18c2ff03fc2a46ca45808d5061db21e1f1b501ae
Summary: This makes it clean up stale files more aggressively.
Reviewed By: DurhamG
Differential Revision: D24744461
fbshipit-source-id: 76d163c9f16d8f8d1bf628e9197a3086d7cd48aa
Summary:
The goal of this code was to divide the cache limit by the number of
logs. Instead it divided the cache limit by the default per-log size (2GB). That
results in a very small max-bytes-per-log so data was being thrown out
constantly. This fixes it and updates tests to actually demonstrate the issue.
Reviewed By: kulshrax
Differential Revision: D24712842
fbshipit-source-id: 8062758b5bfa40493e2003d5a9028d601b1522b1
Summary:
Python 3 doesn't support line buffering for binary file descriptors.
Let's stop setting it in chg.
This was causing warnings to pop up during prompts for users.
```
.../python3.8/os.py:1023: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
return io.open(fd, *args, **kwargs)
```
Reviewed By: singhsrb
Differential Revision: D24747777
fbshipit-source-id: 0b881b4067e8c7086fe73380f81d526a2ecc364a
Summary:
Downloading and applying mercurial bundles directly for list of given heads.
Backup store stores commits as mercurial bundles that can be fetched directly from the store and applied (everstore).
The command could be useful when we migrate our server from one backend to another (Mononoke) and some commits can be missing in Mononoke.
The command could probably be deleted after a while once we migrate completely...
Reviewed By: mitrandir77
Differential Revision: D24756583
fbshipit-source-id: 1629c3756f244621efb965dfe15b75c7509a1cd1
Summary:
This diff makes "Calculating additional actions for sparse profile update" more
efficient by using xormatcher instead of unionmatcher. Indeed, we are
interested only in files that changed their "state" after sparse profile change
e.g. either a file was included in sparse profile and then became excluded.
Reviewed By: sfilipco
Differential Revision: D24725902
fbshipit-source-id: ee611e7c123b95937652ced828b5bea6d75a3daf
Summary:
At the moment differencematcher.visitdir never returns "all".
This diff changes it to return all in the case if self._m2 doesn't visit the directory at all and
self.m1.visitdir(dir) returns "all". This makes sense - if m1 visits all files
in the directory and m2 doesn't exclude any file then it's safe to return all
in this case.
This optimization will be used in the next diff.
Reviewed By: sfilipco
Differential Revision: D24725903
fbshipit-source-id: 2a049cfb1ea4878331e8640cbb20af74da86a1a1
Summary:
Whenever a sparse profile changes (e.g. we include or exclude a directory or a file) we do a full prefetch for all trees in the revision and then for each file in a revision we check if this file has changed its state after sparse profile change (i.e. whether it was included before the change and became excluded after the change and vice versa). It can be quite expensive for large repos and looks like checking all the files is unnecessary.
For example, there might be top-level directories that are excluded in sparse profile before and after the change. In that case there's no reason to check every file in this directory, and there's no reason to prefetch manifests for this directory.
More importantly, `mf.walk()` method is already smart enough to do manifest prefetches if treemanifest.ondemandfetch is set to True, so it looks like there's no reason to do any additional prefetching at all (at least in theory).
So this diff does a few things:
1) The default mode is to use mf.walk() method with a union matcher to find all the files that were are included either in old or new sparse profile. In order for it to prefetch efficiently we force enable treemanifest.ondemandfetch config option.
2) It also adds a fallback option to full prefetch (i.e. the same thing we do right now) Hopefully this fallback option won't be necessary and we'll delete them soon. I've added them only to be able to fallback to current behaviour in case there are problems with the new behaviour
I think we can do an even more efficient fetch by using xor matcher instead of union matcher. I'll try to implement it in the next diffs
Reviewed By: sfilipco
Differential Revision: D24705823
fbshipit-source-id: 2c232a66cc74ee95bdaa84201df46448412f087f
Summary:
The revlog changelog has incompatible rev numbers with changelog2 backends. Do
not construct it. Instead, just use the current changelog.
Reviewed By: DurhamG
Differential Revision: D24513444
fbshipit-source-id: 35d9326cd9fde4af8b98d628f6df66bd80883f92
Summary:
The goal of this diff is to provide more visibility into how long the client
takes to create/upload an infinitepush bundle. This is done in two ways:
- by adding more `perftrace` calls (useful when invistigating individual slow
pushes)
- by adding `ui.timesection` scopes (useful for aggregation purposes)
Two main things that are measured:
- creation of the bundle purely on the client
- sending of the bundle over the wire
In addition, in the perftrace recording, this measures how long it takes to
process the reply handlers, how much bytes are sent over the wire, what are the
part names and sizes (when available). These changes mostly do not distinguish
whether the code is infinitepush push or not, but they are always related to
some sort of a wireproto scenario, which means that the performance impact is
negligible (writing things to thread-local storage is *much* cheaper than
sending them over the network).
Reviewed By: DurhamG
Differential Revision: D24683484
fbshipit-source-id: 53fdfb63dcdfcf38924237c59a1e8f5e24ff96c0
Summary: We were incorrectly marking reverts as landed during pullcreatemarkers.
Reviewed By: quark-zju
Differential Revision: D24608217
fbshipit-source-id: f919f49469d6933c17894b3b0926ba2430a5947a
Summary:
As part of getting buck build to work on OSX, we need procinfo to
include it's OSX specific library.
Reviewed By: sfilipco
Differential Revision: D24513234
fbshipit-source-id: 69d8dd546e28b4403718351ff7984ee6b2ed3d1d
Summary:
Improve ux for setting the token.
This extra bits would help to figure out prompt issues on Windows
Reviewed By: markbt
Differential Revision: D24706085
fbshipit-source-id: 6101b8d7b90aad2d687465a09cc69670ca4a46f6
Summary:
In Mercurial we're finding use cases for conditioning on buck vs
non-buck builds. For instance, in some cases we specify #[link] directives for
non-buck builds, but don't need to for buck builds
(https://fb.workplace.com/groups/rust.language/permalink/4568487309866515/). In
other cases we want to depend on internal buck libraries when building with
buck, and not otherwise.
Let's add a fb_buck_build cfg so other projects may distinguish the same.
Reviewed By: jsgf
Differential Revision: D24493974
fbshipit-source-id: 1d558cbe0ae01ab4a7b4b5d6d4be75bb8ab0f41a
Summary:
The rage local config output was getting polluted by dynamic config.
Let's filter them out.
Reviewed By: farnz
Differential Revision: D24626564
fbshipit-source-id: df5ac04cd549595ecdccc0b2438d4e7c72b80e88
Summary:
Previously, reading from SSH subprocesses did’t return on Python 3. bufsize=0
is default on Python 2, but not Python 3.
This is a backport of: 0fc8020ebe
Reviewed By: krallin
Differential Revision: D24652418
fbshipit-source-id: 1140c76b6f711bfe1726108bd4fe6948e6ee41a0
Summary:
I initially saw the incremental build as something that would be run in places
that had IdMap and IdDag stored side by side in process. I am reconsidering
to use incremental build in the tailing process to keeps Segmented Changelog
artifacts up to date.
Since we update the IdMap before we update the IdDag, it is likely that we
will have runs that only update the IdMap and fail to update IdDags. This diff
adds a mechanism for the IdDag to catch up.
Reviewed By: krallin
Differential Revision: D24516440
fbshipit-source-id: 3a99248451d806ae20a0ba96199a34a8a35edaa4
Summary:
mitrandir77 pointed me to this field which is exactly what I want.
Leave alias handling in place for configerator, which has a different name.
Reviewed By: mitrandir77
Differential Revision: D24628323
fbshipit-source-id: ccd01e6314feefd8ab75e8052a0a8a98b3119691
Summary: This diff removes reading the token from the "cert" property in the `~/.arcrc` and forces the caller to use either an OAuth token or CATs.
Reviewed By: quark-zju
Differential Revision: D24242614
fbshipit-source-id: 18538270102b7aa28731e82c8dd21f5da9e2f2d6
Summary:
We have automation that wants to use this to hold the lock while doing
some maintenance. They want the ability to wait for the lock so they don't have
to busy loop.
Reviewed By: snarkmaster
Differential Revision: D24604466
fbshipit-source-id: be02539908655e183f334865718b68b633b069a5
Summary:
I am wondering whether we should customize the serialization format for the
InProcessStore. I want to have a basis for the comparison before I proceed.
Reviewed By: quark-zju
Differential Revision: D24580273
fbshipit-source-id: d3ddfdc029dbdd84f60acace06fddc80b4d005f4
Summary: We had some aliases for fbsource, and `fbcode` is still in use. Teach phrevset to recognise all aliases via config for ease of patching.
Reviewed By: markbt
Differential Revision: D24589906
fbshipit-source-id: bd61e86135d63ae07fa62d741e16cea4882f691b
Summary:
D24293498 (ea03d62698) is causing 'hg log scm/metagit' to fail with weird revision
number lookup errors. Let's back it out for now.
Reviewed By: singhsrb
Differential Revision: D24549796
fbshipit-source-id: b10c4caf8fdadd2955ae1f829381c66d788042e6
Summary: This is essentially a backout of D24365328 (9f664a8b30).
Reviewed By: DurhamG
Differential Revision: D24544495
fbshipit-source-id: 08f598df0892a8479fac563096f9782038e18dfe
Summary: This change adds wire types for the history API in the most straightforward way possible. In a future change, I'll move the `WireHistoryEntry` / `HistoryResponseChunk` conversion logic into the `ToApi` implementation. This implementation also doesn't add per-item errors, or standardize `history` to match the protocol evolution standards used by `trees`.
Reviewed By: kulshrax
Differential Revision: D24342046
fbshipit-source-id: d46403a823f2a1e89ad9d6d2074241d8bfe4810e
Summary: This change introduces custo `Serialize` and `Deserialize` implementations for wire types containing a fixed-length byte array. This change is more verbose than should be necessary. Because `TryFrom` is implemented to convert `&[T]` to `[T; N]` and `AsRef` to convert `[T; N]` to `&[T]`, we should be able to use a generic serde `with = ` attribute on the byte array fields of the appropriate structs. Unfortunately, I'm getting a lifetime issue that I haven't been able to resolve, and const generics aren't available on stable to implement the trait bounds without an explicit lifetime.
Reviewed By: kulshrax
Differential Revision: D24460527
fbshipit-source-id: d3a4179b833a523ce164d3df934e4cbdc2202546
Summary:
Removes top-level metadata from `TreeEntry` and add a new, specialized type for carrying child entry metadata, `TreeChildEntry`. This change also fully removes the `revisionstore_types::Metadata` from EdenAPI trees, which is only used on files.
In a follow up change I'll optimize handlings of paths / path segments in `TreeChildEntry` keys. Right now they need to be joined manually.
Reviewed By: kulshrax
Differential Revision: D24434716
fbshipit-source-id: d0739471b1f6cef58b435e10b5fb774bfb08f7f6
