Summary: This will be used by upcoming changes.
Reviewed By: andll
Differential Revision: D29114048
fbshipit-source-id: ce963a42ebc5722ca9c61ab0a4e7a7377bbf7abb
Summary:
The mutationstore might contain lots of names (commit hashes) that are unknown
to the repo. When using a lazy changelog, by default, those names will be
looked up remotely, and most of the time the server just returns "name is outside
the graph".
We don't have negative cache to speed it up, because cache invalidation is
tricky - every lazy pull would in theory invalidate them.
To make things faster, let's just skip names in the mutationstore that are unknown
_locally_ without asking the server.
In theory this might affect correctness. Practically, this should only affect "landed as" markers,
because all drafts should be non-lazy. If the "landed as" correctness is an issue, we can fix
forward "landed as" later (ex. by writing down the public commit hash explicitly in
debugmarklanded).
Reviewed By: andll
Differential Revision: D29111710
fbshipit-source-id: 2c1b16a8140ca4f7195bf1206ffe1db4750185b9
Summary: This allows BTreeMap to be used as a provide of config for testing.
Reviewed By: andll
Differential Revision: D28019023
fbshipit-source-id: 94a1585139d6c6612fa163f2ff1aecc909db3a72
Summary: Bumping the crossbeam version so we can use `recv_deadline`. This also necessitates updating the lsp and lsp-types crates.
Reviewed By: alunyov, dtolnay
Differential Revision: D29056473
fbshipit-source-id: 9434e9e0895d82482f4c70afa01a2f77702b965f
Summary:
This functionality is exercised in `test-lfs-copytracing.t` and a few other tests, though I'm not sure if it's necessary outside of that. I've added a `TODO` to investigate and try to eliminate such cases.
Because we'd rather not support this, I implemented it on top of ContentStore - it looks like we might have some lingering datapack-related tests where I think this comes up, too.
Reviewed By: DurhamG
Differential Revision: D29056647
fbshipit-source-id: e6bd5ec31dde06d8c509665f738cfe19270ddc76
Summary:
Modify the `ExtractInnerRef` implementations for `filescmstore` and `treescmstore` to return the new scmstore objects rather than the deprecated ones.
Modify `pycheckout` to use a new methods, `get_oldscmstore`, to access the deprecated scmstore instead (native checkout will be updated to use the new scmstore later on).
Reviewed By: DurhamG
Differential Revision: D29056498
fbshipit-source-id: 09b4655941bca048b01e6d365fcca1ccfb1cd700
Summary: Temporarily implement `upload` on `FileStore` by forwarding to the existing `upload` implementation using `LfsRemote`. Eventually we'll want to re-implement this entirely inside `FileStore`.
Reviewed By: DurhamG
Differential Revision: D29045264
fbshipit-source-id: 8ebbaead4c1032949e5bdd1dcf9a733d0086a77c
Summary: Add a `get_shared_mutable` method to `TreeStore` which behaves like `ContentStore::get_shared_mutable`, returning a store which will read/write only to the local cache stores (aka shared stores).
Reviewed By: DurhamG
Differential Revision: D29042716
fbshipit-source-id: f2236ff8b47ef213b2ffc61501ca301da02dc492
Summary: Add a `get_shared_mutable` method to `FileStore` which behaves like `ContentStore::get_shared_mutable`, returning a store which will read/write only to the local cache stores (aka shared stores).
Reviewed By: DurhamG
Differential Revision: D29042485
fbshipit-source-id: e28af0bafac5eba87523b0ef522f32355106b467
Summary: Add `FetchLogger` to `FileStore` to track remote fetches which match `remotefilelog.undesiredfileregex`.
Reviewed By: DurhamG
Differential Revision: D29042243
fbshipit-source-id: 08ec69d979d66ef93e8bbe8659171a412ade7e22
Summary: `ReportingRemoteDataStore` wraps a store and records all fetches which match `remotefilelog.undesiredfileregex`. Since scmstore doesn't use a hierarchical store composition, this change extracts out the matching/reporting logic into `FetchLogger` so that it can be re-used by both `ReportingRemoteDataStore` and scmstore.
Reviewed By: DurhamG
Differential Revision: D29041488
fbshipit-source-id: db0c4b545886ffebcf2a2841a506d301b2f2f230
Summary: Straightforward update of `debugscmstore` from old scmstore to new scmstore. I'll want to improve this command a bit more in the future, but this at least enables you to easily test arbitrary fetches.
Reviewed By: DurhamG
Differential Revision: D29047527
fbshipit-source-id: 4e10cb88cba4b572d3e413640ca3d800940d675d
Summary:
Our log deletion is a two step process. First it deletes the meta file,
then it deletes the log. There was a bug where if the meta file deletion
succeeded but the log deletion failed (often because Eden has the log open) then
future attempts to delete the log will fail because it fails to delete the meta
file.
Reviewed By: quark-zju
Differential Revision: D29035913
fbshipit-source-id: 33ab9a4e46b285819c4bf14031426983d2b4342f
Summary:
Since all the errors are eaten, we don't have good insight into the
rotatelog cleanup process. Windows users were seeing large growth in their
hgcache and this logging would've been useful, so let's add it.
Reviewed By: quark-zju
Differential Revision: D29035914
fbshipit-source-id: 54f0f05aa8bed1911b6b95438cd2d3ccc45e8ec9
Summary:
The crecord width calculation used a function that requires a string,
so we tried to force the rows from bytes to columns, which crashed if the row
wasn't utf8. Let's just use the error="replace" utf8 error handler since we
don't care if the row computation is slightly off.
Reviewed By: quark-zju
Differential Revision: D29043385
fbshipit-source-id: a00d867bca9c62007ae4f68b6198d159deeb6016
Summary: This method will be used to get all Ids that needs to be included into CloneData::id_map during fast forward pull.
Reviewed By: quark-zju
Differential Revision: D29045538
fbshipit-source-id: f6eb04f537aa5365f7588391ea4c60b3dc010a53
Summary: revert the zstd crates back to previous version
Reviewed By: johansglock
Differential Revision: D29038514
fbshipit-source-id: 3cbc31203052034bca428441d5514557311b86ae
Summary:
implement `known` method based on the existing `lookup` API that has been already implemented
this might be useful for eden api pull project / wireproto deprecation, as the pull algorithm queries the server for whether commits are known by the server
Differential Revision: D29030429
fbshipit-source-id: 4d78c8b1e7983d89de917951d9765b1bd1d90fb0
Summary: This will be used in fastpath pull
Reviewed By: quark-zju
Differential Revision: D29010304
fbshipit-source-id: 9a7464fbc42544f8c5cc52f6fa25db5f7b5d6bd9
Summary: idset_to_flat_segments allows to return intersection between flat segments and span set
Reviewed By: quark-zju
Differential Revision: D28980521
fbshipit-source-id: da90c66a021a1e8bb95edbbc2e06cd2a850d7dcd
Summary:
This diff introduces Subspan trait for objects that contains spans and allows to take a 'subspan' of an object
Implementing Subspan trait allows to intersect arbitrary objects and spans
Reviewed By: quark-zju
Differential Revision: D28980523
fbshipit-source-id: 29a2e58ec3c79a838db751d8a37227b4cdaaa372
Summary:
It is only useful for Python revlog-based changelog, which was removed by the
previous diff.
Reviewed By: DurhamG
Differential Revision: D29020190
fbshipit-source-id: 3332f1532a510dc72f21d68801e093b5eba10145
Summary:
Modify the status command to prefer comparing content sha256 when comparing files over comparing content directly, using cached aux data when possible and writing newly computed hashes to cache.
A new config option, `scmstore.status`, gates this new behavior.
When `scmstore.status` is enabled, the `status` output should be more accurate, not assuming files with non-matching `HgId`s in an arbitrary pair of revisions are different. As written, this will cause fetching of data that wouldn't otherwise be fetched, but remote fetching of aux data is planned.
This change also doesn't introduce batch fetching, this is planned.
Reviewed By: DurhamG
Differential Revision: D28826219
fbshipit-source-id: dcd253f169723983738148559d61ba655dd453a4
Summary: Introduce a config for enabling separate indexedlogs for storing "aux data" in scmstore. The storage format is not yet finalized, and isn't used in production yet, so we gate the constructing it on a cofig.
Reviewed By: DurhamG
Differential Revision: D28826209
fbshipit-source-id: 1ed8c20329f21b3ced598eea16d9fac0f40112e6
Summary:
Make `doctor` aware of non-revlog changelog and do not hide commits
incorrectly.
Reviewed By: DurhamG
Differential Revision: D29020189
fbshipit-source-id: af7fab75a877997e3f11dd21e506c965da078f83
Summary: The doctor command assumes revlog changelog, which does not cover all setups.
Reviewed By: DurhamG
Differential Revision: D29020188
fbshipit-source-id: 68bb4e20efb49482df84a71ab7d86a20a6fa93ce
Summary: For all clients and tests we're using revlog changelog based on Rust.
Reviewed By: DurhamG
Differential Revision: D29020191
fbshipit-source-id: 44ab603ea07b8891831692285e94ea0e4df849f5
Summary:
Verify is tightly coupled with revlog details, and assumes a non-lazy repo.
Revlog is going away and our repo is lazy typically, so let's just make verify
a no-op.
Reviewed By: DurhamG
Differential Revision: D28974546
fbshipit-source-id: 487dc65ceb9a1e445b58d03a53353a8382aefc08
Summary:
Practically, our client-side revlog changelogs should be non-inline now for a
long time. There is no need to keep the migration logic.
The revlog is being deprecated too so its implementation details (inline) is
going to be irreverent. The related test is then removed.
Reviewed By: DurhamG
Differential Revision: D28974551
fbshipit-source-id: ea456c46dac11d6a8b225c269b49598ab34c2548
Summary: It was only useful server-side and will be incompatible with upcoming changes.
Reviewed By: DurhamG
Differential Revision: D28974549
fbshipit-source-id: 70a715ce170baa78adb8b1fcf7d29e3d1479c05e
Summary: hgsql is irrelevant. Remove more tests that will be broken by upcoming changes.
Reviewed By: DurhamG
Differential Revision: D29019287
fbshipit-source-id: 6fd04d2eb088a0ca9c975b25a4f28a5772f0e088
Summary:
This test uses bundle2 details that are hard to maintain.
Let's just remove the test.
Reviewed By: DurhamG
Differential Revision: D29019286
fbshipit-source-id: a64918736039331bf2fc3cd23e9c67dd77510c22
Summary:
This test is too tricky to maintain with modern setups. Namely, we don't
support strip and are dropping revlog usage. Let's just remove the test.
Reviewed By: DurhamG
Differential Revision: D28974550
fbshipit-source-id: e8d30e726735432820ceaf4ef27d1b83753122a4
Summary:
We no longer uses this code base for hg server logic.
The test requires non-Rust commit backend which will be removed.
Reviewed By: DurhamG
Differential Revision: D28974547
fbshipit-source-id: 433a6697f6cbf08450c43ce810490fcdb53cf718
Summary: It tests revlog details that are going to be irrelevant.
Reviewed By: DurhamG
Differential Revision: D28974552
fbshipit-source-id: 3ff08473be236849442c3c30d5cf1e1c2a1b628d
Summary: It tests revlog details that are going to be irrelevant.
Reviewed By: DurhamG
Differential Revision: D28974548
fbshipit-source-id: a44e97daa24aece446d899e7711a59cb4a133398
Summary: Revise the help text so it matches the latest implementation.
Reviewed By: DurhamG
Differential Revision: D28971683
fbshipit-source-id: 3f8fb7ccc42a71fdb65b87e2b99d06fb347983f3
Summary: This option allows to exclude ignored files, this way if user specifices wildcard patterns in --include, they will still not include ignored files
Reviewed By: mrkmndz
Differential Revision: D29007062
fbshipit-source-id: a8458811b4c16e11a91abdc31967b53c3cdf2ed7
Summary:
`df` doesn't exist on Windows, so this part of `rage` isn't populated.
The closest equivalent on Windows is `wmic LogicalDisk`. Use this to query the
free space and size of the disks.
Reviewed By: quark-zju
Differential Revision: D28997337
fbshipit-source-id: 08b3b74d70928f2e9801061f049359a58108f4bf
Summary: Update to latest version. This includes a patch to async-compression crate from [my PR updating it](https://github.com/Nemo157/async-compression/pull/125), I will remove once the crate is released.
Reviewed By: mitrandir77
Differential Revision: D28897019
fbshipit-source-id: 07c72f2880e7f8b85097837d084178c6625e77be
Summary:
Now that we don't publish any Python 2 packages, let's drop make local.
Once we've confirmed that nothing was using make local, we can rename make
local3 to be make local.
Reviewed By: kulshrax
Differential Revision: D28647154
fbshipit-source-id: de277887e93a6dbc0324a30f592198ef7c83f818
Summary:
More things needing fixing to be python 3 compatible. Caught when
trying to remove the Python 2 build.
Reviewed By: quark-zju
Differential Revision: D28880028
fbshipit-source-id: d162c78237f330f1f931c3581b25ead24e3ea375
Summary:
The combination of metalog and the new clone pattern of first creating
the repo causes local copy clones to fail on Windows because the initial metalog
files are held open and the copy can't overwrite them.
Let's drop the destrepo before we do the local copy.
Reviewed By: quark-zju
Differential Revision: D28880029
fbshipit-source-id: 2a4ef52675eebf16afa528e645acd927a6110cb4
Summary: This will be used for rate limiting decisions. Also, could be logged to scuba tables to get more info about clients.
Reviewed By: quark-zju
Differential Revision: D28750197
fbshipit-source-id: 83f54e38f998c9dd824ef2d3834c777a44d0ffed
Summary: Let clients connect to lfs with HTTP through unix socket so we don't have to worry about certificates presence.
Reviewed By: johansglock
Differential Revision: D28683392
fbshipit-source-id: f6228b4099ef04fe584e320cb1892e6cb513e355
Summary:
create end to end intergation for the lookup API on the client
Start prototyping of `hg cloud upload` command.
Currently, it just performs lookup for existing heads.
This way we can end to end test the new APIs.
Reviewed By: markbt
Differential Revision: D28848205
fbshipit-source-id: 730c1ed4a21c1559d5d9b54d533b0cf551c41b9c
Summary:
Files upload will be executed in 2 stages:
* check if content is already present
* upload missing files
The check api is generic, it could be used for any id type. Called 'lookup' API.
Reviewed By: markbt
Differential Revision: D28708934
fbshipit-source-id: 654c73b054790d5a4c6e76f7dac6c97091a4311f
Summary:
Previously we set this in the rpm spec, but we need to set it in make
local as well since sometimes hgbuild invokes make local directly.
Ideally we'd put this in setup.py, since make and rpmspecs go through that, but
we need this environment also set for the dulwich build, which we don't really
control the setup.py for.
Reviewed By: singhsrb
Differential Revision: D28902015
fbshipit-source-id: bfc170c3027cc43b24c6a517512a63a71f433d23
Summary:
The recent change to make run-tests work with Python 3 broke the
allow/deny list functionality because it started testing the full test name
instead of the base. This fixes that.
Reviewed By: quark-zju
Differential Revision: D28885125
fbshipit-source-id: 586a71e66e0f094b79e6a3e07e27813db6f662d3
Summary: create `uncopy` command to unmark files that were copied using `copy`.
Reviewed By: quark-zju
Differential Revision: D28821574
fbshipit-source-id: c1c15f6fb2837cec529860aba70b516ddd794f10
Summary:
Time 0.2 is current, and 0.1 is long obsolete. Unfortunately there's a
large 0.1 -> 0.2 API change, so I preserved 0.1 and updated the targets of its
users. Also unfortunate that `chrono` has `oldtime` as a default feature, which
makes it use `time-0.1`'s `Duration` type. Excluding it from the features
doesn't help because every other user is specifying it by default.
Reviewed By: dtolnay
Differential Revision: D28854148
fbshipit-source-id: 0c41ac6b998dfbdcddc85a22178aadb05e2b2f2b
Summary:
They are breaking and hgsql is not relevant (hg server repo was forked). So
let's just remove the tests.
Reviewed By: andll
Differential Revision: D28852159
fbshipit-source-id: 04a47ea489b3f190cffe7f714a9f4161847a2c86
Summary:
Fix remaining issues like encoding and `bname` vs `name` difference
(bname was deleted by a previous change, but it's not just encoding
difference from name, bname does not have " (case x)" suffix).
Differential Revision: D28852092
fbshipit-source-id: df013b284414600deb6f20a5c0883f09906bf976
Summary:
Instrument file scmstore with tracing logging. There's more we should add here, but this will be a good starting place - I've already discovered some issues from looking at the log output. (Why does drop run twice? How does it run twice?)
It'd also probably be nice to support formatting the output like https://crates.io/crates/tracing-tree, which will be a lot less cluttered by the logged fields (like `attrs` on `fetch`).
Reviewed By: DurhamG
Differential Revision: D28750954
fbshipit-source-id: 63baa602f7147d24ac3e34defa969a70a92f96a4
Summary:
Now that EdenFS is using EdenAPI more, let's let it take advantage of
EdenAPI's better batching. We alread have a batch API for files, let's copy the
pattern for trees as well. This adds the C++ bindings. The next diff consumes
this from EdenFS
This is largely just a copy of how batch blob fetching does this. But I'm a C++
noob, so feel free to tear this apart with nits.
Reviewed By: chadaustin
Differential Revision: D28426789
fbshipit-source-id: 88d359985e849018fb3c2b4ef9e52d07c96bf31a
Summary:
Now that EdenFS is using EdenAPI more, let's let it take advantage of
EdenAPI's better batching. We alread have a batch API for files, let's copy the
pattern for trees as well. This first diff just produces the Rust code. Future
diffs will add the C++ bindings then integrate it into EdenFS.
This is largely just a copy of how batch blob fetching does it.
Reviewed By: chadaustin
Differential Revision: D28426790
fbshipit-source-id: 822ef6e7b3458df5dba7a007657e85351162b9ff
Summary: Windows has an issue where subprocess don't inherit stdout/stderr correctly. util.system() has a workaround for this, so let's use that instead of subprocess when executing shell aliases. This fixes 'hg pull --rebase' which is a shell alias.
Reviewed By: kulshrax
Differential Revision: D28815381
fbshipit-source-id: 7521c17166a2b2c0e4ee872dacfd09d2d97e00ce
Summary: These are breaking buck test runs
Reviewed By: quark-zju
Differential Revision: D28802741
fbshipit-source-id: a30c7b64d72356df05676ffab87291a246033d49
Summary:
Previously it requires migrating to doublewrite first. There is no reason why
the doublewrite migration cannot be done automatically. So let's do it.
Reviewed By: DurhamG
Differential Revision: D28757734
fbshipit-source-id: ba2533b5506309610b87865a838d7efe22bccfac
Summary:
Add the `fetch_contentsha256` python method to `filescmstore`, which accepts a list of keys and returns a list of (key, sha256).
This is intended to be used by the modified `status` command implementation, which will prefer comparing content hashes to directly comparing file content.
Reviewed By: DurhamG
Differential Revision: D28696618
fbshipit-source-id: a0304319b0a19d4f09d07bec02dc41964aec7255
Summary:
Merge `found_file` and `found_aux_indexedlog` into a new `found_attributes` method, which simply "or"s the newly found attributes into the `found` map.
Replaces the `satisfies` concept with a new `pending` check, used the same way by each `pending_*` method, which considers a key pending if fetching from a store which returns a given set of attributes would allow us to resolve any requested by missing attributes, optionally taking into account attributes that can be computed from those already found. This will still need to be adjusted to support preferring remote fetching of attributes to local computation, but it is no longer as brittle as the previous implementation: there's no requirement that aux data be computed as content is fetched in order to avoid redundantly fetching content.
Move attribute computation to a separate phase, and filter out un-requested attributes in the `finish` function.
Reviewed By: DurhamG
Differential Revision: D28694192
fbshipit-source-id: 9b096c056736cadc0f97ff09243ed09d5266504d
Summary: Use associated constants instead of methods for `FileAttributes` bit masks.
Reviewed By: DurhamG
Differential Revision: D28724729
fbshipit-source-id: 441c0d2361166824c4ee7cfd5ad0b6f21ee1ac26
Summary:
Previously, the `found_error` required `&mut self`, even though it only ever interacted with the error fields. This prevents Rust's type checker from validating the safety of logging errors while iterating over the `found` map, for instance.
Replacing the `&mut self` method call with a field access into an existing `&mut self` resolves this problem, and allows logging errors while mutating other fetch state.
Reviewed By: DurhamG
Differential Revision: D28722547
fbshipit-source-id: 59c6a530cbf331282d6f654a56e492d47cafcd2f
Summary:
Don't try to fetch from a store if we don't have any pending keys.
Handle missing content when writing to cache after fetching from remote stores. Currently, `found_in_*` will be populated even if we don't store the content, having just used it for aux data computation. This change won't be necessary, but won't cause any problems either, after the next change which only prunes overfetching in the `finish` method, allowing remote blobs to be written to local cache even if we only fetched them to compute their attributes. I might revert this portion of the change, or warn if content is unexpectedly unavailable.
Reviewed By: DurhamG
Differential Revision: D28694964
fbshipit-source-id: 465211c9257cbf49b1cb68856473323fc940f10b
Summary: Extends the previous change to add support for computing aux data (currently only Content Sha256) and caching it locally. Introduces a `FetchState` config option, `compute_aux_data`, which controls if content will be fetched in order to compute aux data, or if unavailable aux data will be treated as "not found".
Reviewed By: DurhamG
Differential Revision: D28528456
fbshipit-source-id: 26189d18c8e453040f3c1f6e22a34d623a5aa40d
Summary:
The migration to Python 3 broke the unified diff code because difflib
expected the paths to also be bytes.
Reviewed By: quark-zju
Differential Revision: D28758876
fbshipit-source-id: 367ef237594d2908377cd8b81def364b77ee02e2
Summary:
`rm -A` means removing files that are "deleted" (`rm`-ed but not `hg rm`-ed).
It does not need to list clean files. Listing clean files can be very slow
in a large repo.
Avoid listing clean files so `rm -A` can be faster.
This has a side effect that we no longer maintain the exit value (0: repo
becomes empty, 1: repo is not empty) like before. But I guess nobody really
cares the 1 exit value (and it does not really make sense in the `rm -A`
case).
Reviewed By: DurhamG
Differential Revision: D28622558
fbshipit-source-id: 2087d6508932905564a8307e9438895538ecede9
Summary:
The usage of bytes for paths and environment variables makes this entire file hacky and makes it not work on Windows. Let's remove all of that.
We still use bytes for test output and other file content type cases.
Reviewed By: andll
Differential Revision: D28227825
fbshipit-source-id: b15993601db501160c9fa4eb2463678cde1fa554
Summary:
Previously, migrating to lazy means repo requirement changes. This diff uses
the new API to actually make the changelog lazy.
Reviewed By: DurhamG
Differential Revision: D28700896
fbshipit-source-id: 82cfd70645230cd67223195e25ef07ae5abe7df6
Summary:
Switch debugrebuildchangelog from using revlog stream clone to lazy segment clone.
This removes the revlog techdebt and can be used as a way to repair
repos with broken segmented changelog. As we migrate off double-write backend we
can no longer migrate down to revlog then migrate up, and a full reclone can be
slow. So a partial reclone command that just recreates the segmented changelog
seems useful.
This command is one of the two commands that handle emergency situations
when segmented changelog related logic goes wrong. The other command
is the emergency clone mode, added by D27897892 (d1413bbbad), which assumes everything
related to segmented changelog is broken on Mononoke side and we still
need to commit and push. This command relies on segmented changelog
related features, such as hash<->location lookup, and clone on Mononoke
to work properly and the server having a compatible IdMap. So it might
not be able to address all issues if Mononoke goes wrong.
Reviewed By: DurhamG
Differential Revision: D28430885
fbshipit-source-id: 17357a33f6fda4a67d46e2c7e7be6653b530f499
Summary:
Use the interruptible block_on API so the Python methods can be interrupted by Ctrl+C.
This is especially useful if some operation triggers lots of expensive network fetches.
Reviewed By: DurhamG
Differential Revision: D28723008
fbshipit-source-id: b6c692de6290a49852eabcd960ebd9b2fb68685a
Summary:
This will be used by the next change to test migrating from a non-lazy
changelog to a lazy changelog actually makes commits lazy.
More commits were added to the graph to test laziness. The old graph
does not have commits that will be made lazy by the current standard
(parents of merges are not lazy).
Reviewed By: DurhamG
Differential Revision: D28700897
fbshipit-source-id: 527c3ce672327ed5e2398c0d89a8e9e92e5b244f
Summary:
This will be used by the next change to migrate from a non-lazy changelog to a
lazy changelog.
Reviewed By: DurhamG
Differential Revision: D28700898
fbshipit-source-id: ff12754f224586b9d0d62f73b41bbb07fc7a6eea
Summary:
If a patch declared the length of it's last hunk as N lines, but it
only contained N-1 lines, the Rust code would enter into an infinite loop. This
could happen if a text editor remove the trailing spaces from a patch file.
Let's fix it and add a test
Reviewed By: kulshrax
Differential Revision: D28683977
fbshipit-source-id: 0a999ae108676531a2cf18e77a3b426ba4647164
Summary: Sometimes things take longer, make sure we are able to distinguish whether that's due to networking, tls handshake, http parsing, or Mononoke wireproto handling.
Reviewed By: markbt
Differential Revision: D28705508
fbshipit-source-id: 1bafda7fc447f2e429690f47fe7ab81cec511494
Summary:
Extends the `FileScmStoreBuilder` to construct two new indexedlog stores for caching aux data. The stores will be created in a directory adjacent to the normal non-LFS indexedlog stores.
Currently aux data stores will not be constructed for production users, a configuration option will be introduced to gate this before `.store_aux_data()` is called in the `filescmstore` constructor bindings.
Reviewed By: DurhamG
Differential Revision: D28689693
fbshipit-source-id: e3ad1594e5beee00b1a8b9fe489e3b6af3a2e93e
Summary:
Modify `FileStore` to introduce basic aux data fetching. Aux data is currently read from a separate IndexedLog store, serialized with `serde_json` (chosen for expediency / ease of debugging, I intend to optimize the storage format before releasing this, at the very least to avoid unnecessarily serializing the key path).
Currently aux data fetching will never succeed, as aux data fetching is not supported in the EdenApi "files" API and nothing else exists to populate the local aux data stores. Later in this stack, computing aux data (currently only content sha256) to populate the aux data storage is implemented.
Reviewed By: DurhamG
Differential Revision: D28526788
fbshipit-source-id: c8e21a1377689d7913a68426a3a480d53148da66
Summary:
Simplify tracking of incomplete fetches in preparation for attributes support in the next change.
Now, all keys which have not been completely and successfully fetched are recorded in `pending`, and are removed only when the complete fetch is recorded in `found`. Keys are now removed from `lfs_pointers` and `pointer_origin` as they are completed, as they aren't needed for anything other than fetching from local LFS and remote LFS respectively.
Reviewed By: DurhamG
Differential Revision: D28546515
fbshipit-source-id: c657e5c6350cadc8da970f57bb7694ed71022efb
Summary:
Now metalog can no longer be `None`. Let's just remove logic handling its
`None` case.
This changes the commitcloud-sync-race test because the metalog itself has
internal locking and changes are atomic.
Reviewed By: DurhamG
Differential Revision: D28595292
fbshipit-source-id: bd9851f5f3bb25f28f15d673f608af2863953c46
Summary:
fncache and store have been default on for years. Enable them unconditionally.
This also makes sure that metalog is always available.
Practically, the only place that does not use fncache is hgsql server repos and
they are irrelevant now.
Reviewed By: DurhamG
Differential Revision: D28595289
fbshipit-source-id: 32b9906c179518acdb17a206b54f98a3dc994921
Summary: I have modified the places where most of the errors were raised that users reported and were resolved by renewal of certificates.
Reviewed By: krallin
Differential Revision: D28568561
fbshipit-source-id: 44fb127a49bde83efee1c934e0435b31f8602a8d
Summary: Upcoming changes will force enable metalog so there will be no way to migrate down.
Reviewed By: DurhamG
Differential Revision: D28595290
fbshipit-source-id: a130b3c60c5b553d024868f28a28e48c50d44783
Summary:
It was added by D8527475 (72c3d8afc1) to workaround hgsql with no-fncache and long file
names synced from svn. Upcoming changes will force fncache to simplify
configuration and the hgsql server code was forked. So let's just delete
the test.
Reviewed By: DurhamG
Differential Revision: D28595291
fbshipit-source-id: 60d2449cca7af46b8b5b3c3b557a36507ff1576e
Summary: This will be used by fbclone to ship lazy commit hash backend.
Reviewed By: DurhamG
Differential Revision: D28554445
fbshipit-source-id: a263ae7683124b3b86f4025b02c7de20dcb9813e
Summary: This makes it possible to use non-debugshell to compact the metalog.
Reviewed By: DurhamG
Differential Revision: D28550902
fbshipit-source-id: 789830ba35243d248397e6a52ee343584c1e01a9
Summary:
The "compact" API rebuilds the metalog by removing older history. It could be
useful to reduce the size overhead of the metalog.
This is also useful if we're doing other "rebuild" work, such as rebuilding the
changelog.
Reviewed By: DurhamG
Differential Revision: D28550903
fbshipit-source-id: 56f875bd955247181236a976dcce6163d126a4b6
Summary:
The zipimport logic requires the pyc mtime to match its source. However, the
Windows system time zone can invalidate it and cause slow startups.
Workaround it by making the zipimport mtime function return a fallback value so
the mtime check is then bypassed.
# zipimport.py, _unmarshal_code
source_mtime, source_size = \
_get_mtime_and_size_of_source(self, fullpath)
if source_mtime: # if source_mtime is false, then the check is bypassed.
# We don't use _bootstrap_external._validate_timestamp_pyc
# to allow for a more lenient timestamp check.
if (not _eq_mtime(_unpack_uint32(data[8:12]), source_mtime) or
_unpack_uint32(data[12:16]) != source_size):
_bootstrap._verbose_message(
f'bytecode is stale for {fullname!r}')
return None
Change my Windows time zone from GMT-7 to GMT-4. Set PYTHONVERBOSE and
PYTHONDEBUG to 1. Ran `hg init -h` and check its stderr. It prints:
# bytecode is stale for 'edenscm.traceimport'
and alike before this change, and no longer after replacing the `__init__.py`
in the zip with the new version.
Reviewed By: DurhamG
Differential Revision: D28622287
fbshipit-source-id: bb3e8e378ea168e4f83f4b6aa9713103b2c90ef8
Summary:
don't apply an old public bookmark if the commit is older than max_sync_age
there is a complicated logic because we need to make sure if we later run it with different commitcloud.max_sync_age value or hg cloud sync --full the bookmarks will be appear back.
So, the changes required in both:
* checkomission
* _mergebookmarks
but both cases covered in the tests
also, if you run with max_sync_age=1000 and later max_sync_age=0, the bookmarks will not disappear, which is expected.
Reviewed By: markbt
Differential Revision: D28572875
fbshipit-source-id: 317e897a2b81c3371dbea7eb39b8925570c1d40a
Summary:
this output is not noisy for big workspaces
if a head is omitted, don't warn about the bookmark because it is expected
Reviewed By: markbt
Differential Revision: D28568919
fbshipit-source-id: eb19e1d155f65de411c1dd41a8be6d83ca71c264
Summary:
The output is a bit too noisy for large workspaces.
We could skip older commits, we know the list coming ordered from commit cloud service.
All hashes are anyway available via `hg cloud sl`.
Also I fixed several look-ups in the list. Omittedheads are heavily used to check if something is present there.
Reviewed By: markbt
Differential Revision: D28568421
fbshipit-source-id: bcf62522798fed92df7ca546c73aa14da95f1567
