Summary:
This particular situation happens in the wild when the amend doesn't rebase
because of conflicts and users work on their stack using `hg prev` and `hg next
--rebase`. In this case when there's non-obsolete child that's always the child
we want to choose.
We're verbose about what we're doing so it's not confusing to the users.
Reviewed By: quark-zju
Differential Revision: D14560584
fbshipit-source-id: a453c0301a5156eea0d19ceb40d9a64e80b7fca7
Summary:
Move the logic of adding the common ancestor to make the DAG connected to the
smartlog revset. This makes it handy for power users to just use `log` and
`smartlog` revset to get interesting graphs. `sl` is now a very thin wrapper
around the `smartlog` revset function.
Reviewed By: DurhamG
Differential Revision: D14461520
fbshipit-source-id: 78e3991059c9da7ef4410e252a2b69b1e54918cb
Summary:
Wrap user-provided revs with `smartlog` revset function. This makes more sense
together with the next change.
The test change is because "parents" of drafts are selected.
Reviewed By: DurhamG
Differential Revision: D14461519
fbshipit-source-id: 2a48931680f0dc50b80b87cea827152c21cf4791
Summary:
With the last change, the benefit of ancestorcache is limited. Therefore just
remove it to reduce complexity. This also makes `smartlog` closer to `log`.
Reviewed By: DurhamG
Differential Revision: D14461523
fbshipit-source-id: eb108a09e12b07e5012f70aef0b2940b07d746fb
Summary:
Use the `ancestor` revset to replace the adhoc ancestor calcuation. This makes
the code much shorter.
It's in theory slightly different from the old logic. But there are no test changes.
The new code can no longer take advantage of ancestorcache. Fortunately, with
optimizations, it is pretty close to a fully warmed up ancestorcache. Of course,
it's much faster than a cold ancestorcache.
Before (ancestorcache disabled):
quark@devvm33994 ~/fbcode/scm/hg % ./hg sl -T '.' --time --pager=off --all --config smartlog.useancestorcache=0 >/dev/null
time: real 75.050 secs (user 52.540+0.000 sys 22.520+0.000)
Before (ancestorcache warmed up):
quark@devvm33994 ~/fbcode/scm/hg % ./hg sl -T '.' --time --pager=off --all --config smartlog.useancestorcache=1 >/dev/null
time: real 2.670 secs (user 2.550+0.000 sys 0.100+0.000)
After:
quark@devvm33994 ~/fbcode/scm/hg % ./hg sl -T '.' --time --pager=off --all --config smartlog.useancestorcache=0 >/dev/null
time: real 2.970 secs (user 2.760+0.000 sys 0.160+0.000)
There are 5110 commits in the above smartlog graph.
Reviewed By: DurhamG
Differential Revision: D14461524
fbshipit-source-id: 68bee3c4397be833e381c582c20a849b768b144d
Summary:
Previously, the default master is `.^` when `--rev` is passed. Change it to
null so we're not adding unexpected "master" if `--rev` is used.
Reviewed By: DurhamG, sfilipco
Differential Revision: D14516266
fbshipit-source-id: ce93f5e905d674c21cc07bb5a2957d0fad302722
Summary:
We encountered a bug where if a file descriptor was not flushed prior
to forking the process, then both processes would end up flushing the same data
to the file, resulting in duplicate data and a corrupt file.
The real fix is to stop using fork, but for now let's address the most visible
problem by flushing pack files before forking.
Reviewed By: kulshrax
Differential Revision: D14569005
fbshipit-source-id: e002abe72c8014cbe49ccffab6159f8372affdb0
Summary: Deprecate due to complexity of the code.
Reviewed By: mitrandir77
Differential Revision: D14561405
fbshipit-source-id: 6184317f549c0ab84335b09c4b48efccdf31f7fc
Summary: Added new metrics to log loose files size and number during repack. We need it to understand how much better the pack files work in terms of disc and memory usage.
Reviewed By: markbt
Differential Revision: D14544811
fbshipit-source-id: 5a4d894bd5a3358c7e0f93ecc9db5e9f2c2f2372
Summary:
Basically we should check that the commits have been backed up.
If this is not true and the commits are local we can just back them up.
If they are not known by this repo, pull from the old one and then back them
up.
Reviewed By: markbt
Differential Revision: D14508239
fbshipit-source-id: 3fdd83335cb937b153510ec3c7510ecd1167d0ca
Summary: Log data about round-trip count, and object count for files, trees, and SSH calls.
Differential Revision: D14515776
fbshipit-source-id: cce416fd7dccdad3c73a9f1751a04ddac0d2c507
Summary:
Make it possible to use `ui.metrics.gauge` to collect metrics for a single
command, via the sampling extension.
Differential Revision: D14515775
fbshipit-source-id: e8a53549b00c1bc7b6509a5990a51d955d767d7e
Summary: This makes sure `ui.log` always runs through the sampling code path.
Differential Revision: D14515774
fbshipit-source-id: 585cd14eaecda12a9c2dd6ed003f0a457d67daf1
Summary:
Before this patch, metrics was designed to send stats to a global counter. I'd
like to use it for stats local to the current command (ex. count of file
fetching, count of round-trips, etc).
Change the API so "entity=None" forbids stats from sending to a global counter.
Differential Revision: D14515779
fbshipit-source-id: b5b3b040d674c71f467153c308b56aa6f506eb0c
Summary:
Disable parsing `.hgignore` and related fileset `hgignore()` by default. They can
still be enabled via configuration. The plan is to completely remove them
later.
A replacement for `hgignore()` fileset was added as `gitignore()`.
The `hgignore()` fileset seems to be only used by zertosh in the past 3 months.
Reviewed By: singhsrb
Differential Revision: D14543232
fbshipit-source-id: f2385062a0e816331f693239f62448979876078a
Summary: This simplifies testing setup for all crates in CI.
Reviewed By: quark-zju
Differential Revision: D14543989
fbshipit-source-id: 83693fada6e64b7c21fd89a880d6452d811ea90d
Summary:
Part way through our hg-git repo's history we started adding the git
hash to the extras so it was easier to generate the map files. Let's make
git-updatemeta stop when it passes this boundary.
This is useful for speeding up internal infrastructure that needs to generate
the mapfile from scratch.
Reviewed By: kulshrax
Differential Revision: D14542354
fbshipit-source-id: 7c17fb1b1439f9b4c0c0acf8b5a85790d02a0861
Summary:
git-updatemeta performance starts to matter when we have to run
it from scratch in new containers. Let's optimize this a bit. In a large repo
this improves the speed by 30%.
Reviewed By: quark-zju
Differential Revision: D14542339
fbshipit-source-id: 34f06369543b8d4d22838fd4e3878c6bec9a597c
Summary:
Improve the performance of the revsets that calculate which commits to back up
by limiting them to consider only the non-obsolete commits that are also draft.
Reviewed By: quark-zju
Differential Revision: D14544883
fbshipit-source-id: db9ed4a7abd81956762f56140321242dbccf2df0
Summary:
Not every command requires a valid repo, but when one is used, it is always
properly closed. Hence, let's simply wrap the repo.close method instead of
wrapping around the runcommand function.
Reviewed By: kulshrax
Differential Revision: D14531515
fbshipit-source-id: bcdbe6530c94041c1185b18570846ba609b5f605
Summary:
Attempt to detect oscillation of commit cloud workspaces by comparing the new
state after the current sync with the state before the previous application of
cloud changes. If the version number is incremented by 1, the workspace is
brought back to the same state, and less than a minute has passed since the
last time that commit cloud sync ran, abort the current sync step.
This happens after the commits have been backed up, but before the new cloud
workspace is synced to the commit cloud service. This prevents further
oscillation whilst ensuring the user's commits are still backed up.
Reviewed By: quark-zju
Differential Revision: D14540355
fbshipit-source-id: 20e4b0333f5a7e34b512967a03099625f62ff9d5
Summary:
Change how infinitepush determines what to back up.
Commits to back up are all draft ancestors of non-obsolete commits, *plus* all
draft ancestors of bookmarked commits, which may be obsolete.
Previously, in a stack of the form:
```
| x B (obsolete, bookmarked)
| |
| o A
|/
o
```
neither A or B would be backed up, despite normally being visible to the user.
Reviewed By: liubov-dmitrieva
Differential Revision: D14540356
fbshipit-source-id: 0d6ad330c53c818b08f736a9af64704cf0be7cd5
Summary:
Change how commit cloud determines what to sync and what has been successfully
synced when some commits fail to push.
Commits to sync are all draft ancestors of non-obsolete commits, *plus* all
draft ancestors of bookmarked commits.
Commits to sync when only some commits have successfully been pushed are
ancestors of the newly pushed heads, *plus* ancestors of the commits to be
synced that were successfully synced last time.
Reviewed By: liubov-dmitrieva
Differential Revision: D14540357
fbshipit-source-id: c082a2f2822f8bce4cd2bbac93a70e27e2ffaa59
Summary:
Now at the beginning of pushbackup we always check what the server already
have, this triggered changes in the output of the tests.
Before the limit of the check was 3 heads. I made it to 0 by default earlier and broke tests. Now we have fast way to check on the server side, so the limit 3 was not good. There is no reason to re-back up those.
Basically before we could backup the same thing again.
Reviewed By: markbt
Differential Revision: D14539582
fbshipit-source-id: 569b354580128c944a95fa64c0c964304a2cca8b
Summary:
SIGPIPE is a weird signal. It is sent to a process that tries to write to a
pipe (or socket) whose other end has been closed. On top of the signal being
sent, the write will also fails with EPIPE, and thus SIGPIPE is sometimes used
to avoid checking the return status of write (!!!) and to gracefully exit.
In Mercurial, the SIGPIPE is used to raise an exception to do exactly this, but
it also appears that the writes to stdout/stderr are already checking the
return value and doing the right thing, hence SIGPIPE is unecessary.
In the case of Mercurial, catching SIGPIPE has another interesting behavior.
The command `hg log -p` has a different behavior once the pager is killed
depending on whether HGPLAIN is set or not. When unset, `stderr` is also
redirected to the pipe, and thus any logs printed after the pipe is closed will
force Mercurial to quit. While this is the expected behavior in most cases,
it is not when the logs are happening during cleanup, as they will effectively
abort the teardown process.
Overall, it's better to just not install a SIGPIPE handler, so let's remove it.
Reviewed By: kulshrax
Differential Revision: D14528603
fbshipit-source-id: f51fb13016cc7b8d622e91c60d4c5286c7b404e5
Summary:
The check was added to "distrust" st_nlink, to workaround issues on CIFS [1].
The check only works for filesystem supporting hardlinks.
If a filesystem is marked as "st_nlink cannot be trusted", then vfs.py will take
a conservative approach to break hardlinks when writing, by copying the file
followed by a rename.
Therefore, filesystems without hardlink support (esp. edenfs) would go through
the slow path - every file written via vfs.py would be copied unnecessarily.
Fix it by skipping the check for known filesystems, including edenfs.
[1]: https://bz.mercurial-scm.org/1866
Reviewed By: DurhamG
Differential Revision: D14527370
fbshipit-source-id: 579111f5c72fa3d016f98c248b29c96afdc59086
Summary:
the test have been broken
```
buck test mode/dev scm/hg/fb/tests:fb_run_tests -- 'test_sl_output_t \(scm\.hg\.fb\.tests\.unittestify\.hgtests\)'
```
In the test the remote is not defined, so the test failed to check with the
server what has been backed up
Reviewed By: ikostia
Differential Revision: D14521930
fbshipit-source-id: 3363bb3055941decdfc65165860e1ef25a7a7891
Summary:
This is particulary useful for `hg cloud sync` when it calls pushbackup in the
background to the secondary storage at the end of cloud sync.
Pushbackup is not smart enough, so it will back up again what we just pulled from cloud sync.
However, all those commits are probably already backed up on the secondary
storage on another machine while cloud sync.
Reviewed By: singhsrb
Differential Revision: D14386616
fbshipit-source-id: e62ed0afb89c28fe6880346077c279e6705da602
Summary:
Using lookup command is slow because if something is not backed up, it throws
exception and we have to re-establish ssh connection
Differential Revision: D14386150
fbshipit-source-id: 8d5caea93516571ff36c80adb6406b0347d90384
Summary:
Reading a line over a pipe involves reading every character of the line
individually. This is extremely inefficient and slow, which cause prefetch to
be overly slow when most of the data isn't in memcache.
Using buffered reads tries to read 4096 bytes at a time, drastically reducing
the cost of reading a missing path/node pair from memcache.
Reviewed By: ikostia
Differential Revision: D14507063
fbshipit-source-id: e0910d7a303e15fe2d3c61fe2739e6c13370058f
Summary:
As part of the mononoke write path rollout we want to be
able to dynamically block writes to a repo. This is implemented as
a table in hgsql (repo_lock) that both hg and mononoke can read, but
will only be updated by scmadmin.
Expose a function that can be used by in-process hooks to check if a repo is
locked for mercurial.
Reviewed By: quark-zju
Differential Revision: D14279169
fbshipit-source-id: f8bb4afeeeda67796cf806ab7f3fe42f4089818f
Summary: This is removal of unused code that left from D14455360
Reviewed By: ikostia
Differential Revision: D14502837
fbshipit-source-id: df1912c7997847b0628b492b3fe735d5e3d7f201
Summary:
On connections that might experience higher latency the throughput for a clone
is easy to spike, raising the chunk size results in more consistent throughput.
Given that hardware these days has plenty of memory, this change shouldn't make
any significant difference either.
Reviewed By: farnz
Differential Revision: D14502443
fbshipit-source-id: 0e90f955304e9955df0ec69b5733db7c34b09e83
Summary:
Change `ancestor` to send 24 revs to the native ancestor logic at a time,
instead of 2 revs at a time.
This makes it much faster when calculating ancestor of many commits.
Before:
quark@devvm33994 ~/fbcode/scm/hg % hg.real log -r 'ancestor(limit(draft(),6000))' -T '{node}\n' --time --hidden
ca4beab06459e016ac1b4041400029165b701afa
time: real 66.620 secs (user 44.800+0.010 sys 21.840+0.000)
After:
quark@devvm33994 ~/fbcode/scm/hg % ./hg log -r 'ancestor(limit(draft(),6000))' -T '{node}\n' --time --hidden
ca4beab06459e016ac1b4041400029165b701afa
time: real 3.470 secs (user 2.380+0.000 sys 1.000+0.020)
This also affects `merge.preferancestor`. That's a dangerous config option and
seems fine to be ignored in the revset implementation.
Reviewed By: DurhamG
Differential Revision: D14461521
fbshipit-source-id: ec00921aece0adc6aaca49e5580bff52784c4ca5
Summary:
Use "interestingmaster()" to make it easier to see how "master" gets decided.
Change the type of "master" argument taken by "smartlog" revset from string to
revset. This is more consistent with other revset functions.
Reviewed By: DurhamG
Differential Revision: D14436003
fbshipit-source-id: 5aa166b523f36672f77dc4f161ae8d64c2b50579
Summary:
Similar to `interestingbookmarks()`, this exposes more smartlog logic to the
user.
Reviewed By: DurhamG
Differential Revision: D14436004
fbshipit-source-id: bd4ef1dcee8e7b29c43ce43fe6c1a3e7b5286774
Summary:
Make `heads` in `smartlog` customizable. This makes smartlog more flexible.
Instead of using the default selection, the user can choose draft branches, and
potentially pass in `interestingbookmarks()` to include bookmarks and remote
bookmarks. For example, `smartlog(.::)` shows the current branch and the public
commit it bases on.
Drop `recentdays` as it can now be expressed using `heads` directly. See the
test change.
This would also hopefully make test-fb-hgext-smartlog-hide-before.t stable,
as it no longer uses `time.time()`.
Reviewed By: DurhamG, sfilipco
Differential Revision: D14436007
fbshipit-source-id: 5e0a76e4424b01312fef02fae23a3abd74e863c6
Summary:
The old code basically selects ancestors of heads.
Rewrite the logic using revsets. Assuming we're only interested in ancestors
that are drafts, we can take advantage of `draft() & ::x` optimization.
The new logic also assumes master rev is public. Otherwise it can be slightly
different from the old logic.
The new code is much faster on my repo:
New code:
quark@devvm33994 ~/fbcode/scm/hg % ./hg log -r 'smartlog()' --hidden -T . --time | wc -c
time: real 0.630 secs (user 0.550+0.000 sys 0.030+0.000)
6716
Old code:
quark@devvm33994 ~/fbcode/scm/hg % hg.real log -r 'smartlog()' --hidden -T . --time | wc -c
time: real 5.470 secs (user 3.920+0.000 sys 1.550+0.000)
6716
This might make the ancestorcache hack (D5135746) unnecessary.
Reviewed By: DurhamG, sfilipco
Differential Revision: D14436008
fbshipit-source-id: 3c3bf47ccb67ea0e238542995009da9b9250b43b
Summary:
The `smartlog()` revset does a lot of things.
Add a new revset `interestingbookmarks()` to expose part of the smartlog features.
Reviewed By: DurhamG, sfilipco
Differential Revision: D14436006
fbshipit-source-id: 15b8d203b6547e63f8d062660ad27bdbc25b2c1c
Summary:
The code falls back to head() if there are no remotenames. We don't need that
behavior. Therefore just simplify it by always using `heads(draft())`.
Reviewed By: DurhamG
Differential Revision: D14436009
fbshipit-source-id: 25c2d245ed64a29e3e1677ededb4c2ba7b4a3ceb
Summary: D14387734 added 2 new arguments to the `httprequestpacks` function, but didn't update the callsite to pass those arguments. This diff fixes the problem.
Differential Revision: D14468592
fbshipit-source-id: 7e573838916067fc2cc12204ea1da460eb3955c8
Summary:
Currently archive is almost useless because it fetches each file
one-by-one. Let's add prefetching.
Differential Revision: D14460880
fbshipit-source-id: 1f06e1ac9d03aae3ab27d3064f9fe6141051be06
Summary:
In preparation to support Mononoke clean up the features that are Mercurial
specific and Mercurial infinitepush implementation specific.
For Mononoke migration we will to write a whole new set of logic what to do if
the "infinitepush" path has been changed. So, clean up is useful before
writing this logic.
Reviewed By: singhsrb
Differential Revision: D14455360
fbshipit-source-id: d15c3a9032b4888a1aa391da34ad5e499aba9a15
Summary: See the linked task for details.
Reviewed By: quark-zju
Differential Revision: D14448505
fbshipit-source-id: fc2efa71510b718c25a2cea3acf39663d280f19a
Summary:
After going over the code review for D14332967, I have decided to keep
things simple for now and only allow making commits to same target parent as
the original parent. This was already the intention with the existing code.
Therefore, this commit just further enforces the requirement.
Reviewed By: quark-zju
Differential Revision: D14422351
fbshipit-source-id: 2f786fc3596b17c5020de9906adf8f22b50be4dd
Summary:
These was probably introduced by moving to black.
The changes in the diff were generated by script.
Reviewed By: mitrandir77, singhsrb
Differential Revision: D14439667
fbshipit-source-id: 54f6e0bdcc59c1c6deb4eea46dc6f865bcd48cf8
Summary:
The code currently assumes that the target parent is the same as the
original parent by totally ignoring the original parent which can seem
surprising and more importantly, hinders supporting behaviors such as allowing
commits to a new parent. Therefore, this commit fixes the code to identify the
original and target parent separately.
Reviewed By: quark-zju
Differential Revision: D14422352
fbshipit-source-id: bc175f2fe636f9bf47a68f64c8efd52660e3b1b7
Summary:
D14183009 made commit cloud accept cloud bookmarks for hidden commits, rather
that omitting them. However, this only works for future bookmarks. If the
bookmark was already omitted, then `_checkomissions` would not recover the
situation for the same reason.
Update `_checkomissions` to also allow cloud bookmarks on hidden commits.
Reviewed By: liubov-dmitrieva
Differential Revision: D14437656
fbshipit-source-id: 2372323022a59bfd4210bc76f39b9a74872d5efe
Summary:
Now that hg_memcache_client will voluntarily exit when hg terminates, we no
longer need to wait for hg_memcache_client to finish before exiting hg.
Reviewed By: DurhamG
Differential Revision: D14396510
fbshipit-source-id: 7e73d9b70d481e58a0c47cd0f408580e6d548fd9
Summary:
This is useful for testing the `memcommit` command as the clients can
specify different destination parents than the original parents of the commit.
Differential Revision: D14410213
fbshipit-source-id: 846e0d764b9606f00aed95995c694f379457eec7
Summary:
Earlier message suggested the feature is not supported when in fact it
is allowed via a configuration.
Differential Revision: D14410214
fbshipit-source-id: 0ec2a22920417c378cf3ac596565f9d2fa5f6d5c
Summary:
Operations like `hg log -p` will inherently cause many requests to
hg_memcache_client. This will force many small packfiles to be created which
will significantly slow down future invocation of hg.
Now that `hg repack --incremental --packsonly` is fast, we can afford to run it
when mercurial exit after a prefetch operation was run.
Differential Revision: D14387735
fbshipit-source-id: 45f89f1120458c8b2471a1c55cafb6bc87263dd0
Summary:
When using packfiles, history and data are in different files, and thus it's
possible to only fetch one.
For now, besides the requests coming from contentstore and metadatastore, both
will be fetched, as the code hasn't been audited to know whether we only want
history or data.
Differential Revision: D14387734
fbshipit-source-id: 6aafd477ff486b9316458ce0e80636152db45b89
Summary:
For pushbackup it is needed to make hg rage more informative because we store
states for different paths separately anyway.
For cloud sync we will have to write some migration logic: if the remotepath
has been changed, we have to check what the server has to make sure everything
is backed up as cloud sync would expect.
Differential Revision: D14420713
fbshipit-source-id: 2046e9d7b16291a49d1bc40da5285de58017f4f2
Summary:
Corrupted packfiles, or background removal of them could cause repack to fail,
let's simply ignore these transient errors and continue repacking.
Reviewed By: DurhamG
Differential Revision: D14373901
fbshipit-source-id: afe88e89a3bd0d010459975abecb2fef7f8dff6f
Summary:
Fixes the below type error when pulling from a git repo using hg-git on my laptop:
File "edenscm/hgext/remotenames.py", line 225, in expull
pullremotenames(repo, remote, bookmarks)
File "edenscm/hgext/remotenames.py", line 314, in pullremotenames
path = activepath(repo.ui, remote)
File "edenscm/hgext/remotenames.py", line 1464, in activepath
rpath = _normalizeremote(remote.url)
File "edenscm/hgext/remotenames.py", line 1439, in _normalizeremote
u = util.url(remote)
File "edenscm/hgext/hggit/__init__.py", line 164, in _url
if not (path.startswith(pycompat.ossep) and ":" in path):
AttributeError: 'function' object has no attribute 'startswith'
Basically, `peer.url()` is the API, `peer._url` is a private field that does
not always exist somehow.
Besides, further remove named branches that can crash hg-git with
NotImplementedError:
File "edenscm/hgext/remotenames.py", line 225, in expull
pullremotenames(repo, remote, bookmarks)
File "edenscm/hgext/remotenames.py", line 322, in pullremotenames
for branch, nodes in remote.branchmap().iteritems():
File "edenscm/hgext/hggit/gitrepo.py", line 73, in branchmap
raise NotImplementedError
Reviewed By: DurhamG
Differential Revision: D14144462
fbshipit-source-id: 2e886c639cf6689480f84626eaf0d5ec25512ea0
Summary:
Since state files have been changed, now it is one per path, we should dump
them in a correct way
Reviewed By: markbt
Differential Revision: D14406311
fbshipit-source-id: 8d74a51e63028ec81bcf5e55ad117d3c960b4651
Summary:
To make "draft()" bounded and train users to hide unused commits manually,
change smartlog to "hide commits before <a static date>", instead of
"hide commits that are older than <a static time span>". Then we can
incrementally decrease the static date, and eventually show everything, and
force the user to think about what commits to keep or hide.
Reviewed By: singhsrb
Differential Revision: D13993584
fbshipit-source-id: 1a2b56f50d7f014a589f798cd2feaa6931e64fe3
Summary:
Subrepo is another unloved features that we don't want to support.
Aggressively remove it everywhere, instead of just turning off configs.
I didn't spend much time to split this commit so it's smaller and more friendly
to review. But it seems tests are passing.
Reviewed By: sfilipco
Differential Revision: D14220099
fbshipit-source-id: adc512a047d99cd4bafd0362e3e9b24e71defe13
Summary:
If an infinitepush bundle contains flat manifests and is served from a
treemanifest repository, it can potentially fail to send all the needed data to
the client.
Understanding the bug requires two bits of context:
1. When sending commits from a tree server to a tree client, we generally don't
send any trees because they can be fetched by the client ondemand later. The one
exception to this is for infinitepush bundles, where the trees inside the bundle
cannot be served ondemand, and therefore must be served at pull time. To do this
we check if a given manifest node exists in the repositories permanent storage,
and if it doesn't, we assume it came from an infinitepush bundle and serve it
with the pull.
2. When we lookup a manifest and fail to find a tree, our last resort is the
ondemandstore which knows how to convert a flat manifest into a tree manifest.
On the server, this is responsible for converting each of the flat bundle's
manifests into treemanifests before we serve the bundle to the client. As part
of converting the flat manifests into treemanifests, it writes the new tree
data into a pack file.
The bug is then, when serving a stack of commits, if we try to package up the
top tree first (i.e. the most recent tree), we end up converting the entire
stack from flat into trees, which inserts the bottom most trees into the
temporary pack file. Because they exist in the temporary pack file, when we
later check if they are part of the repositories store we end up finding them,
which causes us to treat them as not-infinitepush-trees which means we don't
serve them to the client.
The fix is to change the infinitepush tree-serving code to not consider the
mutable packs when checking if it should send trees.
Reviewed By: mitrandir77
Differential Revision: D14403925
fbshipit-source-id: 38043dfc49df5ff9ea2fae1d3cac341c4936509c
Summary:
When calculating bundleroots (the commits that are the common ancestors for the
infinitepush bundle), we currently include all the `nullid` parents (i.e. p2 for
most commits). This bloats the size of the list, and is unnecessary.
Reviewed By: quark-zju
Differential Revision: D14385912
fbshipit-source-id: c518b8b1aa27cff8562c2358a024b8a08ced8cba
Summary: Make it possible to override dates in commits.
Reviewed By: singhsrb
Differential Revision: D13993585
fbshipit-source-id: 59a72302d7ed0cb22f4eff84c1325e167963508c
Summary:
The feature was completed by Phil in D9816270. It's handy and can probably
reduce user support burden like: https://fb.intern.facebook.com/groups/scm/permalink/2039619916087618/
Therefore let's enable it.
Reviewed By: DurhamG
Differential Revision: D14293405
fbshipit-source-id: 54e934e0bf495c090109462e4f743d427df39380
Summary:
Some dependant libraries may only care about the serialization logic. As an
example, see D14332987 where the `hgrepo.py` only needs to depend on the
serialization. Therefore, its cleaner to extract out the serialization from the
`memcommit` data.
Reviewed By: quark-zju
Differential Revision: D14388474
fbshipit-source-id: 6f049dcc596b66b9ad72905f133529bdc9092382
Summary:
The shared code can be potentially called by clients using python 3. Therefore,
let's be compatible with python 3.
Reviewed By: quark-zju
Differential Revision: D14387005
fbshipit-source-id: 2ffb359d4d2762ffcba4a26a3ae5a7b45e89572b
Summary:
Eventually we'd like "default" to be not a special name. This is one step
torwards that.
Differential Revision: D14233455
fbshipit-source-id: 739091a124bc667c607c283bf00abc66b4081d25
Summary:
Some hooks (ex. mergedriver) are checked in old release branches. Provide
"mercurial" module compatibility so they won't break.
Reviewed By: mitrandir77
Differential Revision: D14366343
fbshipit-source-id: d47cc4fd512f63e4f6cdc5d7e5ab2c4398216b2f
Summary:
Introducing new command for Mercurial only to support check of existing commits in both the repo and the infinitepush storage.
For Mononoke case, just the standard 'known' works fine.
We can not overload the standard 'known' in mercurial case, because it is used in discovery and having there infinitepushcommits breaks some commands.
Next diff is replacing isbackedupnodes with isbackedupnodes2
Reviewed By: markbt
Differential Revision: D14370603
fbshipit-source-id: 8d7b64b4d556c0f1caa7f797dba360501571daad
Summary:
If Mercurial detects `ENOTCONN` when trying to open the current directory,
there is a high likelihood this is a disconnected eden virtual checkout.
Provide some advice as to what to do: run `hg fs start`.
Reviewed By: strager
Differential Revision: D13888873
fbshipit-source-id: 7619df0681d15b862d1a6f86d90491aa873bf86b
Summary: Include the output from edenfs rage in the hg rage output.
Reviewed By: chadaustin
Differential Revision: D14007061
fbshipit-source-id: fe0baf6c19dd4f2afd470ba70304c78582dfe879
Summary: Add `hg fs` subcommands for the other main `edenfsctl` commands.
Reviewed By: chadaustin
Differential Revision: D14007062
fbshipit-source-id: 9b5f56b14b5812216c929232b2697233f38288cc
Summary:
Support updating of commit visibility when using pushrebase. Since obsmarkers
may not be available, this also involves looking at the commit mutation data
returned from the server.
Reviewed By: DurhamG
Differential Revision: D12980783
fbshipit-source-id: 837e356e500e7bf9710a3619a31094cae21d36c9
Summary:
When commits are added or modified, update the set of visible heads if
visibility tracking is enabled.
Reviewed By: DurhamG
Differential Revision: D12980779
fbshipit-source-id: 8f44045159c86a374ae530fa4327ee0807b4320d
Summary:
Add a method of tracking explicit visible heads. Rather than using the
implicit set of commits that are not obsoleted (which may differ between repos
that are connected to a single commit cloud workspace), we instead track commit
visibility explicitly.
This is more like the git model, where only commits that are reachable from
refs are considered for most operations, except that we keep track of the heads
automatically, and use obsmarkers to keep track of the obsoleted commits,
rather than garbage collecting them.
Reviewed By: DurhamG
Differential Revision: D9560361
fbshipit-source-id: 07dabfc04415f2ecb97d57c4e3944c071579ee50
Summary:
Disable the various templates that attempt to determine the fate of a
particular commit based on obsmakers when mutation is enabled. The old
templates were either insufficiently generic (e.g. `amendsuccessors`), or
leaked internal implementation (e.g. `succsandmarkers`).
With mutation enabled, callers should use the `mutations` template to get a
list of a commit's mutations.
Reviewed By: DurhamG
Differential Revision: D12980772
fbshipit-source-id: 920d47f7c61ad52f562cd90f1cb405250c14bc25
Summary:
Add support for detecting landed commits.
Since we don't currently have an index for successor information in the
changelog, we can only detect the successor relationship for predecessors of
draft commits (for which we build a cache). As a temporary workaround,
make it possible to put mutation records in the mutation store that lead to
landed commits. These will allow land detection to work until we have a
changelog that supports indexing on predecessor.
Reviewed By: quark-zju
Differential Revision: D12980780
fbshipit-source-id: d7b14fa073d0406990b92731fe66dfe1c73b404c
Summary:
The `mutations` template keyword expands to a list of the results of mutating
the commit. Each element of the list has an `operation` field, which is a
string describing the mutation operation, and a `successors` field, which is a
list of the successor commits for this operation. Sequences of mutations that
result in a single successor are collapsed into a single `rewrite` operation.
Reviewed By: quark-zju
Differential Revision: D12980787
fbshipit-source-id: 82be2f9131832209cc3ab088f587c45f8c45a9ad
Summary:
Include mutation records for all predecessors of the pushed commits in
infinitepush bundles.
When received from infinitepush, store these additional records in the local
store. This allows us to bridge any gaps in mutation that are omitted from the
local commits when they are received from infinitepush.
Reviewed By: DurhamG, quark-zju
Differential Revision: D12980777
fbshipit-source-id: b1535ca29c0fca3e6cb0f563d78c4c60d4aef58e
Summary:
The debugmutation command should also show information about mutation entries
in the mutation store.
Reviewed By: DurhamG, quark-zju
Differential Revision: D12980785
fbshipit-source-id: 06c3ec2cb9c42edf43729ba3b7c471b1bf8dfb96
Summary:
When traversing mutation entries, if we don't have any information for a
commit, then look instead in the store to see if we have a cached entry
from a non-local commit there.
There aren't any ways of putting entries there, yet. Those will come in
later commits.
Reviewed By: DurhamG
Differential Revision: D12980776
fbshipit-source-id: 63ff12382eb9294aa43ff100a4fe19b7c05f9e61
Summary:
Histedit needs to adjust its records of what commits are replaced. Currently
it does this by examining obsmarkers. If mutation is enabled, use the mutation
information instead.
Reviewed By: quark-zju
Differential Revision: D13279987
fbshipit-source-id: e9622a67635afe2023088fdf0e0b43b0bcd9223f
Summary:
We will be making looking up entries for complex by adding a secondary store of
the information. Make accesses to this information common through lookup
functions.
Reviewed By: quark-zju
Differential Revision: D13279986
fbshipit-source-id: a30084b548762e69cb354c3760d7ec66027a24e1
Summary:
Implement successorssets and foreground in terms of mutation records and
replace them when mutation metadata usage is enabled.
Reviewed By: quark-zju
Differential Revision: D10149263
fbshipit-source-id: bbf6d1fc44a9787660147e15936a9ee1951373ca
Summary:
When enabled, use mutation metadata for the `obsolete`, `extinct`, `orphan`,
`phasedivergent` and `contentdivergent` revsets.
Reviewed By: quark-zju
Differential Revision: D10149265
fbshipit-source-id: 5559fa22a6025e1d341538f3eb2257d8efee15e5
Summary:
Add a mutationcache to the repo. This computes the following information:
* The successor sets for all predecessors of visible commits - these are the
sets of immediate successors for each commit.
* A map from commits which are the results of splits to the final split commit.
* The set of public commits that have visible draft successors.
* The set of draft commits that have multiple sets of visible eventual successors.
* The set of obsolete commits - draft commits with visible eventual successors.
* The set of orphan commits - commits with obsolete ancestors.
* The set of extinct commits - obsolete commits with no orphaned descendants.
These sets will be used later on to replace the obsmarker-based equivalents.
Reviewed By: DurhamG
Differential Revision: D13279988
fbshipit-source-id: 3f063bb68aaba2f19da257efdf79b485b947b7b1
Summary:
Original commit changeset: af43d4cce555
D14338313 didn't trigger the main `.t` tests. A lot of things actually
depend on `zstd` APIs returning `bytes`. Therefore back out the change.
Reviewed By: DurhamG
Differential Revision: D14372351
fbshipit-source-id: d8daa7d1d2a49d9d0c4d48de22ed0567d1d953a7
Summary:
Unfortunately, Mononoke team needs to rename paths to add the markers everywhere.
They deprecated mononoke url:
ssh://hg.vip.facebook.com//mononoke/fbsource
And they are asking us not to use ssh://hg.vip.facebook.com//data/scm/fbsource url without markers.
We finally agreed to have:
```
infinitepush = ssh://hg.vip.facebook.com//data/scm/fbsource?force_mercurial
infinitepush-other = ssh://hg.vip.facebook.com/data/scm/fbsource?force_mononoke
```
So, we would like that rename of the path don't cause `hg sl` show that nothing is backed up.
We use the path as part of our filename.
So, we will go to the server to check the commits, it might slow down a bit the very first `hg sl` or `pushbackup` after the path change, but it should be acceptable.
Reviewed By: quark-zju
Differential Revision: D14366820
fbshipit-source-id: a0fd7bad530dd6690926fe02d38b93c2a72df00b
Summary:
Make them zero-copy to reduce overhead. A side effect is the types become
`bytearray`, instead of `bytes`. It seems fine for zstd use-case.
Reviewed By: ikostia
Differential Revision: D14338313
fbshipit-source-id: af43d4cce5559fe877373737a71e1e1678a17ca7
Summary: These 2 commands were broken when treemanifest.treeonly=True was set
Reviewed By: DurhamG
Differential Revision: D14316779
fbshipit-source-id: e626df41c92036fa3bd61c072f09b0d6c99c6f9f
Summary:
If an executable filenode was reused from p2 and mode hasn't been
changed (i.e. a filenode was executable and it is still an executable after a
merge) and and p1 didn't have this file at all then file will still be listed as
changed just because manifest1.flags(fname) returns ''.
After this diff if a filenode is reused then file will be listed as changed
only if it's a parent from where this filenode was reused have different flags.
So if filenode was reused from p1/p2 and p1/p2 has same flags and new
filenode, then file is not listed. But if filenode was reused from p1 and p1
has different flags, then filenode won't be reused.
It's possible that both new filenode is equal to both p1 and p2. In that case
we'll compare only with p1's mode.
Note that it's only get triggered during pushrebase, because during local
commits `fctx.filenode()` is None
Reviewed By: DurhamG
Differential Revision: D14300214
fbshipit-source-id: 1bf6c4802cfce5db6654da673333a56389432617
Summary:
When searching for data, mercurial will search the datastores by first looking
into the local cache, then try to find the data over the network. When
remotefilelog.fetchpacks is enabled, all the data fetched over the network will
be stored into packfiles, but those fetches are done via the loose-files remote
datastore. Due to this, even if memcache successfully find the requested data,
the datastore won't find it, due to it expecting loosefiles.
Fixing this simply requires the fetches to be done via a packfile store when
remotefilelog.fetchpacks is enabled.
Reviewed By: DurhamG
Differential Revision: D14216815
fbshipit-source-id: ed97c64651a733b36e0f2b4e209ce8ccdbb7911e
