Marker are now written as soon as possible but within a transaction. Using a
transaction ensure a proper behavior on error and rollback compatibility.
Flush logic are not necessary anymore and are dropped from lock release.
With this changeset, the obsstore is open, written and closed for every single
added marker. This is expected to be highly inefficient and batched write should
be implemented "quickly".
Another issue is that every flush of the file will invalidate the obsstore
filecache and trigger a full re instantiation of the repo.obsstore attribute
(including, reading and parsing entry). This is also expected to be highly
inefficient and proper filecache operation should be implemented "quickly" too.
A side benefit of the filecache issue is that repo.obsstore object is properly
invalidated on transaction abortion.
This is the second step toward incremental writing of marker inside a
transaction. The obsstore file is now handled append only.
Header writing have been extracted from _writemarkers.
Because the _writemarkers method have been dropped, the push code
directly reuse the serialised content of local repo `listkeys`. This
is not very pretty, but this part of the protocol still need major
improvement anyway.
This is the first step toward incremental writing of obsolete marker within a
transaction.
For this purpose, obsstore is now given its repo sopener. This make it able to
handles read and write to the obsstore file itself. Most IO logic is removed
from localrepo and handled by obsstore object directly.
This function augments strip to incrementally update the branchheads cache
rather than recompute it from scratch. This speeds up the performance of strip
and rebase on repos with long history. The performance optimization only
happens if the revisions stripped are all on the same branch and the parents of
the stripped revisions are also on that same branch.
This adds a few test cases, particularly one that reproduces the extra heads
that mpm observed.
_updatebranchcache used to use revlog.reachable. After the switch to
revlog.ancestors, we can now clean it up a bit and switch the algorithm from
nodes to revs.
Accepting a variable number of arguments as the old API did is
deeply ugly, particularly as it means the API can't be extended
with new arguments. Partly as a result, we have at least three
different implementations of the same ancestors algorithm (!?).
Most callers were forced to call ancestors(*somelist), adding to
both inefficiency and ugliness.
b67b333b0d8a attempted to force the filecaches in localrepo to reload
everything after a rollback. But simply clearing _filecache isn't enough,
invalidate() needs to be called before/after. localrepo._rollback calls
invalidate() already, so we clear the map right afterwards which ensures
everything will be reread.
Destroying history via strip used to invalidate the branchheads cache,
causing it to be regenerated the next time it is read. This is
expensive in large repos. This change converts strip to pass info to
localrepo.destroyed() to enable to it to incrementally update the
cache, improving the performance of strip and other operations that
depend on it (e.g., rebase).
This change also strengthens a bit the integrity checking of the
branchheads cache when it is read, by rejecting the cache if it has
nodes in it that no longer exist.
Similar to branch heads we introduce the notion of bookmarkheads.
Bookmarkheads are changests that are bookmarked with the given bookmark
or a diverged version
The original motivation was changectx.phase() had special logic to
correctly lookup in repo._phaserev, including invalidating it when
necessary. And at other places, repo._phaserev was accessed directly.
This led to the discovery that phases state including _phaseroots,
_phaserev and _dirtyphase was manipulated in localrepository.py,
phases.py, repair.py, etc. phasecache helps encapsulating that.
This patch replaces all phase state in localrepo with phasecache and
adjust related code except for advance/retractboundary() in phases.
These still access to phasecache internals directly. This will be
addressed in a followup.
Introduce manifestdict.withflags() to get a set of all files which have any
flags set, since these are likely to be a minority. Otherwise checking .flags()
for every file is a lot of dictionary lookups and is quite slow.
Introduce match.always() to check if a match object always says yes, i.e.
None was passed in. If so, mfmatches should not bother iterating every file in
the repository.
Cosmetic cleanups. Fix comment typo referring to the notion of multiple tips.
Make variable describing a generator end in 'gen'.
Fix another var containing a node not to end with 'rev'.
The fix introduced in 3509b9cf8f86 was only partially successful. It is correct
to turn dirstate 'm' merge records into normal/dirty ones but copy records are
lost in the process. To adjust them as well, we need to look in the first
parent manifest to know which files were added and preserve only related
records. But the dirstate does not have access to changesets, the logic has to
moved at another level, in localrepo.
Here is a script illustrating the previous behaviour:
The merge brings a new file 'b' from remote
$ hg merge 1 --debug
searching for copies back to rev 1
unmatched files in other:
b
resolving manifests
overwrite: False, partial: False
ancestor: 07f494440405, local: 540395c44225+, remote: 102a90ea7b4a
b: remote created -> g
updating: b 1/1 files (100.00%)
getting b
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)
Delete but do not remove b
$ rm b
$ hg st
! b
The commit succeeds
$ hg commit -m merge
$ hg parents --template "{rev} {desc|firstline} files: {files}\n"
3 merge files:
$ hg st
! b
b changes were ignored, but even b existence was ignored
$ hg manifest
a
This happens because localrepo.commitctx() checks the input ctx.files(), which
is empty for workingctx.files() only returns added, modified or removed
entries, and bypass files/manifest updates completely. So the committed
revision manifest is the same as its first parent one, not containing the 'b'
file.
This patch forces the commit to abort in presence of a merge and missing files.
test-merge4.t is modified accordingly as it was introduced to check hg was not
just terminating with a traceback (5cc0d3ba11f9).
bookmarks is copied to journal.bookmarks differently from how dirstate is
copied to journal.dirstate. The different way is less robust, which can render
the repo unpushable by other users if the first pushing user aborts their
transaction.
The underlying cause is that the copyfile method attempts an unnecessary chmod,
which fails if the user is not the owner of the journal.bookmarks file.
This patch makes the bookmarks journaling more consistent with the rest of the
journaling, and will allow users to update lingering journal.bookmarks files
that they're not the owners of.
New users of filecache use different names for the function used to compute
the runtime path of the cached file.
Users should subclass filecache and provide their own version of this
function to call the appropriate join function on 'obj' (an instance
of the class that its member function was decorated).
