The callback will likely need to perform some operation related to the
transaction (eg: backing files up). So we better pass the current transaction as
the callback argument. Otherwise callback that needs it has to rely on horrible
weak reference trick.
The first foreseen user of this is changelog._writepending. We would like it to
register the temporary file it create for cleanup purpose.
The case where a backup of a missing file was requested was previously
handled by the 'entries' list. As the 'backupentries' is about to gain
ability to backup files outside of '.hg/store', we want it to be able
to handle the missing file too.
Reminder: using 'addbackup' on a missing file means that such file needs to be
deleted if we rollback the transaction.
The `startgroup` and `endgroup` methods are used in a very specific
context to wrap a very specific operation (revlog truncation). It does
not make sense to perform any other operations during such a "group"
(eg:file backup). There is currently no user of backupfile during a
"group" so we drop the group-specific code and restrict authorized
operations during "group".
The addchangegroup code considers the transaction done after a 'tr.close()' call
and schedules the hook's execution for after lock release. In the nested transaction
case, the transaction is not yet committed and we must delay this scheduling.
We add an 'addpostclose' method (like the 'addpending' and 'addfinalize' ones) that
registers code to be run if the transaction is successfully committed.
The new 'addfinalize' method allows people to register a callback to
be triggered when the transaction is closed. This aims to get rid of
explicit calls to 'changelog.finalize'. This also obsoletes the
'onclose' function but removing it is not in the scope of this series.
The contents of the transaction must be flushed to disk before running
a hook. But it must be flushed to a special file so that the normal
reader does not use it. This logic is currently in the changelog only.
We add some facility to register such operations in the transaction
itself.
We extract the code generating files into its own function. We are
about to move this code around to fix a bug. We'll need it in a
function soon to reuse it for "pending" logic. So we move the code
into a function instead of moving it twice.
Previously the journal.backupfiles file was delimited by \0. Now we delimit it
using \n (same as the journal file). This allows us to change the number of
values in each line more easily, rather than relying on the count of \0's.
The transaction format will be changing a bit over the next releases, so let's
go ahead and add a version number to make backwards compatibility easier. This
whole file format was broken prior to 3.2 (see previous patch), so changing it
now is pretty low risk.
The transaction backupfiles logic was broken for 'hg recover'. The file format
is XXX\0XXX\0YYY\0YYY\0 but the parser did a couple things wrong. 1) It went one
step beyond the final \0 and tried to read past the end of the array. 2)
array[i:i+1] returns a single item, instead of two items as intended.
Added a test to catch it, which turns out to be the first actual 'hg recover'
test.
We allow a vfs argument to `addfilegenerator`. This allows generating files outside
of the store directory like bookmarks. However, this is not really working since
we do not have the infrastructure to backup and restore files outside of store.
By chance, the bookmark file is already backed up by another mechanism so we can
restrict this new feature to bookmarks (which is our only interest here) and
proceed.
The backup restoration is actually hard-coded for the main file. And this
hard-coded list is the only one used when repairing an interrupted transaction
from another process.
Solving this problem is out of the scope of this series so we document it and
work around its implications.
_ is usually used for i18n markup but we also used it for I-don't-care
variables.
Instead, name don't-care variables in a slightly descriptive way but use the _
prefix to designate unused variable.
This will mute some pyflakes "import '_' ... shadowed by loop variable"
warnings.
A new `transaction.addfilegenerator` function is added. It allows external code
to register files to be generated. See inline documentation for details.
It is important to gather all file creation logic on the transaction
as at some point we'll want to mimic the "pre-transaction-commit"
logic that we use for revlog. I'm refering to the logic that lets
hooks see the result of the transaction before it actually gets
committed.
The journal.backupfiles descriptor wasn't being closed. This resulted in
hgsubversion test runs having a bagillion descriptors open, which crashed on
platforms with low open file limits (like OSX).
It is now possible to register parameters to be used when invoking hooks in this
transaction. This will cope with the fact that bundle2 adds multiple data types
in a single transaction.
Do not expect any wide and consistent usages of this in the next release. This
will be used by bundle2 experiments first. It will be made better for the release
after.
This adds support for normal, non-append-only files in transactions. For
example, .hg/store/fncache and .hg/store/phaseroots should be written as part of
the transaction, but are not append only files.
This adds a journal.backupfiles along side the normal journal. This tracks which
files have been backed up as part of the transaction. transaction.addbackup()
creates a backup of the file (using a hardlink), which is later used to recover
in the event of the transaction failing.
Using a seperate journal allows the repository to still be used by older
versions of Mercurial. A future patch will use this functionality and add tests
for it.
Adds an optional onclose parameter to transactions that gets called just before
the transaction is committed. This allows things that build up data over the
course of the transaction (like the fncache) to commit their data.
Also adds onabort. It's not used, but will allow extensions to hook into onclose
and onabort to provide transaction support.
Before this patch, unlink target file is once opened before unlinking,
because "opener" before vfs migration doesn't have "unlink()"
function.
This patch uses "vfs.unlink()" instead of "open()" and "fp.name".
Add missing calls to close() to many places where files are
opened. Relying on reference counting to catch them soon-ish is not
portable and fails in environments with a proper GC, such as PyPy.
When transactions without entries were aborted, the journal (of size 0) was not
unlinked, which prevents subsequent operations until hg recover is run on the
repository.
We also make sure the journal is unlinked when committing, even if the provided
hook doesn't do so.
When performing a strip operation on a repository, it is vital that all the
truncations are performed, or that none of them are. This is done by adding
support for writing a number of entries in a single operation.
Co-contributor: Sune Foldager <cryo@cyanite.org>
This adds a change to the way that abort is processed, as it will not continue
truncating files beyond the first failure, otherwise the respective
functionality is maintained, i.e. abort will not unlink files, but rollback
will.
Co-contributor: Sune Foldager <cryo@cyanite.org>
All transactional methods on the transaction class have had a decorator
added that ensures the transaction is running.
Co-contributor: Sune Foldager <cryo@cyanite.org>
This solves that the journal file was always deleted when the transaction
was deleted, no matter whether the abort (rollback) succeeded or not.
Thus, never supporting a hg recover. The journal file is now only deleted
on close (commit) or a successful abort.
This associates a transaction handle with a given repository object, and
any additional calls to start new transactions reuse that transaction.
For the 2700 patch import run, this brings the system time down from
1m20s to 50s, mostly by skipping backups of the dirstate file.
(note, this patch does not change hg import to use the nested transaction,
mq is the only user right now)
Commit would overwrite undo.dirstate unconditionally, so an undo after
an aborted commit would restore the dirstate from the aborted commit
and not the prior transaction.
This copies dirstate to journal.dirstate and moves it after a
successful transaction.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Remove all remaining print statements
Convert most prints to ui.warn or ui.write
Pass a write function into transactions
manifest hash: d1b0af7a344fc087a5acfe3ae87b782c20d043e0
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
iD8DBQFCx1Q3ywK+sNU5EO8RAqSTAJwM5t/m+JOlf2ZXOjuItCSdFiubcwCdFm3G
HoicikSYpTgfCj2pIRfyLjo=
=Loqo
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
transaction: nullify journal after close()
This keeps us from trying to cleanup in __del__.
bug spotted by K Thananchayan <thananck@yahoo.com>
manifest hash: 1f4c5fa43d2458cdcb6ec0f0a7066b3c3e699f33
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
iD8DBQFCxwQ6ywK+sNU5EO8RAuGhAJ9+BnKILwRCF9EKlPTz2zptiysQ/QCgsB3s
VKIehiStv+ibWQQi15k4mwk=
=nZD1
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
[PATCH] rename under the other OS
From: K Thananchayan <thananck@yahoo.com>
Rename fails under windows if dest file exists. This
patch add a rename method to util module that removes
the dest file and retries if initial attempt fails.
manifest hash: 2744d9fd1717e15133b411a269df909fa8ec0faf
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
iD8DBQFCuNzDywK+sNU5EO8RAsPBAJ9NQN3bTuJvTem5x+utGnoMkhYc0QCbBFSJ
PoMP69H1UnVE6drFlnlFE0s=
=pePw
-----END PGP SIGNATURE-----
