With the old fields in the `State` broken out we can now make `State` a
proper enum, replacing the nested `Status`. The naming with both `State`
and `Status` was a bit confusing.
The version is not currently in used but will probably be useful later
when we want Reshape to be backwards compatbility. Having the version
stored in the database will then allow us to perform schema migrations
on the metadata of Reshape.
Before, the entire state was stored as a JSON encoded blob including all
completed migrations. This commits moves the migrations into a dedicated
table, which prevents the encoded from growing indefinitely. The size of
the state probably wouldn't have become a problem but it feels weird for
a schema migration tool to not use a proper schema.
Next, the version stored in the state struct will be moved to a separate
key in the `reshape.data` table and `State` will become an enum with the
same structure as `Status`. The naming right now is a bit confusing.
Until now, using alter_column would cause indices to be lost as they
weren't copied to the temporary column. This fixes that for indices that
consist of one or more actual columns. It won't work for indices on
expressions.
This allows people new to the project to understand how Reshape works before getting into the technical detail of setting it up and using it (which they may or may not want to do just yet).
Some migrations can be completed without needing a transaction for
atomicity. We should avoid transactions for DDL as far as possible to
avoid interfering with other queries and holding locks too long. It's
possible that all actions could be completed without a transaction but
I'm not sure of that yet.
Some migrations also won't work with transactions. For example dropping
an index using `DROP INDEX CONCURRENTLY` doesn't work inside a
transation.
Next step is to perform the same change for aborts.
This builds on the previous commit to make the transactions even more
granular when completing a migration. Keeping the transaction span short
is important to avoid interfering with other queries.
We previously used one transaction across all the migrations being
completed, which is not ideal from a locking perspective. Preferably we
want to keep the transactions as short-lived as possible to avoid
interfering with other queries. The next step will be to use one
transaction for each action.
To achieve this, we need to introduce a new intermediate state called
`Completing` which tracks which migrations have been completed so far.
Changing the default value only affects the new schema, the old schema
will still use the existing default value. This is to make sure a new
default value doesn't break the old application.
The builder will help simplify the tests and avoid boilerplate updates
when need fields are added to the create_table migration. We might want
to add builders all migrations later.
This checks for dangling temporary columns, triggers and function which
were used by migrations but should be removed once the migration has
been completed or aborted.
Improved backtrace support for errors in Rust is currently in progress:
https://github.com/rust-lang/rust/issues/53487. Adding the feature lets
anyhow add backtraces already which greatly improves readability of
errors in tests.
Column tracking refers to how temporary columns for a column are
tracked. This changes the tracking to use a single vector, the first
element is the original column name and subsequent ones are temporary
columns.
Migrations shouldn't need to know about any temporary schema changes
as they should already have been applied once the migration is going
to be completed.
This state is set as soon as a migration is started, before any
migrations are run. If a migration unexpectedly fails and doesn't abort
properly (which should happen automatically), the state will be left
dangling in `Applying`. In that case, it's fine for the user to run
migrate again as all migrations are idempotent.
