This action enables migrations to run custom SQL and is meant to be used
when no other actions fit. It doesn't provide any guarantees for
zero-downtime.
All the existing commands have been replaced with new commands which
read a bit better and leave better room for more commands in the future.
The following changes have been made:
- `reshape start` -> `reshape migration start`
- `reshape complete` -> `reshape migration complete`
- `reshape abort` -> `reshape migration abort`
- `reshape generate-schema-query` -> `reshape schema-query`
The old commands are still available but are marked as deprecated in the
help output and will be removed in the future. The README will be
updated later.
Closes#10
The existing test to set a column as NOT NULL didn't actually check that
the constraint was added, which has also been fixed. I also added a new
test for the opposite change, making a NOT NULL column nullable. The
README has also been updated with a new example for setting a column as
NOT NULL.
For Reshape to work with an existing database, the first migration
shouldn't autocomplete as then there can't be a zero-downtime
deployment. The user should instead use `--complete` if they don't care
about zero-downtime, for example when setting up a dev or CI
environment. We should update the `--complete` flag to apply and
complete each migration one by one for efficiency.
We want it to be possible for Reshape to be adopted for existing
databases with an existing schema. To make this possible, we can't check
if the schema path matches the current migration as there won't be a
current migration at first. Instead we'll compare against the target
migration.
The next step in making Reshape work with existing databases is to not
automatically complete the first migration. This will not be safe for an
existing application adopting Reshape.
Database queries will now be retried if possible using exponential
backoff with jitter. This should help protect against connection
problems and also timeouts caused by us now setting lock_timeout to
avoid blocking other queries. If such a timeout occurs, we should fail
to let waiting queries execute and then try again until the blocking
queries have completed.
When attempting to acquire a lock for DDL queries, Postgres will wait
for any long-running queries to complete first. It will also block any
other queries that need access, which causes a queue of queries until we
release our lock. This has the effect of long-running queries blocking
other queries when we perform migrations, which is equivalent to
downtime if the query runs for long enough.
To avoid this, we set the `lock_timeout` setting. This puts an upper
bound on how long we will wait for a lock and also for how long other
queries can be blocked by a long-running query. The next step will be to
add automatic retries to handle timeouts gracefully.
Reference: https://medium.com/paypal-tech/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680
