mirror of
https://github.com/xataio/pgroll.git
synced 2024-08-16 09:10:26 +03:00
PostgreSQL zero-downtime migrations made easy
c76ea9ce48
Add support for adding a foreign key constraint to an existing column.
Such a migration looks like:
```json
{
"name": "21_add_foreign_key_constraint",
"operations": [
{
"set_foreign_key": {
"table": "posts",
"column": "user_id",
"references": {
"table": "users",
"column": "id"
},
"up": "(SELECT CASE WHEN EXISTS (SELECT 1 FROM users WHERE users.id = user_id) THEN user_id ELSE NULL END)",
"down": "user_id"
}
}
]
}
```
This migration adds a foreign key constraint to the `user_id` column in
the `posts` table, referencing the `id` column in the `users` table.
The implementation is similar to the **set not null** and **change
column type** operations:
* On `Start`:
* Create a new column, duplicating the one to which the FK constraint
should be added.
* The new column has the foreign key constraint added as `NOT VALID` to
avoid taking a long lived `SHARE ROW EXCLUSIVE` lock (see
[here](https://medium.com/paypal-tech/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680#00dc)).
* Backfill the new column with values from the existing column,
rewriting values using the `up` SQL.
* Create a trigger to populate the new column when values are written to
the old column, converting values with `up`.
* Create a trigger to populate the old column when values are written to
the new column, converting values with `down`.
* On `Complete`
* Validate the foreign key constraint.
* Remove triggers
* Drop the old column
* Rename the new column to the old column name.
* Rename the foreign key constraint to be consistent with the new name
of the column.
* On `Rollback`
* Remove the new column and both triggers. Removing the new column also
removes the foreign key constraint on it.
The `up` SQL in this operation is critical. The old column does not have
a foreign key constraint imposed on it after `Start` as that would
violate the guarantee that `pg-roll` does not make changes to the
existing schema. The `up` SQL therefore needs to take into account that
not all rows inserted into the old schema will have a valid foreign key.
In the example `json` above, the `up` SQL ensures that values for which
there is no corresponding user in the `users` table result in `NULL`
values in the new column. Failure to do this would result in the old
schema failing to insert rows without a valid `user_id`. An alternative
would be to implement data quarantining for these values, as discussed
last week @exekias .
|
||
|---|---|---|
| .github | ||
| .vscode | ||
| cmd | ||
| examples | ||
| pkg | ||
| .golangci.yml | ||
| docker-compose.yml | ||
| go.mod | ||
| go.sum | ||
| main.go | ||
| README.md | ||
pg-roll
⚠️ Under development ⚠️
PostgreSQL zero-downtime migrations made easy.
Getting started (development)
-
Bring a development PostgreSQL up:
docker compose up -
Initialize pg-roll (first time only):
go run . init -
Start a migration:
go run . start examples/01_create_tables.json -
Inspect the results:
psql postgres://localhost -U postgres\d+ public.* \d+ 01_create_tables.* -
(Optional) Rollback the migration (undo):
go run . rollback -
Complete the migration:
go run . complete