xataio/pgroll
1
1
Fork 0
mirror of https://github.com/xataio/pgroll.git synced 2024-09-11 05:45:48 +03:00
Code Issues Packages Projects Releases Wiki Activity
21 Commits 3 Branches 13 Tags 4.5 MiB
1a7eb3271a
Commit Graph

21 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Andrew Farries
1a7eb3271a
Format the examples with jq (#21) 2023-07-06 15:26:29 +01:00
Andrew Farries
cb5f191acf
Restructure tests (#17) 
* Add tests for the `roll` package to ensure that the new versioned
schema is created on `start` and removed on `rollback`. We already had a
test there to ensure the previous versioned schema is dropped on
`complete`.
* Remove parts of tests for the create table operation that concerned
themselves with checking for the existence/non-existence of the
versioned schema. That is now tested in the `roll` package and we want
the tests for each operation to be focussed on the operation itself, not
schema creation.
* Add one more test for the create table operation to ensure that the
view for the new table is usable after `complete` (we already had a test
to ensure that it's usable on `start`).
 2023-07-06 08:56:01 +01:00
dependabot[bot]
0139d75560
Bump google.golang.org/grpc from 1.47.0 to 1.53.0 (#18) 
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from
1.47.0 to 1.53.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/grpc/grpc-go/releases">google.golang.org/grpc's
releases</a>.</em></p>
<blockquote>
<h2>Release 1.53.0</h2>
<h1>API Changes</h1>
<ul>
<li>balancer: support injection of per-call metadata from LB policies
(<a
href="https://redirect.github.com/grpc/grpc-go/issues/5853">#5853</a>)</li>
<li>resolver: remove deprecated field
<code>resolver.Target.Endpoint</code> and replace with
<code>resolver.Target.Endpoint()</code> (<a
href="https://redirect.github.com/grpc/grpc-go/issues/5852">#5852</a>)
<ul>
<li>Special Thanks: <a
href="https://github.com/kylejb"><code>@​kylejb</code></a></li>
</ul>
</li>
</ul>
<h1>New Features</h1>
<ul>
<li>xds/ringhash: introduce <code>GRPC_RING_HASH_CAP</code> environment
variable to override the maximum ring size. (<a
href="https://redirect.github.com/grpc/grpc-go/issues/5884">#5884</a>)</li>
<li>rls: propagate headers received in RLS response to backends (<a
href="https://redirect.github.com/grpc/grpc-go/issues/5883">#5883</a>)</li>
</ul>
<h1>Bug Fixes</h1>
<ul>
<li>transport: drain client transport when streamID approaches
MaxStreamID (<a
href="https://redirect.github.com/grpc/grpc-go/issues/5889">#5889</a>)</li>
<li>server: after GracefulStop, ensure connections are closed when final
RPC completes (<a
href="https://redirect.github.com/grpc/grpc-go/issues/5968">#5968</a>)</li>
<li>server: fix a few issues where grpc server uses RST_STREAM for
non-HTTP/2 errors (<a
href="https://redirect.github.com/grpc/grpc-go/issues/5893">#5893</a>)</li>
<li>xdsclient: fix race which can happen when multiple load reporting
calls are made at the same time. (<a
href="https://redirect.github.com/grpc/grpc-go/issues/5927">#5927</a>)</li>
<li>rls: fix a data race involving the LRU cache (<a
href="https://redirect.github.com/grpc/grpc-go/issues/5925">#5925</a>)</li>
<li>xds: fix panic involving double close of channel in xDS transport
(<a
href="https://redirect.github.com/grpc/grpc-go/issues/5959">#5959</a>)</li>
<li>gcp/observability: update method name validation (<a
href="https://redirect.github.com/grpc/grpc-go/issues/5951">#5951</a>)</li>
</ul>
<h1>Documentation</h1>
<ul>
<li>credentials/oauth: mark <code>NewOauthAccess</code> as deprecated
(<a
href="https://redirect.github.com/grpc/grpc-go/issues/5882">#5882</a>)
<ul>
<li>Special Thanks: <a
href="https://github.com/buzzsurfr"><code>@​buzzsurfr</code></a></li>
</ul>
</li>
</ul>
<h2>Release 1.52.3</h2>
<h1>Bug Fixes</h1>
<ul>
<li>Fix user-agent version</li>
</ul>
<h2>Release 1.52.2</h2>
<h1>Bug Fixes</h1>
<ul>
<li>xds: fix panic involving double close of channel in xDS transport
(<a
href="https://redirect.github.com/grpc/grpc-go/issues/5959">#5959</a>)</li>
</ul>
<h2>Release 1.52.1</h2>
<h1>Bug Fixes</h1>
<ul>
<li>grpclb: rename grpclbstate package back to state (<a
href="https://redirect.github.com/grpc/grpc-go/issues/5963">#5963</a>)</li>
</ul>
<h2>Release 1.52.0</h2>
<h1>New Features</h1>
<ul>
<li>xdsclient: log node ID with verbosity INFO (<a
href="https://redirect.github.com/grpc/grpc-go/issues/5860">#5860</a>)</li>
<li>ringhash: impose cap on <code>max_ring_size</code> to reduce
possibility of OOMs (<a
href="https://redirect.github.com/grpc/grpc-go/issues/5801">#5801</a>)</li>
</ul>
<h1>Behavior Changes</h1>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="dba26e15a0"><code>dba26e1</code></a>
Change version to 1.53.0 (<a
href="https://redirect.github.com/grpc/grpc-go/issues/5983">#5983</a>)</li>
<li><a
href="2a1e9348ff"><code>2a1e934</code></a>
server: after GracefulStop, ensure connections are closed when final RPC
comp...</li>
<li><a
href="e2d69aa076"><code>e2d69aa</code></a>
tests: fix spelling of variable (<a
href="https://redirect.github.com/grpc/grpc-go/issues/5966">#5966</a>)</li>
<li><a
href="a6376c9893"><code>a6376c9</code></a>
xds/resolver: cleanup tests to use real xDS client 3/n (<a
href="https://redirect.github.com/grpc/grpc-go/issues/5953">#5953</a>)</li>
<li><a
href="bf8fc46fa6"><code>bf8fc46</code></a>
xds/resolver: cleanup tests to use real xDS client 5/n (<a
href="https://redirect.github.com/grpc/grpc-go/issues/5955">#5955</a>)</li>
<li><a
href="3930549b38"><code>3930549</code></a>
resolver: replace resolver.Target.Endpoint field with Endpoint() method
(<a
href="https://redirect.github.com/grpc/grpc-go/issues/5852">#5852</a>)</li>
<li><a
href="894816c487"><code>894816c</code></a>
grpclb: rename <code>grpclbstate</code> package back to
<code>state</code> (<a
href="https://redirect.github.com/grpc/grpc-go/issues/5962">#5962</a>)</li>
<li><a
href="e5a0237a46"><code>e5a0237</code></a>
encoding: fix duplicate compressor names (<a
href="https://redirect.github.com/grpc/grpc-go/issues/5958">#5958</a>)</li>
<li><a
href="4adb2a7a00"><code>4adb2a7</code></a>
xds/resolver: cleanup tests to use real xDS client 2/n (<a
href="https://redirect.github.com/grpc/grpc-go/issues/5952">#5952</a>)</li>
<li><a
href="52a8392f37"><code>52a8392</code></a>
gcp/observability: update method name validation (<a
href="https://redirect.github.com/grpc/grpc-go/issues/5951">#5951</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/grpc/grpc-go/compare/v1.47.0...v1.53.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=google.golang.org/grpc&package-manager=go_modules&previous-version=1.47.0&new-version=1.53.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/xataio/pg-roll/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-06 09:16:11 +02:00
Andrew Farries
ac7a3d0a82
Qualify versioned schema names (#16) 
Close https://github.com/xataio/pg-roll/issues/15.

Qualify versioned schema with the name of the schema they represent.

For a migration called `01_create_table` running in the `public` schema,
the versioned schema is called `public_01_create_table`.
 2023-07-05 16:56:24 +01:00
Andrew Farries
7bcbee1256
Remove old schema version on pg-roll complete (#13) 
Ensure that the previous version of the schema is removed by `pg-roll
complete`.
 2023-07-05 13:58:55 +01:00
Carlos Pérez-Aradros Herce
5e61b040e1
Make migrations schema aware (#12) 
This change will retrieve and store the resulting schema after a
migration is completed. This schema will be used as the base to execute
the next migration, making it possible to create views that are aware of
the full schema, and not only the one created by the last migration.

We use a function to retrieve the schema directly from Postgres instead
of building it from the migration files. This allows for more features
in the future, like doing an initial sync on top of the existing schema
or automatically detecting and storing out of band migrations from
triggers.

Example JSON stored schema:

```
{
  "tables": {
    "bills": {
      "oid": "18272",
      "name": "bills",
      "columns": {
        "id": {
          "name": "id",
          "type": "integer",
          "comment": null,
          "default": null,
          "nullable": false
        },
        "date": {
          "name": "date",
          "type": "time with time zone",
          "comment": null,
          "default": null,
          "nullable": false
        },
        "quantity": {
          "name": "quantity",
          "type": "integer",
          "comment": null,
          "default": null,
          "nullable": false
        }
      },
      "comment": null
    },
    "products": {
      "oid": "18286",
      "name": "products",
      "columns": {
        "id": {
          "name": "id",
          "type": "integer",
          "comment": null,
          "default": "nextval(_pgroll_new_products_id_seq::regclass)",
          "nullable": false
        },
        "name": {
          "name": "name",
          "type": "varchar(255)",
          "comment": null,
          "default": null,
          "nullable": false
        },
        "price": {
          "name": "price",
          "type": "numeric(10,2)",
          "comment": null,
          "default": null,
          "nullable": false
        }
      },
      "comment": null
    },
    "customers": {
      "oid": "18263",
      "name": "customers",
      "columns": {
        "id": {
          "name": "id",
          "type": "integer",
          "comment": null,
          "default": null,
          "nullable": false
        },
        "name": {
          "name": "name",
          "type": "varchar(255)",
          "comment": null,
          "default": null,
          "nullable": false
        },
        "credit_card": {
          "name": "credit_card",
          "type": "text",
          "comment": null,
          "default": null,
          "nullable": true
        }
      },
      "comment": null
    }
  }
}
```


After this change, I believe that the `create_table` operation is
feature complete and can be used for many sequential migrations.
 2023-07-05 13:20:59 +02:00
Andrew Farries
69ab631485
Add a sentinel error for no active migration (#11) 
Add a sentinel error `ErrNoActiveMigration` for the case where there is
no active migration. This improves the error strings presented to users
by not mentioning SQL errors.

**`pg-roll start` when there is a migration in progess:**
```
Error: a migration for schema "public" is already in progress 
```

**`pg-roll rollback` when there is no migration in progress:**
```
Error: unable to get active migration: no active migration 
```

**`pg-complete` when there is no active migration:**
```
Error: unable to get active migration: no active migration
```
 2023-07-03 14:18:31 +01:00
Carlos Pérez-Aradros Herce
a8c4fddd14
Add migrations state handling (#7) 
This migrations introduces state handling by creating a dedicated
`pgroll` schema (name configurable). We will store migrations there, as
well as their state. So we keep some useful information, ie the
migration definition (so we don't need it for the `complete` state).

Schema includes the proper constraints to guarantee that:
* Only a migration is active at a time
* Migration history is linear (all migrations have a unique parent,
except the first one which is NULL)
* We now the current migration at all times

Some helper functions are included:

* `is_active_migration_period()` will return true if there is an active
migration.
* `latest_version()` will return the name of the latest version of the
schema.
 2023-06-28 11:10:03 +02:00
Andrew Farries
f9a530c900
Add test cleanup of Migrations value (#10) 
Call its `Close` method via `t.Cleanup`.
 2023-06-28 07:36:11 +01:00
Carlos Pérez-Aradros Herce
6c4748916c
Add linter to tests (#9) 
Found some lint warnings, let's have them in CI
 2023-06-27 16:33:50 +01:00
Carlos Pérez-Aradros Herce
838f8d4c46
Fix tests referencing from GitHub branch protections (#8) 
Settings file was incorrectly referencing a non existing job
 2023-06-27 13:46:36 +01:00
Andrew Farries
5f3094822e
Add a rollback command to the CLI (#6) 
Add a `rollback` command to the CLI.

Use the rollback functionality added to the `migrations` package in #5
to perform the rollback.

Example:

```bash
go run . start examples/01_create_tables.json 
go run . rollback examples/01_create_tables.json 
```
The schema for the `01_create_table` version is removed from the
database along with the underlying tables.

⚠️ We currently don't have a way to ensure that only uncompleted
migrations can be rolled back. Once we have some state in the db
recording with migrations have been applied, we can revisit this command
to ensure completed migrations can't be rolled back ⚠️
 2023-06-27 13:45:13 +01:00
Andrew Farries
b209d8c283
Implement the Rollback method for the create table operation (#5) 
Implement rollbacks for the create table operation.

* Delete the new version of the schema and any views it contains.
* Drop the tables created by the operations.

Add an integration test to check that these resources are successfully
dropped.

This PR adds the rollback operation to the `migrations` package;
supporting the operation through the CLI will be in a later PR.
 2023-06-27 13:42:04 +01:00
Carlos Pérez-Aradros Herce
5ad868af01
Add GitHub settings (#4) 
I followed the conventions we have across the baord
 2023-06-27 14:14:42 +02:00
Andrew Farries
3baf4f9e58
Merge pull request #3 from xataio/add-ci-workflow 
Add a simple CI workflow
 2023-06-27 08:47:50 +01:00
Andrew Farries
3f24fe0e89 Add a simple CI workflow 2023-06-26 15:37:43 +01:00
Andrew Farries
51ec2204c6
Merge pull request #2 from xataio/integration-tests 
Add integration tests for the  `migrations` package using Testcontainers
 2023-06-26 14:44:55 +01:00
Andrew Farries
20c57553b6 Add tests for create table operation 2023-06-26 12:48:00 +01:00
Andrew Farries
0a1fe86e0d
Merge pull request #1 from xataio/add-analyze-command 
Add `analyze` as a hidden subcommand
 2023-06-26 12:19:21 +01:00
Andrew Farries
2c6a6631fa Add analyze as a hidden subcommand 2023-06-23 18:43:59 +01:00
Carlos Pérez-Aradros Herce
bdaf08f54c Initial commit 
Basic skeleton and simple op
 2023-06-22 17:30:40 +02:00