Commit Graph

56 Commits

Author SHA1 Message Date
Rakesh Emmadi
3888ceb24a computed fields in export/import metadata, other fixes (#3211) 2019-11-07 20:09:48 +05:30
Aravind Shankar
980c65dbe2 cli(migrations): new folder structure and squash (#3072)
### Description
This PR introduces three new features:

- Support for a new migrations folder structure.
- Add `squash` command in preview.
- ~List of migrations on the Console and ability to squash them from console.~

#### New migrations folder structure

Starting with this commit, Hasura CLI supports a new directory structure for migrations folder and defaults to that for all new migrations created. 

Each migration will get a new directory with the name format `timestamp_name` and inside the directory, there will be four files:

```bash
└── migrations
    ├── 1572237730898_squashed
    │   ├── up.sql
    │   ├── up.yaml
    │   ├── down.yaml
    │   └── down.sql
```

Existing files old migration format `timestamp_name.up|down.yaml|sql` will continue to work alongside new migration files.

#### Squash command

Lots of users have expressed their interest in squashing migrations (see #2724 and #2254) and some even built [their own tools](https://github.com/domasx2/hasura-squasher) to do squash. In this PR, we take a systematic approach to squash migrations.

A new command called `migrate squash` is introduced. Note that this command is in **PREVIEW** and the correctness of squashed migration is not guaranteed (especially for down migrations). From our tests, **it works for most use cases**, but we have found some issues with squashing all the down migrations, partly because the console doesn't generate down migrations for all actions.

Hence, until we add an extensive test suite for squashing, we'll keep the command in preview. We recommend you to confirm the correctness yourself by diffing the SQL and Metadata before and after applying the squashed migrations (we're also thinking about embedding some checks into the command itself).

```bash
$ hasura migrate squash --help
(PREVIEW) Squash multiple migrations leading upto the latest one into a single migration file

Usage:
  hasura migrate squash [flags]

Examples:
  # NOTE: This command is in PREVIEW, correctness is not guaranteed and the usage may change.

  # squash all migrations from version 1572238297262 to the latest one:
  hasura migrate squash --from 1572238297262

Flags:
      --from uint             start squashing form this version
      --name string           name for the new squashed migration (default "squashed")
      --delete-source         delete the source files after squashing without any confirmation
```

### Affected components 
<!-- Remove non-affected components from the list -->

- CLI

### Related Issues
<!-- Please make sure you have an issue associated with this Pull Request -->
<!-- And then add `(close #<issue-no>)` to the pull request title -->
<!-- Add the issue number below (e.g. #234) -->
Close #2724, Close #2254, 

### Solution and Design
<!-- How is this issue solved/fixed? What is the design? -->
<!-- It's better if we elaborate -->

For the squash command, a state machine is implemented to track changes to Hasura metadata. After applying each action on the metadata state, a list of incremental changes is created.

### Steps to test and verify
1. Open console via cli and create some migrations.
2. Run `hasura migrate squash --from <version>`

### Limitations, known bugs & workarounds
<!-- Limitations of the PR, known bugs and suggested workarounds -->

<!-- Feel free to delete these comment lines -->
- The `squash` command is in preview
- Support for squashing from the console is WIP
- Support for squashing migrations that are not committed yet is planned.
- Un-tracking or dropping a table will cause inconsistent squashed down migration since console doesn't generate correct down migration.
- If cascade setting is set to `true` on any of the metadata action, generated migration may be wrong
2019-10-31 07:51:15 +05:30
Jeff Sieu
2ee7f7d76e cli(metadata): add diff command and dry-run flag (#3157)
### Description
Adds a `metadata diff` command to show comparisons between two different sets of Hasura metadata.
```
# Show changes between server metadata and the exported metadata file:
hasura metadata diff

# Show changes between server metadata and that in local_metadata.yaml:
hasura metadata diff local_metadata.yaml

# Show changes between metadata from metadata.yaml and metadata_old.yaml:
hasura metadata diff metadata.yaml metadata_old.yaml
```

Also adds a `--dry-run` flag to `metadata apply` command which will print the diff and exit rather than actually applying the metadata.

### Affected components 
- CLI
- Docs

### Related Issues
Close #3126, Close #3127

### Solution and Design
- Added `metadata_diff.go` and `metadata_diff_test.go`


### Steps to test and verify
```
hasura metadata export
# Make changes to migrations/metadata.yaml
hasura metadata diff
```

### Limitations, known bugs & workarounds
This is just a general-purpose diff. 

A more contextual diff with the understanding of metadata can be added once #3072  is merged.
2019-10-30 19:24:22 +05:30
Rakesh Emmadi
9bd5826020 allow customising graphql schema for a table (close #981) (#2509)
* allow customizing GraphQL root field names, close #981

* document v2 track_table API in reference

* support customising column field names in GraphQL schema

* [docs] add custom column fields doc in API reference

* add tests

* rename 'ColField' to 'ColumnField'

* embed column's graphql field in 'PGColumnInfo'

-> Value constructor of 'PGCol' is not exposed
-> Using 'parseJSON' to construct 'PGCol' in 'FromJSON' instances

* avoid using 'Maybe TableConfig'

* refactors & 'custom_column_fields' -> 'custom_column_names'

* cli-test: add configuration field in metadata export test

* update expected keys in `FromJSON` instance of `TableMeta`

* use `buildSchemaCacheFor` to update configuration in v2 track_table

* remove 'GraphQLName' type and use 'isValidName' exposed from parser lib

* point graphql-parser-hs library git repo to hasura

* support 'set_table_custom_fields' query API & added docs and tests
2019-09-19 10:17:36 +05:30
Aravind Shankar
5f3294f4a0 optimise migrate api for console on cli (#2895) 2019-09-18 11:06:16 +05:30
ziga
5b0a790b59 fix typo in cli (#2617) 2019-08-28 13:33:05 +05:30
Alexis King
ed26da59a6 Add support for GraphQL enum types via enum table references
These changes also add a new type, PGColumnType, between PGColInfo and
PGScalarType, and they process PGRawColumnType values into PGColumnType
values during schema cache generation.
2019-08-26 00:54:56 -05:00
Charlie Saunders
e90866752e fix some typos in cli (#2708)
* fix typo in manual CLI update message

* fix a typo that was indicated in #2689
2019-08-10 17:13:45 +00:00
Daven Paul Casia
7f06470082 add flag to disable ci colors (close #2072) (#2634)
* added log-level flag

* fix up the colors

* fix description
2019-07-30 06:55:38 +00:00
Vamshi Surabhi
4facb3c780 remove support for query templates (#2560)
query templates is a little known feature that lets you template rql
queries and serve them as rest apis. This is not relevant anymore
given the GraphQL interface and getting rid of it reduces the dev
time when adding features in few subsystems.

This feature has never been used outside hasura's internal projects or
documented or exposed through console and hence can safely be removed.
2019-07-23 19:11:34 +00:00
Ajeet D'Souza
b184ce0188 add server version to console html (close #2444) (#2452) 2019-07-02 12:44:29 +00:00
Rakesh Emmadi
0b210cc245 support allow-list for graphql queries (closes #989) (#2075) 2019-05-16 11:43:25 +05:30
Shahidh K Muhammed
71cf017197 add an api to dump postgres schema (close #1939) (#1967) 2019-04-30 14:04:08 +05:30
Aravind Shankar
5f02ebdefa added feedback on migration commands (close #1410) (#1931) 2019-04-03 16:59:58 +05:30
Aravind Shankar
1e20026ea2 added feedback for metadata commands (close #1411) (#1929) 2019-04-03 11:22:03 +05:30
Aravind Shankar
ca7d8b3df5 rename metadata reset to metadta clear (close #1851) (#1893)
The word "reset" is little ambiguous (reset to scratch or reset to last known consistent state). metadata reset clears the all the metadata to mimic a freshly installed hasura instance. This actually invokes the clear_metadata API in the backend and hence should also be called metadata clear for perfect clarity.

All references to metadata reset should be changed to metadata clear.
2019-03-29 10:44:56 +05:30
Simone Busoli
f6c11e545d fix bash completion example docs on cli (#1832) 2019-03-28 07:28:20 +05:30
Aravind Shankar
8043ddeebf accept a json or yaml file for metadata apply command (close #1698) (#1746) 2019-03-20 10:40:06 +05:30
Aravind Shankar
040bef2fd5 add options to create migration from files from sql and server (close #1699) (#1761) 2019-03-18 22:10:04 +05:30
nizar-m
f83a8e591f rename access-key to admin-secret (close #1347) (#1540)
Rename the admin secret key header used to access GraphQL engine from X-Hasura-Access-Key to X-Hasura-Admin-Secret.

Server CLI and console all support the older flag but marks it as deprecated.
2019-02-14 15:07:47 +05:30
Aravind Shankar
d489f2d90f added a cli flag to skip update checks (close #1586) (#1600)
adds a global flag `--skip-update-check`
2019-02-14 11:46:36 +05:30
Shahidh K Muhammed
55cc5fb044
cli: add new install script, commands (#1556) 2019-02-05 18:21:21 +05:30
Aravind Shankar
20be2e5ab1 add update-cli command to cli (close #1239) (#1251) 2019-02-04 16:21:29 +05:30
Daniel Compton
a5c9347d75 update help text for console command (close #1507) (#1508) 2019-01-29 21:32:49 +05:30
Shahidh K Muhammed
11e7c3f9d6 add anonymous telemetry (#1401) 2019-01-28 19:25:28 +05:30
Rakesh Emmadi
0bf2457e23 allow exposing postgres functions through GraphQL interface (close #333) (#1073) 2019-01-25 09:01:54 +05:30
Aravind Shankar
c64a83f073 display error message for invalid migration file (fix #1224) (#1238) 2019-01-04 15:15:54 +05:30
Anon Ray
512ee6fb9f adds basic support for remote schemas/schema stitching (#952) 2018-11-23 18:32:46 +05:30
Taru
2127ae878b fix a typo in metadata command (#788) 2018-10-20 08:53:20 +05:30
Abilash Menon
0993740786 add reload metadata command to the cli (close #722) (#725) 2018-10-16 11:55:30 +05:30
Aravind
de2ceea132 update help for metadata export command (#661) 2018-10-08 12:54:43 +05:30
Shahidh K Muhammed
ac004fec38
validate the directory before executing migrate create (fix #585) (#586) 2018-10-01 15:42:12 +05:30
Shahidh K Muhammed
330f90de40
add project flag, change persistentprerun to prerun (close #552) (#557)
### Description

Adds `--project` flag to the CLI and changes all `PersistentPreRuns` to `PreRuns`

What component does this PR affect? 

- [ ] Server
- [ ] Console
- [x] CLI
- [ ] Docs
- [ ] Community Content
- [ ] Build System

### Related Issue

#552
2018-09-27 19:27:17 +05:30
Maxim Makarov
0fc4ebf09a fix typos in metadata command (#493) 2018-09-20 07:00:37 +05:30
Aravind Shankar
9c7c650d2b migration apply should not fail if there are no migrations (close #389) (#390) 2018-09-10 18:51:30 +05:30
Tirumarai Selvan
82e09efce6 add event triggers (#329) 2018-09-05 16:56:46 +05:30
Aravind Shankar
ea9b187e7c update tests to use access key (close #113) (#296)
Closes #113
2018-08-30 21:24:12 +05:30
Aravind Shankar
396a2ccf8d change hasuradb url format, export some migrate functions (#294)
* fix migrate to use headers in query params

* export executeMigrations and PrintStatus
2018-08-22 00:07:47 +05:30
Shahidh K Muhammed
0ed3ff6418
remove manifests directory from init (close #254) (#258) 2018-08-06 17:03:17 +05:30
Aravind Shankar
75dbe35a21 create down migrations for actions from console (close #164) (#173) 2018-07-20 16:01:33 +05:30
Shahidh K Muhammed
9c848e2b94
add option to serve console assets from a local directory (#145)
* add option to serve console assets from a local directory

* remove /console prefix

* update gin context only if static dir is present
2018-07-18 16:22:07 +05:30
Aravind Shankar
80c1994dc2 inserts into schema_migration only once per version (fix #127) (#128) 2018-07-12 19:54:46 +05:30
Aravind Shankar
6a20784b95 move metadata.yml file inside migrations folder, fix #122 (#125)
metadata.yaml will be present inside migrations folder
2018-07-12 18:22:25 +05:30
Aravind Shankar
1bff452cb4 handle server endpoints that contains a url path (#64) 2018-07-09 19:17:38 +05:30
Aravind Shankar
1998e211af cli: update tests (#47)
* remove docker from migrate cmd test

* export migration status

* [cli] add fake writer for spinner

* Update console.go

* Update migrate_test.go
2018-07-06 10:36:27 +05:30
Shahidh K Muhammed
06fd236068
cli: close #38 handle server, cli and console versions (#43) 2018-07-04 18:13:52 +05:30
Shahidh K Muhammed
98c827449b
build: integrate circleci
integration with circleci for various workflows
2018-07-03 22:40:13 +05:30
Shahidh K Muhammed
e0a4ee88fe
cli: updates to support console versioning etc (#30)
* cli: fix version command and makefile

* cli: update console asset versioning format

* cli: change .realize from dir to file

* cli: update dependencies
2018-07-03 14:54:46 +05:30
Aravind Shankar
260d565e6e cli: tests for metadata commands (#18)
* [cli] added urlPrefix, consoleMode

* [cli] fix: down=all should throw error if down migration is missing

* [cli] added tests for metadata
2018-06-29 15:32:33 +05:30
shahidhk
c0bf346d60 [cli] fix tests 2018-06-28 14:14:01 +05:30