#### Issue: https://github.com/hasura/graphql-engine-mono/issues/2179
### Problem:
1. There are few warnings when tests are executed with cli if it is built with `race` flag
It is because of a race condition between stdin in migrate_delete for the prompt and stdout for other commands.
2. The integration test of the console used to behave as follows:
```
- Initially there won't be any wg.adds in wait-group so the second go-routine created in console-test (integration-test)
- It will send the interrupt signal to the channel before the server has been started
- So practically the server won't start
```
3. The read and write for consoleopts.WG has been happening in different go-routines without control or lock system(reading and writing into same memory location without lock system). So there is a chance of data-race(warning for data-race while integration tests are executed)
4. Data-race errors from `promptui` package
### Solution:
1. Use `--force` flag to avoid getting the input from the prompt in `migrate_delete_test.go`
2. Introduced two fields in server and console opts to let know other go-routines whether the server has been started or not
3. To avoid data-race above which are shared b/w go-routines to know the status of servers has been operated atomically.
4. using new package https://github.com/AlecAivazis/surveyhttps://github.com/hasura/graphql-engine-mono/pull/2231
GitOrigin-RevId: 387eb1be74f24dda34bb3588314b5a909adac227
Reference: https://hasurahq.slack.com/archives/C024GF6FCTH/p1629193174042700
Problem: The `metadata diff` is conventional where it takes the project metadata and w.r.t to the signs `---`, `+++`, it shows the diff output but it is not similar to git diff as taking the remote repo as base and `+` denotes it has, `-` not. So make the metadata diff output follow as git diff output
Solution: To change the order of arguments in diff function and assign `---` to server,`+++` to project, so that it will be similar to prev diff output
https://github.com/hasura/graphql-engine-mono/pull/2166
GitOrigin-RevId: bf35b5a3ef835fc64f590b65160c01c9a8c1382a
- add support for query_tags metadata object
- fix filename for cron_triggers metadata object
https://github.com/hasura/graphql-engine-mono/pull/1901
Co-authored-by: Aravind K P <8335904+scriptonist@users.noreply.github.com>
GitOrigin-RevId: d9266d723f06d11d92f156f70660e0122412e41a
- actions use-codegen
- Assert if starter kit files are created
- metadata apply
- Assert server metadata after apply
- assert that --dry-run flag will not modify metadata on server
- metadata clear
- Assert with server metadata that operation was successful.
- metadata inconsistency drop
- Assert with server metadata that operation was successful.
- metadata inconsistency status
- Assert for all possible states
- metadata reload
- Assert effect of operation with server
- migrate and seed subcommands
- Add tests for MSSQL sources
- Structure tests in such a way that it’s easy to add tests for new sources
https://github.com/hasura/graphql-engine-mono/pull/1693
Co-authored-by: Aravind K P <8335904+scriptonist@users.noreply.github.com>
GitOrigin-RevId: fcdbb806d105978a07487a40f7fa6e16b3ab8816
>
### Description
Update `go.mod` to allow other packages to import [v2.0.0 versions](https://blog.golang.org/v2-go-modules).
### Changelog
- [x] `CHANGELOG.md` is updated with user-facing content relevant to this PR. If no changelog is required, then add the `no-changelog-required` label.
### Affected components
- [x] CLI
https://github.com/hasura/graphql-engine-mono/pull/1584
GitOrigin-RevId: a5d17ad20289d1cd7217763f56ef3ba6552d69c4
It contains 2 flags
all - To delete all migrations locally and on database
version - To delete a single migration locally and on database
Usage :
`hasura migrate delete --all`
`hasura migrate delete --version <version_number>`
Additional fix :
The `migrate squash` will deletes the migration history on server after squashing if user opts to delete the migrations.
closes https://github.com/hasura/graphql-engine-mono/issues/292
closes https://github.com/hasura/graphql-engine/issues/5373
closes https://github.com/hasura/graphql-engine/issues/6434
Co-authored-by: Aravind K P <8335904+scriptonist@users.noreply.github.com>
GitOrigin-RevId: fa7ceae7a1970d6724fb601a147900e880ad2e6f
- modified the `dry-run` flag of metadata apply command.
- added new flag `o` which takes "json" or "yaml" as parameters for metadata apply command.
- added new flag `o` which takes "json" or "yaml" as parameters for metadata export command. It outputs the metadata from server to stdout in form of json or yaml and won't change the project's metadata.
- added deprecation warnings for `--from-file` flag
- added info message for describing change of behavior of `--dry-run` flag
- v3 metadata objects like `rest_endpoints` was also added to list of metadata objects in config v2 (fix)
- the order in which metadata objects are appended to metadata objects list matter when using `--dry-run` flag, refactored this order to match server metadata.
- `metadata apply` command can now accept json/yaml metadata from io pipe
- config v3 `metadata apply` didn't show any information to user when applied metadata is inconsistent, this is addressed.
- removed `github.com/ghodss/yaml` dependency from repo
- version metadata object was added twice during intialization (fix)
Co-authored-by: Aravind K P <8335904+scriptonist@users.noreply.github.com>
GitOrigin-RevId: 2316f519eb40645efd86ffee2a85d3c90543ec17
* gofmt
* cli: Update promptui to v0.8.0
This is in order to avoid the dependency on the abandoned
kingping.v3-unstable, which was causing dependency conflicts
compiling the pro/cli securelink code.
GitOrigin-RevId: 1264a86ca31b15b2a48c375cd123977084ac558e
- Created new job test_and_build_cli_migrations which runs after test_and_build_cli
- Build the cli-migrations and cli-migrations-v2 and save the images as tar image.
- Run the test defined in each workflow v1 and v2.
- Load the image that was built earlier in deploy step
When using console from CLI and a migration is created, the files are written first and an API call is made to Hasura to execute that migration. There was a bug which caused this file to remain when the API call to Hasura failed. This commit fixes the bug by deleting the files if Hasura API call fails and propagates the error to console.
* update docs install manifests to use stable branch
* change the link to master at some more places
Co-authored-by: Shahidh K Muhammed <muhammedshahid.k@gmail.com>
* cli: add --goto <version> flag to migrate command
* cli: fix error encoutered in goto, when on version -1
* do down migrations one step down
* reorganize code
* in case of up "gotos" make sure that all previous migration in migration chain are applied
* modify readUp and readDown functions to account for --goto use case
* refactor to clean up and adopt a better implementation addressing the comments
* remove empty error handling step
* return ErrNoChange when no migrations were applied
* respect m.stop() can panic so place conditional cases after that and add a couple other small fixes
* fix bug caused by early checking of versions
* fix to add migration_mode and nil version check
* add some more examples for migrate apply command
Co-authored-by: Aravind Shankar <face11301@gmail.com>
Co-authored-by: Shahidh K Muhammed <muhammedshahid.k@gmail.com>
- do not quit on server-cli version mismatch, show warning
- fix a bug in update-cli command which prevets updates to pre-release
versions
- if a console template for a particular version is not found, use the
latest template
* export metadata without nulls, empty arrays
* property tests for 'ReplaceMetadata' using QuickCheck
-> Derive Arbitrary class for 'ReplaceMetadata' dependant types
* reduce property test cases number to 30
QuickCheck generates the `ReplaceMetadata` value really large
for higher number test cases. Encoded JSON for such values is large and
consumes more memory. Thus, CI is giving up while running property
tests.
* circle-ci: Add property tests as saperate job
* add no command mode to tests
* add yaml.v2 to go mod
* remove indirect comment for yaml.v2 dependency
### Description
This PR introduces three new features:
- Support for a new migrations folder structure.
- Add `squash` command in preview.
- ~List of migrations on the Console and ability to squash them from console.~
#### New migrations folder structure
Starting with this commit, Hasura CLI supports a new directory structure for migrations folder and defaults to that for all new migrations created.
Each migration will get a new directory with the name format `timestamp_name` and inside the directory, there will be four files:
```bash
└── migrations
├── 1572237730898_squashed
│ ├── up.sql
│ ├── up.yaml
│ ├── down.yaml
│ └── down.sql
```
Existing files old migration format `timestamp_name.up|down.yaml|sql` will continue to work alongside new migration files.
#### Squash command
Lots of users have expressed their interest in squashing migrations (see #2724 and #2254) and some even built [their own tools](https://github.com/domasx2/hasura-squasher) to do squash. In this PR, we take a systematic approach to squash migrations.
A new command called `migrate squash` is introduced. Note that this command is in **PREVIEW** and the correctness of squashed migration is not guaranteed (especially for down migrations). From our tests, **it works for most use cases**, but we have found some issues with squashing all the down migrations, partly because the console doesn't generate down migrations for all actions.
Hence, until we add an extensive test suite for squashing, we'll keep the command in preview. We recommend you to confirm the correctness yourself by diffing the SQL and Metadata before and after applying the squashed migrations (we're also thinking about embedding some checks into the command itself).
```bash
$ hasura migrate squash --help
(PREVIEW) Squash multiple migrations leading upto the latest one into a single migration file
Usage:
hasura migrate squash [flags]
Examples:
# NOTE: This command is in PREVIEW, correctness is not guaranteed and the usage may change.
# squash all migrations from version 1572238297262 to the latest one:
hasura migrate squash --from 1572238297262
Flags:
--from uint start squashing form this version
--name string name for the new squashed migration (default "squashed")
--delete-source delete the source files after squashing without any confirmation
```
### Affected components
<!-- Remove non-affected components from the list -->
- CLI
### Related Issues
<!-- Please make sure you have an issue associated with this Pull Request -->
<!-- And then add `(close #<issue-no>)` to the pull request title -->
<!-- Add the issue number below (e.g. #234) -->
Close#2724, Close#2254,
### Solution and Design
<!-- How is this issue solved/fixed? What is the design? -->
<!-- It's better if we elaborate -->
For the squash command, a state machine is implemented to track changes to Hasura metadata. After applying each action on the metadata state, a list of incremental changes is created.
### Steps to test and verify
1. Open console via cli and create some migrations.
2. Run `hasura migrate squash --from <version>`
### Limitations, known bugs & workarounds
<!-- Limitations of the PR, known bugs and suggested workarounds -->
<!-- Feel free to delete these comment lines -->
- The `squash` command is in preview
- Support for squashing from the console is WIP
- Support for squashing migrations that are not committed yet is planned.
- Un-tracking or dropping a table will cause inconsistent squashed down migration since console doesn't generate correct down migration.
- If cascade setting is set to `true` on any of the metadata action, generated migration may be wrong
### Description
Adds a `metadata diff` command to show comparisons between two different sets of Hasura metadata.
```
# Show changes between server metadata and the exported metadata file:
hasura metadata diff
# Show changes between server metadata and that in local_metadata.yaml:
hasura metadata diff local_metadata.yaml
# Show changes between metadata from metadata.yaml and metadata_old.yaml:
hasura metadata diff metadata.yaml metadata_old.yaml
```
Also adds a `--dry-run` flag to `metadata apply` command which will print the diff and exit rather than actually applying the metadata.
### Affected components
- CLI
- Docs
### Related Issues
Close#3126, Close#3127
### Solution and Design
- Added `metadata_diff.go` and `metadata_diff_test.go`
### Steps to test and verify
```
hasura metadata export
# Make changes to migrations/metadata.yaml
hasura metadata diff
```
### Limitations, known bugs & workarounds
This is just a general-purpose diff.
A more contextual diff with the understanding of metadata can be added once #3072 is merged.
* allow customizing GraphQL root field names, close#981
* document v2 track_table API in reference
* support customising column field names in GraphQL schema
* [docs] add custom column fields doc in API reference
* add tests
* rename 'ColField' to 'ColumnField'
* embed column's graphql field in 'PGColumnInfo'
-> Value constructor of 'PGCol' is not exposed
-> Using 'parseJSON' to construct 'PGCol' in 'FromJSON' instances
* avoid using 'Maybe TableConfig'
* refactors & 'custom_column_fields' -> 'custom_column_names'
* cli-test: add configuration field in metadata export test
* update expected keys in `FromJSON` instance of `TableMeta`
* use `buildSchemaCacheFor` to update configuration in v2 track_table
* remove 'GraphQLName' type and use 'isValidName' exposed from parser lib
* point graphql-parser-hs library git repo to hasura
* support 'set_table_custom_fields' query API & added docs and tests
These changes also add a new type, PGColumnType, between PGColInfo and
PGScalarType, and they process PGRawColumnType values into PGColumnType
values during schema cache generation.
query templates is a little known feature that lets you template rql
queries and serve them as rest apis. This is not relevant anymore
given the GraphQL interface and getting rid of it reduces the dev
time when adding features in few subsystems.
This feature has never been used outside hasura's internal projects or
documented or exposed through console and hence can safely be removed.
This PR builds console static assets into the server docker image at `/srv/console-assets`. When env var `HASURA_GRAPHQL_CONSOLE_ASSETS_DIR=/srv/console-assets` or flag `--console-assets-dir=/srv/console-assets` is set on the server, the files in this directory are served at `/console/assets/*`.
The console html template will have a variable called `cdnAssets: false` when this flag is set and it loads assets from server itself instead of CDN.
The assets are moved to a new bucket with a new naming scheme:
```
graphql-engine-cdn.hasura.io/console/assets/
/common/{}
/versioned/<version/{}
/channel/<channel>/<version>/{}
```
Console served by CLI will still load assets from CDN - will fix that in the next release.
The word "reset" is little ambiguous (reset to scratch or reset to last known consistent state). metadata reset clears the all the metadata to mimic a freshly installed hasura instance. This actually invokes the clear_metadata API in the backend and hence should also be called metadata clear for perfect clarity.
All references to metadata reset should be changed to metadata clear.
Rename the admin secret key header used to access GraphQL engine from X-Hasura-Access-Key to X-Hasura-Admin-Secret.
Server CLI and console all support the older flag but marks it as deprecated.
