Commit Graph

125 Commits

Author SHA1 Message Date
Rakesh Emmadi
dc31b835e1
allow reusing Postgres scalars in custom types & actions (close #4125) (#4333)
* allow re-using Postgres scalars in custom types, close #4125

* add pytest tests

* update CHANGELOG.md

* add a doc pointer for reusable postgres scalars

* document the code, improve the CHANGELOG entry

As suggested by @lexi-lambda

* a bit more source code documentation, use WriterT to collect reused scalars

* Apply suggestions from code review

Co-Authored-By: Marion Schleifer <marion@hasura.io>

* improve doc for Postgres scalars in custom graphql types

* Add some more references to Note; fix Haddock syntax

Also a few very minor tweaks:
  * Use HashSet instead of [] more pervasively
  * Export execWriterT from Hasura.Prelude
  * Use pattern guards in multi-way if
  * Tweak a few names/comments

* Pull buildActions out of buildAndCollectInfo, use buildInfoMap

* Tweak wording in documentation

* incorporate changes in console code

* account Postgres scalars for action input arguments

-> Avoid unnecessary 'throw500' in making action schema

* Review changes

Co-authored-by: Marion Schleifer <marion@hasura.io>
Co-authored-by: Alexis King <lexi.lambda@gmail.com>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>
2020-04-15 17:33:13 +05:30
nizar-m
0e666a9203
Fix catalog version for v1.1.1 (close #4354) (#4355)
* Fix catalog version for v1.1.1

* Remove entries of removed tables from hdb_catalog

While downgrading catalog version from 32 -> 31, not removing entries
in hdb_table and hdb_relationship for the tables that are removed in
the downgrade, results in incosistent schema, when the server with
downgraded version is started. This should probably be handled in
a better fashion.

With the change in this commit, the server is able to successfully
start with downgraded catalog version 31.

* Test downgrade command along with upgrade tests
2020-04-09 19:57:59 -05:00
Shahidh K Muhammed
1c4f203e4a
update manifests to v1.1.1 (#4220) 2020-03-28 16:45:43 +05:30
Brandon Simmons
eafdda4f33
Add missing migration to initialise.sql for #3839 (#4212) 2020-03-26 23:50:45 -05:00
Phil Freeman
4ce4eb1d2f
Remove any outstanding hdb_views (fixes #3940) (#3965) 2020-03-26 15:42:33 -05:00
Brandon Simmons
c425b554b8
server(events): utilize proper backpressure scheme (close #3839) (#4013)
* Test working through a backlog of change events

* Use a slightly more performant threaded http server in eventing pytests

This helped locally but not on CI it seems...

* Rework event processing for backpressure. Closes #3839

With loo low `HASURA_GRAPHQL_EVENTS_FETCH_INTERVAL` and/or slow webhooks
and/or too small `HASURA_GRAPHQL_EVENTS_HTTP_POOL_SIZE` we might
previously check out events from the DB faster than we can service them,
leading to space leaks, weirdness, etc.

Other changes:
- avoid fetch interval sleep latency when we previously did a non-empty
  fetch
- prefetch event batch while http pool is working
- warn when it appears we can't keep up with events being generated
- make some effort to process events in creation order so we don't
  starve older ones.

ALSO NOTE: HASURA_GRAPHQL_EVENTS_FETCH_INTERVAL changes semantics
slightly, since it only comes into play after an empty fetch. The old
semantics weren't documented in detail, so I think this is fine.
2020-03-11 11:57:31 +05:30
Shahidh K Muhammed
4015e512db
tag release v1.2.0-beta.2 (#4028) 2020-03-05 22:49:17 +05:30
Shahidh K Muhammed
da5e4498d3
tag release v1.2.0-beta.1 (#3966) 2020-02-27 16:55:12 +05:30
Phil Freeman
8df0151f99
Maintain downgrade commands in a text file, update when tagging (#3933) 2020-02-24 20:03:56 +05:30
Vamshi Surabhi
b84db36ebb
allow custom mutations through actions (#3042)
* basic doc for actions

* custom_types, sync and async actions

* switch to graphql-parser-hs on github

* update docs

* metadata import/export

* webhook calls are now supported

* relationships in sync actions

* initialise.sql is now in sync with the migration file

* fix metadata tests

* allow specifying arguments of actions

* fix blacklist check on check_build_worthiness job

* track custom_types and actions related tables

* handlers are now triggered on async actions

* default to pgjson unless a field is involved in relationships, for generating definition list

* use 'true' for action filter for non admin role

* fix create_action_permission sql query

* drop permissions when dropping an action

* add a hdb_role view (and relationships) to fetch all roles in the system

* rename 'webhook' key in action definition to 'handler'

* allow templating actions wehook URLs with env vars

* add 'update_action' /v1/query type

* allow forwarding client headers by setting `forward_client_headers` in action definition

* add 'headers' configuration in action definition

* handle webhook error response based on status codes

* support array relationships for custom types

* implement single row mutation, see https://github.com/hasura/graphql-engine/issues/3731

* single row mutation: rename 'pk_columns' -> 'columns' and no-op refactor

* use top level primary key inputs for delete_by_pk & account select permissions for single row mutations

* use only REST semantics to resolve the webhook response

* use 'pk_columns' instead of 'columns' for update_by_pk input

* add python basic tests for single row mutations

* add action context (name) in webhook payload

* Async action response is accessible for non admin roles only if
  the request session vars equals to action's

* clean nulls, empty arrays for actions, custom types in export metadata

* async action mutation returns only the UUID of the action

* unit tests for URL template parser

* Basic sync actions python tests

* fix output in async query & add async tests

* add admin secret header in async actions python test

* document async action architecture in Resolve/Action.hs file

* support actions returning array of objects

* tests for list type response actions

* update docs with actions and custom types metadata API reference

* update actions python tests as per #f8e1330

Co-authored-by: Tirumarai Selvan <tirumarai.selvan@gmail.com>
Co-authored-by: Aravind Shankar <face11301@gmail.com>
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
2020-02-13 23:08:23 +05:30
Phil Freeman
94102c0460
Add downgrade command (close #1156) (#3760)
* Add downgrade command

* Add docs per @lexi-lambda's suggestions

* make tests pass

* Update hdb_version once, from Haskell

* more work based on feedback

* Improve the usage message

* Small docs changes

* Test downgrades exist for each tag

* Update downgrading.rst

* Use git-log to find tags which are ancestors of the current commit

Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
2020-02-07 16:33:12 +05:30
Alexis King
8ef205fba5 Explicitly invalidate enum values when metadata is reloaded
This fixes #3759. Also, while we’re at it, also improve the way
invalidations are synced across instances so enums and remote schemas
are appropriately reloaded by the schema syncing process.
2020-01-30 18:17:29 -06:00
Phil Freeman
9ed8f717a7 remove hdb_views for inserts (#3598)
* WIP: Remove hdb_views for inserts

* Show failing row in check constraint error

* Revert "Show failing row in check constraint error"

This reverts commit dd2cac29d0.

* Use the better query plan

* Simplify things

* fix cli test

* Update downgrading.rst

* remove 1.1 asset for cli
2020-01-16 10:23:28 +05:30
Alexis King
feb4a05618
Add fast path for server internal metadata migrations (#3686) 2020-01-14 01:09:10 -06:00
Rikin Kachhia
db6b2aa378
add computed fields support on console (close #3203 #3565) (#3522) 2020-01-13 12:41:30 +05:30
Alexis King
d9bb9f2bed Alter the type of hdb_catalog columns that reference SQL identifiers
As explained in the note included in the diff, this can lead to
dramatically better query plans, and it seems to be especially important
for versions of Postgres <12.
2020-01-08 16:45:54 -06:00
Alexis King
25c5f97de2 Fix new hdb_table_info_agg query to fetch column base types
We mostly want to do this to make queries against information_schema
tables work, which the console cares about. information_schema tables
use types like sql_identifier, which have no corresponding array types
defined! Therefore, in order to generate valid queries for _in and _nin
conditions, we need to treat them as their base types, instead.
2020-01-08 16:45:54 -06:00
Alexis King
c322e8a5d4 Use a significantly more efficient table_info_agg view
Also, use the view in Schema.Diff to share some more logic.
2020-01-08 16:45:46 -06:00
Rakesh Emmadi
9b8e6b42d1 functions can access session info via input arg (close #2322) (#3143) 2019-11-20 12:17:06 +05:30
Tirumarai Selvan
3cad1319c4 improve event fetch query (#3236) 2019-11-13 12:59:19 +05:30
Rakesh Emmadi
6d92e4f9db save permissions, relationships and collections in catalog with 'is_system_defined' explicitly (#3165)
* save permissions, relationships and collections in catalog with 'is_system_defined'
* Use common stanzas in the .cabal file
* Refactor migration code into lib instead of exe
* Add new server test suite that exercises migrations
* Make graphql-engine clean succeed even if the schema does not exist
2019-10-21 11:01:05 -05:00
Rakesh Emmadi
d8d21c1487 support computed fields returning scalars or set of tables (close #1387) (#2917) 2019-10-18 13:59:47 +05:30
Ajeet D'Souza
a66fb42ce2 Make catalog metadata migrations work on all schema versions (fix #2826) (#2379)
* Separate DB and metadata migrations
* Refactor Migrate.hs to generate list of migrations at compile-time
* Replace ginger with shakespeare to improve performance
* Improve migration log messages
2019-10-11 00:13:57 -05:00
Rakesh Emmadi
b0d60ec14f fix hdb_catalog.hdb_column view (fix #3083) (#3084) 2019-10-10 10:11:17 -05:00
Rakesh Emmadi
9bd5826020 allow customising graphql schema for a table (close #981) (#2509)
* allow customizing GraphQL root field names, close #981

* document v2 track_table API in reference

* support customising column field names in GraphQL schema

* [docs] add custom column fields doc in API reference

* add tests

* rename 'ColField' to 'ColumnField'

* embed column's graphql field in 'PGColumnInfo'

-> Value constructor of 'PGCol' is not exposed
-> Using 'parseJSON' to construct 'PGCol' in 'FromJSON' instances

* avoid using 'Maybe TableConfig'

* refactors & 'custom_column_fields' -> 'custom_column_names'

* cli-test: add configuration field in metadata export test

* update expected keys in `FromJSON` instance of `TableMeta`

* use `buildSchemaCacheFor` to update configuration in v2 track_table

* remove 'GraphQLName' type and use 'isValidName' exposed from parser lib

* point graphql-parser-hs library git repo to hasura

* support 'set_table_custom_fields' query API & added docs and tests
2019-09-19 10:17:36 +05:30
Ajeet D'Souza
99174cca9b propagate Postgres table comments to GraphQL schema descriptions (close #446) (#2397) 2019-09-16 20:51:11 -05:00
Rakesh Emmadi
e5eb0c4f34 fix row comparison operator in event triggers (fix #2036) (#2868)
Update trigger is failing if any json/geometry columns are present in
event payload rows. Use '*<>' operator instead of '<>' to compare the
internal binary representation of rows if '<>' doesn’t work.
2019-09-12 18:22:01 -05:00
Rakesh Emmadi
d9fb0f8780 use named notation for function arguments if any argument is not specified (fix #2730) (#2777)
* use positional arguments in SQL functions
* only allow omitting set of last arguments in functions
* disallow omitting of a non default argument in functions
2019-08-28 14:27:15 -05:00
Tirumarai Selvan
67d3eb3e27 unlock only locked rows during startup (#2778) 2019-08-26 19:41:38 -05:00
Alexis King
ed26da59a6 Add support for GraphQL enum types via enum table references
These changes also add a new type, PGColumnType, between PGColInfo and
PGScalarType, and they process PGRawColumnType values into PGColumnType
values during schema cache generation.
2019-08-26 00:54:56 -05:00
Rakesh Emmadi
db24d3248f schema sync process will not generate garbage data (close #2542) (#2585)
schema syncing logic now requires only a single row in `hdb_schema_update_event`.
2019-07-31 14:31:48 +05:30
Vamshi Surabhi
4facb3c780 remove support for query templates (#2560)
query templates is a little known feature that lets you template rql
queries and serve them as rest apis. This is not relevant anymore
given the GraphQL interface and getting rid of it reduces the dev
time when adding features in few subsystems.

This feature has never been used outside hasura's internal projects or
documented or exposed through console and hence can safely be removed.
2019-07-23 19:11:34 +00:00
Alexis King
fb3794c31a Support Node v12.x and fix tests on macOS (#2455)
* Bump node-sass to version ^4.12.0

This avoids sass/node-sass#2632, which causes compilation failures on
Node v12.x.

* Do most of the work in /pg_dump in Haskell instead of shell

The shell version caused problems on non-Linux systems since it relied
on the non-POSIX -i option for sed, which works slightly differently on
BSD and macOS.
2019-07-03 14:07:13 +05:30
Ajeet D'Souza
b184ce0188 add server version to console html (close #2444) (#2452) 2019-07-02 12:44:29 +00:00
Rakesh Emmadi
31cf7314e2 better change detection when altering foreign key constraints (fix #2060) (#2252)
foreign keys are considered to be dropped only if constraint oid changes
2019-06-03 15:51:55 +05:30
Rikin Kachhia
7ab9d5eaeb update console favicons to png (#2287) 2019-05-29 10:03:29 +05:30
Rishichandra Wawhal
3ef1219fa0 change default endpoint to v1/graphql; remove semver checks (#2181) 2019-05-16 14:38:44 +05:30
Shahidh K Muhammed
76ceb707f4
bundle console assets into server (close #516, close #521, close #2130) (#2192)
This PR builds console static assets into the server docker image at `/srv/console-assets`. When env var `HASURA_GRAPHQL_CONSOLE_ASSETS_DIR=/srv/console-assets` or flag `--console-assets-dir=/srv/console-assets` is set on the server, the files in this directory are served at `/console/assets/*`.

The console html template will have a variable called `cdnAssets: false` when this flag is set and it loads assets from server itself instead of CDN.

The assets are moved to a new bucket with a new naming scheme:

```
graphql-engine-cdn.hasura.io/console/assets/
   /common/{}
   /versioned/<version/{}
   /channel/<channel>/<version>/{}
```

Console served by CLI will still load assets from CDN - will fix that in the next release.
2019-05-16 13:15:29 +05:30
Rakesh Emmadi
0b210cc245 support allow-list for graphql queries (closes #989) (#2075) 2019-05-16 11:43:25 +05:30
Tirumarai Selvan
36e49ea75b manual event triggers (close #1687) (#2077) 2019-05-13 15:11:07 +05:30
Rakesh Emmadi
204cd3514b optimise server startup time (close #1430) (#2120)
1. Reuses postgres connections during startup which reduces the overhead of opening and closing connections. 
2. Faster schema cache building. This is done by fetching all the required data in a single sql statement.
2019-05-08 13:06:42 +05:30
Shahidh K Muhammed
71cf017197 add an api to dump postgres schema (close #1939) (#1967) 2019-04-30 14:04:08 +05:30
Rakesh Emmadi
3708f95036 fix input argument types for custom SQL functions (close #1952) (#1953) 2019-04-08 12:28:56 +05:30
Tirumarai Selvan
36781199d0 breaking: drop id from event_triggers table (fix #1840) (#1857) 2019-03-25 22:40:52 +05:30
Rikin Kachhia
1ba382193e update console and docs favicons (close #1485) (#1760) 2019-03-25 15:42:56 +05:30
Rakesh Emmadi
5bafdce9a3 fix delete mutation returning incorrect data (fix #1794) (fix #1763) (#1827)
From `alpha-40` we've been using a `WHERE` clause to fetch required rows and generate mutation response. This has a few limitations like the requirement of a primary key/unique constraint. This also returns inconsistent data on `delete` mutation as mentioned in #1794. 
Now, we're using `VALUES (..)` (refer [here](https://www.postgresql.org/docs/current/sql-values.html)) expression to form virtual table rows in `SQL` to generate mutation response.

Internal changes:-
- Not to use primary key/unique constraint columns:-
  - Revert back to `ConstraintName` from `TableConstraint` in `TableInfo` type
  - Remove `tcCols` field in `TableConstraint` type
  - Modify `table_info.sql` and `fetchTableMeta` function `SQL`
- A test case to perform `delete` mutation and returning relational objects.
2019-03-22 12:38:42 +05:30
Rakesh Emmadi
e32f5a1fb1 sync metadata cache across multiple instances connected to same db (closes #1182) (#1574)
1. Haskel library `pg-client-hs` has been updated to expose a function that helps listen to `postgres` notifications over a `channel` in this [PR](https://github.com/hasura/pg-client-hs/pull/5)
2. The server records an event in a table `hdb_catalog.hdb_cache_update_event` whenever any `/v1/query` (that changes metadata) is requested. A trigger notifies a `cache update` event via `hasura_cache_update` channel
3. The server runs two concurrent threads namely `listener` and `processor`. The `listener` thread listens to events on `hasura_cache_update` channel and pushed into a `Queue`. The `processor` thread fetches events from that `Queue` and processes it. Thus server rebuilds schema cache from database and updates.
2019-03-12 11:16:27 +05:30
Rakesh Emmadi
5f274b5527 fix mutation returning when relationships are present (fix #1576) (#1703)
If returning field contains nested selections then mutation is performed in two steps
1. Mutation is performed with returning columns of any primary key and unique constraints
2. returning fields are queried on rows returned by selecting from table by filtering with column values returned in Step 1.

Since mutation takes two courses based on selecting relations in returning field, it is hard to maintain sequence of prepared arguments (PrepArg) generated while resolving returning field. So, we're using txtConverter instead of prepare to resolve mutation fields.
2019-03-07 15:54:07 +05:30
Vamshi Surabhi
f794653b69
update event triggers on rename operations (#1684) 2019-03-01 19:29:24 +05:30
Rakesh Emmadi
6c20ca8a55 allow renaming tables, columns and relationships (close #79) (#1542) 2019-03-01 14:47:22 +05:30
nizar-m
f83a8e591f rename access-key to admin-secret (close #1347) (#1540)
Rename the admin secret key header used to access GraphQL engine from X-Hasura-Access-Key to X-Hasura-Admin-Secret.

Server CLI and console all support the older flag but marks it as deprecated.
2019-02-14 15:07:47 +05:30
Shahidh K Muhammed
11e7c3f9d6 add anonymous telemetry (#1401) 2019-01-28 19:25:28 +05:30
Tirumarai Selvan
e590144d02 send session variables in event trigger payload (close #1328) (#1458) 2019-01-28 11:42:52 +05:30
Rakesh Emmadi
0bf2457e23 allow exposing postgres functions through GraphQL interface (close #333) (#1073) 2019-01-25 09:01:54 +05:30
Rakesh Emmadi
e2fe078b89 fix fetching primary key columns for non table owners (close #1269) (#1300) 2019-01-04 16:36:05 +05:30
Vamshi Surabhi
380fdad468 update constraint enum types when a new constraint is added (#1287)
* update metadata when constraints on a table are altered, fix #240

* capture only unique or primary constraints in tableinfo
2019-01-03 09:28:12 +05:30
Vamshi Surabhi
ea4d2644e8
local console during development (#1252)
* console now works on local builds of the server

1. local console assets can be served at /static/ by a build time flag
'local-console'. This can be set with stack as follows:
   `stack build --flag graphql-engine:local-console`
2. the --root-dir option is removed which was used as a temporary hack
for serving graphiql
3. remove server's graphiql source code
2018-12-21 13:21:02 +05:30
Vamshi Surabhi
c28fbd3f98
faster retrieval of table information from postgres (#1235) 2018-12-19 16:11:06 +05:30
Rakesh Emmadi
b5bbb966f2 add consolePath in console.html template (#1222)
Server templates `consolePath` key in `window.__env` object in console html template.
If server is hit at `/console/table/author` then `window.__env` in served html looks like
```
{
   consoleMode: "server",
   urlPrefix: "/console",
   consolePath: "/console/table/author",
   isAccessKeySet: true
}
```
2018-12-18 15:09:01 +05:30
Vamshi Surabhi
db1f6c1a39
remove dependency on 'first' aggregate, closes #1085 (#1089) 2018-11-27 21:40:54 +05:30
Anon Ray
512ee6fb9f adds basic support for remote schemas/schema stitching (#952) 2018-11-23 18:32:46 +05:30
Tirumarai Selvan
317efb81f1 event triggers: take webhook url from env (close #966) (#968) 2018-11-14 12:43:01 +05:30
Tirumarai Selvan
d4d31838cb quote function and trigger names, allow hyphen in trigger name (#1012) 2018-11-13 11:28:55 +05:30
Tirumarai Selvan
810b440089 trigger webhooks on column level changes instead of row (close #547, close #680) (#550) 2018-10-25 12:52:51 +05:30
Rakesh Emmadi
d57be587b3 format schema name and table name as identifiers in event triggers (fix #639) (#644) 2018-10-09 12:09:20 +05:30
Rakesh Emmadi
fc7ea9213c fix non-admin insert returns null column values when query affects zero rows in postgres (fix #563) (#565)
Insert trigger function: If query affects no rows then return `null`

Insert trigger function is modified to have 
 `IF r IS NULL THEN RETURN null; ELSE RETURN r; END IF;` in return statement.
2018-09-29 11:12:47 +05:30
Karthik Venkateswaran
a9dc756a40 update console variables and contributing guide (close #517) (#538)
Changes `consoleMode` variable templated by HGE to `server` and modified console to respect in the same way and adds a patch to continue working with the older HGEs

- [x] Server
- [x] Console
2018-09-27 17:10:32 +05:30
Tirumarai Selvan
2cd2b23b2d add custom headers for webhooks, refactor retry logic (#419) 2018-09-24 17:20:11 +05:30
Rakesh Emmadi
e6c5aa5b43 indicate access key is set in the console context (close #426) (#447) 2018-09-14 18:57:46 +05:30
Anon Ray
4b4c675ab7 better handling for pgcrypto extension (fix #338) (#423) 2018-09-14 17:11:51 +05:30
Tirumarai Selvan
e402d11318 update catalog version to 2 (#357) 2018-09-05 17:12:20 +05:30
Tirumarai Selvan
82e09efce6 add event triggers (#329) 2018-09-05 16:56:46 +05:30
Rakesh Emmadi
400a0e3f16 server: add v1/version api, fix #34 (#37) 2018-07-03 21:04:25 +05:30
Rakesh Emmadi
128ed2388b server: console is served at /console Closes #16 (#22) 2018-06-29 16:35:09 +05:30
Vamshi Surabhi
530027cf20 move raven into graphql-engine repo 2018-06-28 00:32:00 +05:30