Commit Graph

147 Commits

Author SHA1 Message Date
Vishnu Bharathi P
58c44f55dd Merge oss/master onto mono/main
GitOrigin-RevId: 1c8c4d60e033c8a0bc8b2beed24c5bceb7d4bcc8
2020-11-12 22:37:19 +05:30
Vishnu Bharathi P
666058ab7f oss: renames dot files and folders
GitOrigin-RevId: 540aeec3be091e1cfb7b05a988f50445534ed663
2020-11-12 22:37:19 +05:30
Auke Booij
3bcde3d4b8
server: metadata separation: reorganize metadata types (#6103)
https://github.com/hasura/graphql-engine/pull/6103
2020-11-03 18:01:33 +00:00
Auke Booij
0540b279db
server: make more use of hlint (#6059)
https://github.com/hasura/graphql-engine/pull/6059
2020-10-28 16:40:33 +00:00
Alexis King
bf466e3b63
server: Fix fine-grained incremental cache invalidation (fix #3759) (#6027)
This issue was very tricky to track down, but fortunately easy to fix.
The interaction here is subtle enough that it’s difficult to put into
English what would go wrong in what circumstances, but the new unit test
captures precisely that interaction to ensure it remains fixed.
2020-10-27 14:52:19 -05:00
Antoine Leblanc
a8ed6a82e2
server: move Hasura.SQL to Hasura.Backends.Postgres (#6053)
https://github.com/hasura/graphql-engine/pull/6053
2020-10-27 13:53:49 +00:00
Sameer Kolhar
10f41e7559
server: accept only non-negative integers for batch size and refetch interval (close #5653) (#5759)
https://github.com/hasura/graphql-engine/pull/5759
2020-09-17 10:56:41 +00:00
Rakesh Emmadi
4ce6002af2
support customizing JWT claims (close #3485) (#3575)
* improve jsonpath parser to accept special characters and property tests for the same

* make the JWTClaimsMapValueG parametrizable

* add documentation in the JWT file

* modify processAuthZHeader

Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
Co-authored-by: Marion Schleifer <marion@hasura.io>
2020-08-31 22:10:01 +05:30
Brandon Simmons
2638136f58 Restore AuthSpec tests erroneously removed in 8904e063
Said commit had a lot of whitespace, formatting, and trivial
refactorings. We should be mindful that these have a real cost in terms
of review time and potential for bugs to be introduced.

a weeder pass in CI would have caught this.
2020-08-20 13:31:14 -04:00
Phil Freeman
b2561a719b
server: set hasura.tracecontext in RQL mutations [#5542] (#5555)
* server: set hasura.tracecontext in RQL mutations [#5542]

* Update test suite

Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
2020-08-10 12:34:24 -07:00
Brandon Simmons
1d4ec4eafb Support only the bounded cache, with default HASURA_GRAPHQL_QUERY_PLAN_CACHE_SIZE of 4000. Closes #5363 2020-07-28 19:02:44 -04:00
Brandon Simmons
3a6b2ec744 Bugfix to support 0-size HASURA_GRAPHQL_QUERY_PLAN_CACHE_SIZE
Also some minor refactoring of bounded cache module:
 - the maxBound check in `trim` was confusing and unnecessary
 - consequently trim was unnecessary for lookupPure

Also add some basic tests
2020-07-27 18:40:17 -04:00
Brandon Simmons
2eab6a89aa Fix latency buckets for telemetry data
These must have gotten messed up during a refactor. As a consequence
almost all samples received so far fall into the single erroneous 0 to
1K seconds (originally supposed to be 1ms?) bucket.

I also re-thought what the numbers should be, but these are still
arbitrary and might want adjusting in the future.
2020-07-22 12:29:38 -04:00
Phil Freeman
0dddbe9e9d
Add MonadTrace and MonadExecuteQuery abstractions (#5383)
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
2020-07-15 16:10:48 +05:30
Lyndon Maydwell
24592a516b
Pass environment variables around as a data structure, via @sordina (#5374)
* Pass environment variables around as a data structure, via @sordina

* Resolving build error

* Adding Environment passing note to changelog

* Removing references to ILTPollerLog as this seems to have been reintroduced from a bad merge

* removing commented-out imports

* Language pragmas already set by project

* Linking async thread

* Apply suggestions from code review

Use `runQueryTx` instead of `runLazyTx` for queries.

* remove the non-user facing entry in the changelog

Co-authored-by: Phil Freeman <paf31@cantab.net>
Co-authored-by: Phil Freeman <phil@hasura.io>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
2020-07-14 12:00:58 -07:00
Brandon Simmons
5cd30e073a
Disable downgrade test for now. Closes #5273 (#5291) 2020-07-06 10:08:26 +05:30
Vamshi Surabhi
6fc404329a
generalize query execution logic on Postgres (#5110)
* generalize PGExecCtx to support specialized functions for various operations

* fix tests compilation

* allow customising PGExecCtx when starting the web server
2020-06-16 23:14:59 +05:30
Brandon Simmons
5e37350561 Refactor and unit test authentication code paths (closes #4736)
The bulk of changes here is some shifting of code around and a little
parameterizing of functions for easier testing.

Also: comments, some renaming for clarity/less-chance-for-misue.
2020-06-08 13:10:58 -04:00
Tirumarai Selvan
c0d2bc6653
Remote Joins: Create relationships across database and remote schemas (#2392)
add remote joins: Create relationships across database and remote schemas (#2392)

Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>

Co-authored-by: Chris Done <chrisdone@gmail.com>
Co-authored-by: Chris Done <github@chrisdone.com>
Co-authored-by: wawhal <rishichandra.wawhal@gmail.com>
Co-authored-by: Aravind Shankar <aravind@hasura.io>
Co-authored-by: Brandon Simmons <brandon.m.simmons@gmail.com>
Co-authored-by: Rishichandra Wawhal <rishi@hasura.io>
Co-authored-by: Brandon Simmons <brandon@hasura.io>
Co-authored-by: nizar-m <19857260+nizar-m@users.noreply.github.com>
Co-authored-by: Praveen Durairaju <praveend.web@gmail.com>
Co-authored-by: rakeshkky <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Anon Ray <rayanon004@gmail.com>
Co-authored-by: Shahidh K Muhammed <shahidh@hasura.io>
Co-authored-by: soorajshankar <soorajshankar@users.noreply.github.com>
Co-authored-by: Sooraj Sanker <sooraj@Soorajs-MacBook-Pro.local>
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>
2020-05-27 20:32:58 +05:30
Brandon Simmons
ff62d5e0bf Migrate to GHC 8.10, upgrade dependencies. Closes #4517
This also seems to squash a stubborn space leak we see with
subscriptions (linking to canonical #3388 for reference).

This may also fix some of the "Unexpected exception" websockets
exceptions we are now surfacing (see e.g. #4344)

Also: dev.sh: fix hpc reporting

Initial work on this done by Vamshi.
2020-05-13 19:13:02 -04:00
Tirumarai Selvan
cc8e2ccc78
Scheduled triggers (close #1914) (#3553)
server: add scheduled triggers 

Co-authored-by: Alexis King <lexi.lambda@gmail.com>
Co-authored-by: Marion Schleifer <marion@hasura.io>
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>
2020-05-13 18:03:16 +05:30
Rakesh Emmadi
d52bfcda4e
backend only insert permissions (rfc #4120) (#4224)
* move user info related code to Hasura.User module

* the RFC #4120 implementation; insert permissions with admin secret

* revert back to old RoleName based schema maps

An attempt made to avoid duplication of schema contexts in types
if any role doesn't possess any admin secret specific schema

* fix compile errors in haskell test

* keep 'user_vars' for session variables in http-logs

* no-op refacto

* tests for admin only inserts

* update docs for admin only inserts

* updated CHANGELOG.md

* default behaviour when admin secret is not set

* fix x-hasura-role to X-Hasura-Role in pytests

* introduce effective timeout in actions async tests

* update docs for admin-secret not configured case

* Update docs/graphql/manual/api-reference/schema-metadata-api/permission.rst

Co-Authored-By: Marion Schleifer <marion@hasura.io>

* Apply suggestions from code review

Co-Authored-By: Marion Schleifer <marion@hasura.io>

* a complete iteration

backend insert permissions accessable via 'x-hasura-backend-privilege'
session variable

* console changes for backend-only permissions

* provide tooltip id; update labels and tooltips;

* requested changes

* requested changes

- remove className from Toggle component
- use appropriate function name (capitalizeFirstChar -> capitalize)

* use toggle props from definitelyTyped

* fix accidental commit

* Revert "introduce effective timeout in actions async tests"

This reverts commit b7a59c19d6.

* generate complete schema for both 'default' and 'backend' sessions

* Apply suggestions from code review

Co-Authored-By: Marion Schleifer <marion@hasura.io>

* remove unnecessary import, export Toggle as is

* update session variable in tooltip

* 'x-hasura-use-backend-only-permissions' variable to switch

* update help texts

* update docs

* update docs

* update console help text

* regenerate package-lock

* serve no backend schema when backend_only: false and header set to true

- Few type name refactor as suggested by @0x777

* update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* fix a merge bug where a certain entity didn't get removed

Co-authored-by: Marion Schleifer <marion@hasura.io>
Co-authored-by: Rishichandra Wawhal <rishi@hasura.io>
Co-authored-by: rikinsk <rikin.kachhia@gmail.com>
Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
2020-04-24 14:40:53 +05:30
Rakesh Emmadi
6f100e0009
improve debug information in actions errors response (close #4031) (#4432)
* config options for internal errors for non-admin role, close #4031

More detailed action debug info is added in response 'internal' field

* add docs

* update CHANGELOG.md

* set admin graphql errors option in ci tests, minor changes to docs

* fix tests

Don't use any auth for sync actions error tests. The request body
changes based on auth type in session_variables (x-hasura-auth-mode)

* Apply suggestions from code review

Co-Authored-By: Marion Schleifer <marion@hasura.io>

* use a new sum type to represent the inclusion of internal errors

As suggested in review by @0x777
-> Move around few modules in to specific API folder
-> Saperate types from Init.hs

* fix tests

Don't use any auth for sync actions error tests. The request body
changes based on auth type in session_variables (x-hasura-auth-mode)

* move 'HttpResponse' to 'Hasura.HTTP' module

* update change log with breaking change warning

* Update CHANGELOG.md

Co-authored-by: Marion Schleifer <marion@hasura.io>
Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
2020-04-24 13:25:51 +05:30
Toan Nguyen
5f84669568
server: support single $ as root json path (#4482)
* support single $ json path

* support sql query with root json path

* update changelog

* update changelog

Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
2020-04-21 13:36:11 +05:30
Toan Nguyen
15c0ebf1ef
allow special characters in json path's property name (close #3890) (#3892)
* allow underscore prefix and special characters in json path

* server: Rewrite/refactor JSONPath parser

The JSONPath parser is also rewritten, the previous implementation
was written in a very explicitly “recursive descent” style, but the whole
point of using attoparsec is to be able to backtrack! Taking advantage
of the combinators makes for a much simpler parser.

Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
Co-authored-by: Alexis King <lexi.lambda@gmail.com>
Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>
Co-authored-by: Shahidh K Muhammed <shahidh@hasura.io>
2020-04-20 14:25:09 +05:30
Karthikeyan Chinnakonda
a26bc80496
accept a new argument claims_namespace_path in JWT config (#4365)
* add new optional field `claims_namespace_path` in JWT config

* return value when empty array is found in executeJSONPath

* update the docs related to claims_namespace_path

* improve encodeJSONPath, add property tests for parseJSONPath

* throw error if both claims_namespace_path and claims_namespace are set

* refactor the Data.Parser.JsonPath to Data.Parser.JSONPathSpec

* update the JWT docs

Co-Authored-By: Marion Schleifer <marion@hasura.io>

Co-authored-by: Marion Schleifer <marion@hasura.io>
Co-authored-by: rakeshkky <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Tirumarai Selvan <tirumarai.selvan@gmail.com>
2020-04-16 12:15:21 +05:30
Rakesh Emmadi
fd6535b861
option to reload remote schemas in 'reload metadata' (fix #3792, #4117) (#4141)
* option to reload remote schemas in 'reload_metadata' API, fix #3792, #4117

* add tests

* update changelog

* update docs api reference for 'reload_metadata'

* send reload_remote_schemas: true with the reload_metadata query

* add reload remote schemas checkbox; minor refactor

* Add a Note about cache invalidation and inconsistent metadata objects

* Small pluralization agreement tweak in docs

* Remove duplicated line in CHANGELOG

* no-op refactor

Suggested by Alexis @lexi-lambda

* Update server/src-lib/Hasura/RQL/DDL/RemoteSchema.hs

As suggested by @lexi-lambda

Co-Authored-By: Alexis King <lexi.lambda@gmail.com>

* fix tests

* requested changes

* comment 'replaceMetadataToOrdJson' unit tests

Co-authored-by: Rishichandra Wawhal <rishi@hasura.io>
Co-authored-by: Alexis King <lexi.lambda@gmail.com>
Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
2020-03-26 17:22:20 +05:30
Vamshi Surabhi
b84db36ebb
allow custom mutations through actions (#3042)
* basic doc for actions

* custom_types, sync and async actions

* switch to graphql-parser-hs on github

* update docs

* metadata import/export

* webhook calls are now supported

* relationships in sync actions

* initialise.sql is now in sync with the migration file

* fix metadata tests

* allow specifying arguments of actions

* fix blacklist check on check_build_worthiness job

* track custom_types and actions related tables

* handlers are now triggered on async actions

* default to pgjson unless a field is involved in relationships, for generating definition list

* use 'true' for action filter for non admin role

* fix create_action_permission sql query

* drop permissions when dropping an action

* add a hdb_role view (and relationships) to fetch all roles in the system

* rename 'webhook' key in action definition to 'handler'

* allow templating actions wehook URLs with env vars

* add 'update_action' /v1/query type

* allow forwarding client headers by setting `forward_client_headers` in action definition

* add 'headers' configuration in action definition

* handle webhook error response based on status codes

* support array relationships for custom types

* implement single row mutation, see https://github.com/hasura/graphql-engine/issues/3731

* single row mutation: rename 'pk_columns' -> 'columns' and no-op refactor

* use top level primary key inputs for delete_by_pk & account select permissions for single row mutations

* use only REST semantics to resolve the webhook response

* use 'pk_columns' instead of 'columns' for update_by_pk input

* add python basic tests for single row mutations

* add action context (name) in webhook payload

* Async action response is accessible for non admin roles only if
  the request session vars equals to action's

* clean nulls, empty arrays for actions, custom types in export metadata

* async action mutation returns only the UUID of the action

* unit tests for URL template parser

* Basic sync actions python tests

* fix output in async query & add async tests

* add admin secret header in async actions python test

* document async action architecture in Resolve/Action.hs file

* support actions returning array of objects

* tests for list type response actions

* update docs with actions and custom types metadata API reference

* update actions python tests as per #f8e1330

Co-authored-by: Tirumarai Selvan <tirumarai.selvan@gmail.com>
Co-authored-by: Aravind Shankar <face11301@gmail.com>
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
2020-02-13 23:08:23 +05:30
Phil Freeman
94102c0460
Add downgrade command (close #1156) (#3760)
* Add downgrade command

* Add docs per @lexi-lambda's suggestions

* make tests pass

* Update hdb_version once, from Haskell

* more work based on feedback

* Improve the usage message

* Small docs changes

* Test downgrades exist for each tag

* Update downgrading.rst

* Use git-log to find tags which are ancestors of the current commit

Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
2020-02-07 16:33:12 +05:30
Vamshi Surabhi
2de663d2a8
Merge branch 'master' into 3759-3791-minor-metadata-build-bugfixes 2020-02-05 18:25:46 +05:30
Brandon Simmons
58ef316118 Add request timings and count histograms to telemetry. Closes #3552
We upload a set of accumulating timers and counters to track service
time for different types of operations, across several dimensions (e.g.
did we hit the plan cache, was a remote involved, etc.)

Also...

Standardize on DiffTime as a standard duration type, and try to use it
consistently.

See discussion here:
https://github.com/hasura/graphql-engine/pull/3584#pullrequestreview-340679369

It should be possible to overwrite that module so the new threadDelay
sticks per the pattern in #3705 blocked on #3558

Rename the Control.Concurrent.Extended.threadDelay to `sleep` since a
naive use with a literal argument would be very bad!

We catch a bug in 'computeTimeDiff'.

Add convenient 'Read' instances to the time unit utility types. Make
'Second' a newtype to support this.
2020-02-03 18:50:10 -06:00
Vamshi Surabhi
15072237e4
Merge branch 'master' into test-improvements-01-2020 2020-02-03 21:10:42 +05:30
Alexis King
8ef205fba5 Explicitly invalidate enum values when metadata is reloaded
This fixes #3759. Also, while we’re at it, also improve the way
invalidations are synced across instances so enums and remote schemas
are appropriately reloaded by the schema syncing process.
2020-01-30 18:17:29 -06:00
Alexis King
5bd5a548fa
server: Parameterize the graphql-engine library over the version (#3668) 2020-01-22 15:55:55 -06:00
Brandon Simmons
8716cf0e08 Unit tests: pass through some useful flags to hspec (--match and --skip)
--match for instance is suggested on test failure, so this is convenient
to have.
2020-01-21 13:12:27 -05:00
Alexis King
feb4a05618
Add fast path for server internal metadata migrations (#3686) 2020-01-14 01:09:10 -06:00
Anon Ray
dca8559703 fix cache-control header parsing for JWK (fix #3655) (#3676)
write a proper parser according to the RFC
https://tools.ietf.org/html/rfc7234#section-5.2
2020-01-13 15:56:51 -06:00
Rakesh Emmadi
319606b5ff include scalars types returning by computed fields in generated schema (fix #3650) (#3651) 2020-01-10 22:05:07 -06:00
Alexis King
fa9077f774 Add support for fine-grained dependency tracking to Incremental 2020-01-08 16:45:54 -06:00
Alexis King
27997107ab Add caching for recreating event trigger functions 2020-01-08 16:45:46 -06:00
Alexis King
447bac74e7 Split up Hasura.RQL.DDL.Schema.Cache
This should hopefully improve compile times by avoiding the need to
specialize everything at once.
2020-01-08 16:45:46 -06:00
Alexis King
5b969208c6 Use arrows instead of monads to define the schema cache construction 2020-01-08 16:43:06 -06:00
Alexis King
1387722970 Refactor schema cache construction to avoid imperative updates
wip: fix error codes in remote schema tests
2020-01-08 16:43:06 -06:00
Alexis King
f2963a25c3 Add Hasura.Incremental, a library for incremental builds 2020-01-08 16:43:06 -06:00
Rakesh Emmadi
421a182f64 export metadata without nulls, empty arrays & default values (#3393)
* export metadata without nulls, empty arrays
* property tests for 'ReplaceMetadata' using QuickCheck
-> Derive Arbitrary class for 'ReplaceMetadata' dependant types

* reduce property test cases number to 30
QuickCheck generates the `ReplaceMetadata` value really large
for higher number test cases. Encoded JSON for such values is large and
consumes more memory. Thus, CI is giving up while running property
tests.

* circle-ci: Add property tests as saperate job
* add no command mode to tests
* add yaml.v2 to go mod
* remove indirect comment for yaml.v2 dependency
2019-12-14 00:47:38 -06:00
Tirumarai Selvan
d2b2a58c0e add read_only to run_sql metadata api (#3191) 2019-11-14 18:20:18 -06:00
Rakesh Emmadi
6d92e4f9db save permissions, relationships and collections in catalog with 'is_system_defined' explicitly (#3165)
* save permissions, relationships and collections in catalog with 'is_system_defined'
* Use common stanzas in the .cabal file
* Refactor migration code into lib instead of exe
* Add new server test suite that exercises migrations
* Make graphql-engine clean succeed even if the schema does not exist
2019-10-21 11:01:05 -05:00