Commit Graph

41 Commits

Author SHA1 Message Date
Samir Talwar
2be2200b1a server: Split CatalogVersion into Source… and Metadata…
This reflects the two different usages, which should not be conflated.

We also propagate the type a little more, to avoid `Text`.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4931
GitOrigin-RevId: 16278f14aa4c2cb5667ea54bbb6b25e6d362835c
2022-07-04 09:32:12 +00:00
Samir Talwar
5f38743f29 server: Encapsulate catalog versions in their own type.
This came about as I tried to add an instance over catalog versions and
found they were just simple integers most of the time (and in one case,
a float).

I think this change also clarifies how catalog versions work.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4864
GitOrigin-RevId: a6b7db86de564b71a8c2b602bee6a456b8e20d63
2022-06-29 08:19:33 +00:00
Antoine Leblanc
609cab89e6 Backend-agnostic custom scalars
### Description

This small clean-up PR makes one further step towards backend-agnostic actions: it makes all the code parsing custom types backend agnostic. Surprisingly, this could be done *without* the need to finish generalizing the column parser. The remaining sore point is async queries, that still target Postgres explicitly.

In theory, this is enough to start allowing non-Postgres scalars in custom types. In practice, however:
- no other backend exposes scalars in a way that would allow users to do that as of this PR;
- we currently have no strategy to avoid / detect scalar collisions across backends.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4691
GitOrigin-RevId: bfe63fb131e306663d4406697ce23c02736566c5
2022-06-22 13:14:40 +00:00
Antoine Leblanc
e3c2bf53a5 Move, document, and prune action types and custom types types.
### Description

This PR is a first step in a series of cleanups of action relationships. This first step does not contain any behavioral change, and it simply reorganizes / prunes / rearranges / documents the code. Mainly:
- it divides some files in RQL.Types between metadata types, schema cache types, execution types;
- it renames some types for consistency;
- it minimizes exports and prunes unnecessary types;
- it moves some types in places where they make more sense;
- it replaces uses of `DMap BackendTag` with `BackendMap`.

Most of the "movement" within files re-organizes declarations in a "top-down" fashion, by moving all TH splices to the end of the file, which avoids order or declarations mattering.

### Optional list types

One main type change this PR makes is a replacement of variant list types in `CustomTypes.hs`; we had `Maybe [a]`, or sometimes `Maybe (NonEmpty a)`. This PR harmonizes all of them to `[a]`, as most of the code would use them as such, by doing `fromMaybe []` or `maybe [] toList`.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4613
GitOrigin-RevId: bc624e10df587eba862ff27a5e8021b32d0d78a2
2022-06-07 15:45:00 +00:00
Rakesh Emmadi
acde210fdc server/bigquery: generate graphql schema for table computed fields
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4460
GitOrigin-RevId: 7b772cd9fba6b612ad05eb1aca1fa13e6ae8556d
2022-05-25 10:25:38 +00:00
Antoine Leblanc
04d8f068b6 Remove explicit case on the backend tag in Cache
### Description

As part of the cache building process, we create / update / migrate the catalog that each DB uses as a place to store event trigger information. The function that decides how this should be done was doing an explicit `case ... of` on the backend tag, instead of delegating to one of the backend classes. The downsides of this is that:
- it adds a "friction point" where the backend matters in the core of the engine, which is otherwise written to be almost entirely backend-agnostic
- it creates imports from deep in the engine to the `Backends`, which we try to restrict to a very small set of clearly identified files (the `Instances` files)
- it is currently implemented using a "catch all" default case, which might not always be correct for new backends

This PR makes the catalog updating process a part of `BackendMetadata`, and cleans the corresponding schema cache code.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4457
GitOrigin-RevId: 592f0eaa97a7c38f4e6d4400e1d2353aab12c97e
2022-05-05 13:44:56 +00:00
Rakesh Emmadi
7474f50ef0 server/bigquery: enable metadata API to add/drop computed fields to BigQuery tables
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4414
GitOrigin-RevId: 7899824a49ba9fd479f4cf08789450153f454168
2022-05-04 14:53:31 +00:00
Daniel Chambers
970d69edd4 Added BackendConfig to allow sources to share configuration of Data Connectors [GDW-78]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4271
GitOrigin-RevId: 6990010bff622a424ca0bb9d24579bf121819fb0
2022-04-29 02:14:10 +00:00
Naveen Naidu
d10b862b61 server: fix bugs in clear_metadata code path
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3927
Co-authored-by: Divi <32202683+imperfect-fourth@users.noreply.github.com>
Co-authored-by: Karthikeyan Chinnakonda <15602904+codingkarthik@users.noreply.github.com>
Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
GitOrigin-RevId: 6f3457698029af94e9157c23b552ac2034f9c740
2022-04-11 11:25:13 +00:00
jkachmar
647231b685 Yeet some default-extensions
Manually enables:
* EmptyCase
* ExistentialQuantification
* QuantifiedConstraints
* QuasiQuotes
* TemplateHaskell
* TypeFamilyDependencies

...in the following components:
* 'graphql-engine' library
* 'graphql-engine' 'src-test'
* 'graphql-engine' 'tests/integration'
* 'graphql-engine' tests-hspec'

Additionally, performs some light refactoring and documentation.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3991
GitOrigin-RevId: 514477d3466b01f60eca8935d0fef60dd0756838
2022-03-16 00:40:17 +00:00
Rakesh Emmadi
aa19f1e0d0 server/postgres: improve fetching tables' and functions' metadata (from database)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3778
GitOrigin-RevId: 13bb97bdb7afad265db899f368c74d9f240b214a
2022-03-08 13:03:14 +00:00
Karthikeyan Chinnakonda
bea650b3e0 server/mssql: source catalog initialization for event triggers (Incremental PR - I)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2505
Co-authored-by: Naveen Naidu <30195193+Naveenaidu@users.noreply.github.com>
GitOrigin-RevId: 56681f90cfbfcf2f99c27f08c01d32790bd03c4d
2022-02-24 08:14:10 +00:00
Anon Ray
e87433c2bb server: remove hdb_lib and add tests for read-only source
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3634
Co-authored-by: Philip Lykke Carlsen <358550+plcplc@users.noreply.github.com>
GitOrigin-RevId: 2db62a279496cd7e5dd57bdf02c3efa7b70042c8
2022-02-21 10:00:00 +00:00
Evie Ciobanu
9337edf096 Evie/postgres/module docs
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3563
GitOrigin-RevId: e35aafbd210d526ed53db7088e1a47f89f0b9c44
2022-02-08 09:25:35 +00:00
Philip Lykke Carlsen
e1918adb52 Replace "identity column" with "column mutability" data for all backends
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3373
GitOrigin-RevId: bf08cc9008a4b0b3ece4952528c15c45e57fc74c
2022-02-03 14:15:35 +00:00
Naveen Naidu
3a8fadb22b server/mssql: support read replicas
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2578
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
GitOrigin-RevId: 88a02f8a617006853b350f48f4317c78ab97435b
2022-01-04 11:54:56 +00:00
Naveen Naidu
3773ba98b0 multitenant: support for starting multitenant in read only mode
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2993
Co-authored-by: Anon Ray <616387+ecthiender@users.noreply.github.com>
GitOrigin-RevId: e598d340d81aa96a85bd1ec043f9b7ed847934ef
2021-12-08 06:27:49 +00:00
Naveen Naidu
5cd6c5e43d multitenant: support for event disabling
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2900
GitOrigin-RevId: b2844fa433cfd8f8a29b7e98c6ec44773fd44a57
2021-11-30 12:32:26 +00:00
Karthikeyan Chinnakonda
adb20d1d7c server: log DB locking queries during source catalog migration
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2676
GitOrigin-RevId: f01574a30d3a6bf95467ce69bb8b5e69ce4cc057
2021-11-09 14:22:44 +00:00
Karthikeyan Chinnakonda
0c21100892 server: relocate some of the pre metadata separation source catalog migrations
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2687
GitOrigin-RevId: 9fd1c9bdfdf8de1e0d39333368ec799ae92a9e71
2021-11-03 14:21:40 +00:00
David Overton
aac64f2c81 Source typename customization (close graphql-engine#6974)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/1616
GitOrigin-RevId: f7eefd2367929209aa77895ea585e96a99a78d47
2021-10-29 14:43:14 +00:00
Karthikeyan Chinnakonda
eb90c236ea server: recreate event triggers on startup only when the source catalog is migrated
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2572
GitOrigin-RevId: a01b3c5003dd921907e1ea0c2dc87464359de167
2021-10-13 13:01:38 +00:00
Robert
11a454c2d6 server, pro: actually reformat the code-base using ormolu
This commit applies ormolu to the whole Haskell code base by running `make format`.

For in-flight branches, simply merging changes from `main` will result in merge conflicts.
To avoid this, update your branch using the following instructions. Replace `<format-commit>`
by the hash of *this* commit.

$ git checkout my-feature-branch
$ git merge <format-commit>^    # and resolve conflicts normally
$ make format
$ git commit -a -m "reformat with ormolu"
$ git merge -s ours post-ormolu

https://github.com/hasura/graphql-engine-mono/pull/2404

GitOrigin-RevId: 75049f5c12f430c615eafb4c6b8e83e371e01c8e
2021-09-23 22:57:37 +00:00
Robert
fe035125f4 server: drop LazyTxT newtype
This is a follow-up to #1959.

Today, I spent a while in review figuring out that a harmless PR change didn't do anything,
because it was moving from a `runLazy...` to something without the `Lazy`. So let's get
that source of confusion removed.

This should be a bit easier to review commit by commit, since some of the functions had
confusing names. (E.g. there was a misnamed `Migrate.Internal.runTx` before.)

The change should be a no-op.

https://github.com/hasura/graphql-engine-mono/pull/2335

GitOrigin-RevId: 0f284c4c0f814482d7827e7732a6d49e7735b302
2021-09-15 20:46:45 +00:00
Antoine Leblanc
49f40a44f0 Enforce that backends use the properly resolved environment variables.
## Description

This PR fixes an oversight in the implementation of the resolvers of different backends. To implement resolution from environment variables, both MSSQL and BigQuery were directly fetching the process' environment variables, instead of using the careful curated set we thread from main. It was working just fine on OSS, but is failing on Cloud.

This PR fixes this by adding an additional argument to `resolveSourceConfig`, to ensure that backends always use the correct set of variables.

https://github.com/hasura/graphql-engine-mono/pull/1891

GitOrigin-RevId: 58644cab7d041a8bf4235e2acfe9cf71533a92a1
2021-07-23 12:26:10 +00:00
Rakesh Emmadi
13bedf5821 server/postgres: fix resetting the metadata catalog version to 43 while initializing postgres source with 1.0 catalog (#1645)
* fix resetting the catalog version to 43 on migration from 1.0 to 2.0

* ci: remove applying patch in test_oss_server_upgrade job

* make the 43 to 46th migrations idempotent

* Set missing HASURA_GRAPHQL_EVENTS_HTTP_POOL_SIZE=8 in upgrade_test

It's not clear why this wasn't caught in CI.

* ci: disable one component of event backpressure test

Co-authored-by: Vishnu Bharathi P <vishnubharathi04@gmail.com>
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
Co-authored-by: Brandon Simmons <brandon@hasura.io>
GitOrigin-RevId: c74c6425266a99165c6beecc3e4f7c34e6884d4d
2021-06-23 18:01:09 +00:00
Karthikeyan Chinnakonda
cd509a0036 server: recreate event triggers while migrating source catalog
GitOrigin-RevId: 9a07fec31fcdbe6ef3b96bb61c95e0dd544e7f50
2021-06-07 13:58:16 +00:00
kodiakhq[bot]
01d8a37706 server: fix asymptotics of event_log batch fetching
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
GitOrigin-RevId: 9b8afb502e94dd371a8596ccde97d023bd2344a9
2021-06-01 17:34:38 +00:00
Karthikeyan Chinnakonda
f1f56ccf75 server: source initialization fix
GitOrigin-RevId: fcb94ca743a99ee3ffe30d40717bb1f0a13cf751
2021-05-31 13:54:59 +00:00
Rakesh Emmadi
e43d0273e0 server: mssql: apply schema changes by mssql_run_sql DDL on metadata (fix #779)
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
GitOrigin-RevId: 6905d5914c8a698445c0ef03d6a8303747701e1c
2021-05-27 15:07:10 +00:00
Swann Moreau
8e3e7501c7 server: do not collect metrics for sources that have been dropped
GitOrigin-RevId: ed439426527237c2907df77f45582dbe566aeb51
2021-05-26 07:41:28 +00:00
Swann Moreau
d3611af58d server: lazy event catalog initialisation
event catalog:
- `hdb_catalog` is no longer automatically created
- catalog is initialised when the first event trigger is created
- catalog initialisation is done during the schema cache build, using `ArrowCache` so it is only run in response to a change to the set of event triggers

event queue:
- `processEventQueue` thread is prevented from starting when `HASURA_GRAPHQL_EVENTS_FETCH_INTERVAL=0`
- `processEventQueue` thread only processes sources for which at least one event trigger exists in some table in the source

Co-authored-by: Anon Ray <616387+ecthiender@users.noreply.github.com>
GitOrigin-RevId: 73f256465d62490cd2b59dcd074718679993d4fe
2021-05-25 06:51:03 +00:00
Abby Sassel
1afa4ac3cc server/citus: feature branch
Co-authored-by: Vladimir Ciobanu <1017953+vladciobanu@users.noreply.github.com>
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
Co-authored-by: Ikechukwu Eze <22247592+iykekings@users.noreply.github.com>
Co-authored-by: Philip Lykke Carlsen <358550+plcplc@users.noreply.github.com>
Co-authored-by: Phil Freeman <630306+paf31@users.noreply.github.com>
GitOrigin-RevId: 1b964fe5f5f50380172cb702b6a328fed782b6b7
2021-05-21 02:47:51 +00:00
Antoine Leblanc
2152911e24 server: introduce Hasura.Base (take 2)
GitOrigin-RevId: 0dd10f1ccd338b1cf382ebff59b6ee7f209d39a1
2021-05-11 15:19:33 +00:00
Antoine Leblanc
ba70ca427a server: switch to a sub-backend approach
GitOrigin-RevId: 660126d5f65620fb58a3ffcbed564e9e35f59938
2021-04-21 21:45:32 +00:00
Karthikeyan Chinnakonda
d09b00040f server: V2 maintenance mode fixes
GitOrigin-RevId: dcd69fb217a4e976523f1bc8e7b2693111091002
2021-04-21 10:56:13 +00:00
Vladimir Ciobanu
91710bba58 server: use relative paths in TH splices
While debugging issues with HLS, Reed Mullanix noticed that we don't use relative paths. This leads to problems when using HLS + Emacs due to a bug in `lsp-mode` which prevents it from finding the correct project root.

However, it is still a good practice to use relative paths in TH for other reasons, including being able to import these modules in GHCI.

This PR should make it so HLS-1.0 & emacs provide type inference, imports, etc., in all modules in our codebase.

GitOrigin-RevId: 5f53b9a7ccf46df1ea7be94ff0a5c6ec861f4ead
2021-03-16 17:36:39 +00:00
Vladimir Ciobanu
281cb771ff server: add MSSQL support
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
Co-authored-by: Aravind K P <8335904+scriptonist@users.noreply.github.com>
GitOrigin-RevId: 699c453b9692e1b822f393f23ff5e6db4e010d57
2021-02-23 17:38:36 +00:00
Naveen Naidu
4fa34e1d5e server: Apply migrations when catalog version < 43
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
GitOrigin-RevId: 4fa1a04f4924002faac1eca2e5092dfa8767fdb9
2021-02-22 15:00:24 +00:00
Anon Ray
0dfe83f931 server: optimize resolving source
GitOrigin-RevId: e0513a714b5e3f6d730b0470e1836ed44badd3c0
2021-02-22 07:53:36 +00:00
Rakesh Emmadi
9ef603360c server: generalize schema cache building (#496)
Co-authored-by: Vamshi Surabhi <vamshi@hasura.io>
Co-authored-by: Vladimir Ciobanu <admin@cvlad.info>
Co-authored-by: Antoine Leblanc <antoine@hasura.io>
Co-authored-by: Stylish Haskell Bot <stylish-haskell@users.noreply.github.com>
GitOrigin-RevId: 9d631878037637f3ed2994b5d0525efd978f7b8f
2021-02-14 06:08:46 +00:00