Commit Graph

2462 Commits

Author SHA1 Message Date
Tom Harding
1d0b466954 Run the DefaultRootFieldSpec tests on all backends.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7293
GitOrigin-RevId: f7c22212badc11016e87a491bb0017642b3ac7c0
2022-12-15 18:18:29 +00:00
Jesse Hallett
c265e303f6 server: codecs for remote schemas metadata
These codecs should fully cover the `remote_schemas` property of the Metadata type.

Ticket: [GDC-522](https://hasurahq.atlassian.net/browse/GDC-522)

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6812
GitOrigin-RevId: 1b256f6829486295957c232b92ff184bd9a86469
2022-12-15 17:39:22 +00:00
Philip Lykke Carlsen
bd2e2080f8 server/tests: Port TestGraphQLQueryFunctions
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7158
GitOrigin-RevId: a4b3ca8565df37905665303f96fd40ec4b9e76dd
2022-12-15 16:42:55 +00:00
Tom Harding
ae33e34e2d Implement permission checks in the API tests
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7264
GitOrigin-RevId: 55ff8ff31609b7598d2382ded5dd96f5825ee4fd
2022-12-15 16:02:31 +00:00
paritosh-08
9f99a40f2d Revert "server: fix MSSQL event trigger next_retry_at"
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7290
GitOrigin-RevId: 4cb461dd6f25e55ba1c9bd587d98a56f71ff8223
2022-12-15 15:31:40 +00:00
Naveen Naidu
282e94b2de pro/server: add user_id of the collaborator in http_log
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7154
GitOrigin-RevId: 6f60a7c7be5b083e6adf6c3ab9991142f62434df
2022-12-15 07:49:37 +00:00
Tom Harding
f2dd4a5eb7 Remove *Source fields from API test permissions
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7275
GitOrigin-RevId: 44345ec361fdbb93f2c166825e5d78983759b5b0
2022-12-14 18:43:44 +00:00
Samir Talwar
beb20c7dab server/tests-py: Remove a check for PostgreSQL 10.
We no longer support this and therefore don't run tests against it.

This also refactors the code a little so it doesn't have to skip running a PostgreSQL-specific test against MS SQL Server.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7201
GitOrigin-RevId: 307c2ab0052162c012f7b1c55866b57f2fa6d9a6
2022-12-14 16:57:28 +00:00
Abby Sassel
57f456a4cf add run-new.sh docs
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7278
GitOrigin-RevId: 6c17cc846b39a4236f65dd5ab6baac4529b38f4c
2022-12-14 16:11:00 +00:00
Divi
62708ca6bd ci: update catalog version for v2.16.1
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7271
GitOrigin-RevId: a7978ae1c0dd4d402f6293a74cf45acba560a456
2022-12-14 07:14:24 +00:00
Brandon Simmons
3b0ad30757 server: don't compress small responses
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7216
GitOrigin-RevId: 20a94267eca42edf8e79470711766a3fecac0f50
2022-12-13 17:50:01 +00:00
paritosh-08
3397c041bc server: fix MSSQL event trigger next_retry_at
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7173
GitOrigin-RevId: 862ab4136e0df6b7fa1e2db2c3edfffa318452de
2022-12-13 15:18:23 +00:00
Gil Mizrahi
570e16efe7 Revert "[server/test] create new HGE per test"
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7255
GitOrigin-RevId: c971e9e227263b0956069c0687acacf4d3ff90a9
2022-12-13 12:37:27 +00:00
Auke Booij
51fc104793 server: don't let query collection validation prevent engine startup
Generate more Metadata Inconsistencies instead of startup failures. Specifically this means that
- errors retrieving the main query of an executable GraphQL document, and
- errors during fragment inlining

no longer fail irrecoverably.

This also makes more parts of `buildSchemaCacheRule` into pure code, which is always nice.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7234
GitOrigin-RevId: aebf636c2fb1aad1c2df9a37f7d0b67c1ee40c42
2022-12-13 09:56:27 +00:00
Daniel Harvey
ed190d6cb6 [server/tests] drop old postgres database better, and use less connections
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7227
GitOrigin-RevId: d7b3b313440d3f3c91d4ecb550513b53cbf2e92e
2022-12-12 17:46:12 +00:00
kodiakhq[bot]
3ad81d18c8 [tooling] import hasura/pool into monorepo
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7226
Co-authored-by: Bryan O'Sullivan <23374+bos@users.noreply.github.com>
Co-authored-by: Falko Peters <41595+informatikr@users.noreply.github.com>
Co-authored-by: Bas van Dijk <576355+basvandijk@users.noreply.github.com>
Co-authored-by: Ozgun Ataman <13820+ozataman@users.noreply.github.com>
Co-authored-by: Michael Snoyman <49415+snoyberg@users.noreply.github.com>
Co-authored-by: Kim Altintop <6163+kim@users.noreply.github.com>
Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
Co-authored-by: Evie Ciobanu <1017953+eviefp@users.noreply.github.com>
Co-authored-by: Robert <132113+robx@users.noreply.github.com>
Co-authored-by: Auke Booij <164426+abooij@users.noreply.github.com>
GitOrigin-RevId: c99a93563f9b1aa25e69ecc030f8dc526df4fb6b
2022-12-12 16:10:25 +00:00
Daniel Chambers
a26ecbc2b0 Enable mutations root field parsing for Data Connectors [GDC-665]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7221
GitOrigin-RevId: 88da78cd758bc8c2273b0d71a79d5fe46a4bc565
2022-12-12 04:42:56 +00:00
Brandon Simmons
6451d9c9ce server: refactor Hasura.Server.Compression for clarity/correctness
context: This is  foundation work, before we change how the server chooses to compress or not
part of effort: #5518

-----

Prior to this change it was difficult to understand how the functionality in this module related to the semantics of Accept-Encoding. We also didn't correctly handle directives with qvalues.

After this change certain technical infelicities are called out without modifying the behavior of the server; for instance we continue to fall back to identity (no compression) in the case where technically we're supposed to return 406, and we also  continue to treat `*` conservatively as meaning “use no compression”.

The only external change here is `gzip;q=x.y` now results in a zipped response.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7213
GitOrigin-RevId: 1910ffd70d29f1ab8825c601f1bd998be70ceeeb
2022-12-09 06:08:27 +00:00
pranshi06
8dab7df169 server: fix error in metadata APIs with inconsistency
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6669
Co-authored-by: Tirumarai Selvan <8663570+tirumaraiselvan@users.noreply.github.com>
Co-authored-by: Sean Park-Ross <94021366+seanparkross@users.noreply.github.com>
GitOrigin-RevId: 1b004074b41ccb6512123cdb1707b39792e97927
2022-12-08 16:04:55 +00:00
Daniel Harvey
e37b91a25a [server/tests] limit capabilities in matrix tester using RTS opts
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7202
GitOrigin-RevId: c2b6ff37a0b5839726d02c04c69fadeff4465022
2022-12-08 08:45:34 +00:00
Samir Talwar
2ee0dbf8eb server: Fix permissions when running an update_table_many query.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6969
Co-authored-by: Gil Mizrahi <8547573+soupi@users.noreply.github.com>
GitOrigin-RevId: 4c6f6959dd69ff861a0333619f7ad90a013ed6ef
2022-12-08 08:04:08 +00:00
Lyndon Maydwell
e9dcbf6491 Adding custom scalar types to SQLite DC Agent - GDC-610
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6783
Co-authored-by: David Overton <7734777+dmoverton@users.noreply.github.com>
Co-authored-by: Daniel Chambers <1214352+daniel-chambers@users.noreply.github.com>
GitOrigin-RevId: ee53c20b0090b6b3c88792ddc98b8287872fe0f3
2022-12-08 06:50:08 +00:00
Daniel Chambers
c14fd3ba4c Add mutability properties to the Data Connector schema API [GDC-664]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7190
GitOrigin-RevId: ce602b5e5cc5aee8716ff3f7a036b18b3bf47188
2022-12-08 02:07:01 +00:00
Daniel Harvey
1dd9e19b69 [server/test] create new HGE per test
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7111
GitOrigin-RevId: 6d3e88cb29ca3e98cbd16141c9f924aa36fa6b43
2022-12-07 17:06:42 +00:00
Krushan Bauva
c540f803a6 server: add server_type information to version API
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7141
GitOrigin-RevId: e1d1c780e6b588f0868003eb6d40019960246aba
2022-12-07 11:30:20 +00:00
kodiakhq[bot]
3f3b19c565 server: tune builder runners, for ByteStrings we're about to send
`toLazyByteString` is a little deficient in two ways:

- It allocates relatively large chunks (4KB + 32KB +32KB, etc…) which is wasteful for small ByteStrings
- It shrinks each chunk (Copying the data to a new chunk of exactly the right size) if it's not more than half filled. If we're running the builder right before we send it over the wire, this copy is totally extraneous (we simply end up with more work for the next GC)

part of the effort: https://github.com/hasura/graphql-engine-mono/issues/5518

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7187
GitOrigin-RevId: b499cd49c33da6cfee96be629a36b5c812486e39
2022-12-07 06:01:08 +00:00
Lyndon Maydwell
3d5fb984b0 Fix metadata defaults bug - Defaults serialised into metadata table - GDC-647
## Description

There is a bug in the metadata defaults code, see [the original PR](https://github.com/hasura/graphql-engine-mono/pull/6286).

Steps to reproduce this issue:

* Start a new HGE project
* Start HGE with a defaults argument: `HASURA_GRAPHQL_LOG_LEVEL=debug cabal run exe:graphql-engine -- serve --enable-console --console-assets-dir=./console/static/dist --metadata-defaults='{"backend_configs": {"dataconnector": {"mongo": {"display_name": "BONGOBB", "uri": "http://localhost:8123"}}}}'`
* Add a source (doesn't need to be related to the defaults)
* Export metadata
* See that the defaults are present in the exported metadata

## Related Issues

* Github Issue: https://github.com/hasura/graphql-engine/issues/9237
* Jira: https://hasurahq.atlassian.net/browse/GDC-647
* Original PR: https://github.com/hasura/graphql-engine-mono/pull/6286

## Solution

* The test for if defaults should be included for metadata api operations has been extended to check for updates
* Metadata inconsistencies have been hidden for `/capabilities` calls on startup

## TODO

* [x] Fix bug
* [x] Write tests
* [x] OSS Metadata Migration to correct persisted data - `server/src-rsr/migrations/47_to_48.sql`
* [x] Cloud Metadata Migration - `pro/server/res/cloud/migrations/6_to_7.sql`
* [x] Bump Catalog Version - `server/src-rsr/catalog_version.txt`
* [x] Update Catalog Versions - `server/src-rsr/catalog_versions.txt` (This will be done by Infra when creating a release)
* [x] Log connection error as it occurs *(Already being logged. Requires `--enabled-log-types startup,webhook-log,websocket-log,http-log,data-connector-log`)
* [x] Don't mark metadata inconsistencies for this call.

## Questions

* [ ] Does the `pro/server/res/cloud/migrations/6_to_7.sql` cover the cloud scenarios?
* [ ] Should we have `SET search_path` in migrations?
* [x] What should be in `server/src-rsr/catalog_versions.txt`?

## Testing

To test the solution locally run:

> docker compose up -d

and

> cabal run  -- exe:api-tests --skip BigQuery --skip SQLServer --skip '/Test.API.Explain/Postgres/'

## Solution

In `runMetadataQuery` in `server/src-lib/Hasura/Server/API/Metadata.hs`:

```diff
-        if (exportsMetadata _rqlMetadata)
+        if (exportsMetadata _rqlMetadata || queryModifiesMetadata _rqlMetadata)
```

This ensures that defaults aren't present in operations that serialise metadata.

Note: You might think that `X_add_source` would need the defaults to be present to add a source that references the defaults, but since the resolution occurs in the schema-cache building phase, the defaults can be excluded for the metadata modifications required for `X_add_source`.

In addition to the code-change, a metadata migration has been introduced in order to clean up serialised defaults.

The following scenarios need to be considered for both OSS and Cloud:

* The user has not had defaults serialised
* The user has had the defaults serialised and no other backends configured
* The user has had the defaults serialised and has also configured other backends

We want to remove as much of the metadata as possible without any user-specified data and this should be reflected in migration `server/src-rsr/migrations/47_to_48.sql`.

## Server checklist

### Catalog upgrade

Does this PR change Hasura Catalog version?
-  Yes

### Metadata
Does this PR add a new Metadata feature?
-  No

### GraphQL
-  No new GraphQL schema is generated

### Breaking changes
-  No Breaking changes

## Changelog

__Component__ : server
__Type__: bugfix
__Product__: community-edition

### Short Changelog

Fixes a metadata defaults serialization bug and introduces a metadata migration to correct data that has been persisted due to the bug.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7034
GitOrigin-RevId: ad7d4f748397a1a607f2c0c886bf0fbbc3f873f2
2022-12-06 22:35:19 +00:00
Gil Mizrahi
a948e3460a add ci scripts for produce-feature-matrix
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7144
Co-authored-by: Daniel Harvey <4729125+danieljharvey@users.noreply.github.com>
GitOrigin-RevId: ab4e2e62ee2e347dfbbe08c16688b2ddfa3a1b1c
2022-12-06 16:42:39 +00:00
paritosh-08
3d4c66041f server: add event_processing_time metric
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6987
GitOrigin-RevId: 4f5948a225b18e2db59572482d2a608117fbb7d0
2022-12-06 15:11:04 +00:00
Solomon
599d9fd602 Dynamic Dispatch For Test Harness BackendType Metadata
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6982
GitOrigin-RevId: faeec9ba0dfcb641c13cdc3f88244237fb370c66
2022-12-06 00:06:53 +00:00
Gil Mizrahi
f45c379866 add concurrent_bulk api for parallel processing of read-only queries
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7087
Co-authored-by: Tom Harding <6302310+i-am-tom@users.noreply.github.com>
Co-authored-by: Luca Restagno <59067245+lucarestagno@users.noreply.github.com>
GitOrigin-RevId: 0d7a212d004908bc014def9d3828647545c9e062
2022-12-05 13:50:19 +00:00
Daniel Harvey
ba800ec978 [pg-client] test cancelled queries actually stop running queries
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7091
GitOrigin-RevId: 71cc1291d45a90a2c4f3519e12e365e968368dca
2022-12-05 12:19:11 +00:00
Daniel Harvey
aa9499ee0e [server/tests] remove unnecessary cleanup
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7157
GitOrigin-RevId: 93dc285c8629586336e56bb1622d94af0faa3e0e
2022-12-05 11:41:38 +00:00
Philip Lykke Carlsen
392ecf72b3 server: Improve modelling of function metadata and schema diffing
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7140
GitOrigin-RevId: 83f54308d002391903d3c3d421e589949e7d4dc4
2022-12-05 10:22:15 +00:00
Gil Mizrahi
1970785970 fix api-tests warnings and enable -Wall
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7137
GitOrigin-RevId: 1c06f99ef94e1343d33c79e59e844c0470ba71aa
2022-12-05 08:31:00 +00:00
Vishnu Bharathi
73ad0275f3 ci: tag release v2.16.0
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7148
GitOrigin-RevId: 25adaa09c57471ed96d3d131abfa6c6c4225d6fd
2022-12-05 07:37:26 +00:00
Daniel Chambers
354ab2f7aa Use Sandwich test framework for Data Connector agent test suite
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7133
GitOrigin-RevId: 625dde70d1c594dcdfd377efac78710174a74efc
2022-12-05 02:16:01 +00:00
Philip Lykke Carlsen
886302fd7e server/tests: Introuce pytest-to-haskell porting script
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7054
GitOrigin-RevId: 2c4597465da28d26993b347693813ce7d42962d3
2022-12-02 15:59:19 +00:00
Gil Mizrahi
2f83899979 Fix mssql aggregation test
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7142
GitOrigin-RevId: 66447f0bd7859b3fc5203d87cba5ea2c6d5f054e
2022-12-02 12:53:32 +00:00
Daniel Harvey
ef5e983aee [server/tests] simplify TestEnvironment
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7123
GitOrigin-RevId: 0a7313dab729de08e273ba06e47f656d766ff82a
2022-12-02 11:36:57 +00:00
Daniel Harvey
16ecd63b01 [server/tests] don't do detailed teardown
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7128
GitOrigin-RevId: cbc52dadaca8ef7d4affb43528e8dcec0c8ec62e
2022-12-02 09:00:07 +00:00
Solomon
6c106c9e35 [GDC] Transform SourceConnConfig in runGetSourceTables
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7112
GitOrigin-RevId: d6ab09ba001fa8d4d33cc8f669b588459360f910
2022-12-02 08:02:20 +00:00
Gil Mizrahi
e752010a83 Gil/fix feature matrix test and apply css
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7125
GitOrigin-RevId: fa051a4942a882efe6e3d673eab084b94ec8dd90
2022-12-01 19:26:39 +00:00
Abby Sassel
13eb1122ed server/tests: Table Computed Fields > Postgres
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7122
GitOrigin-RevId: e8c191a87465c1810908e8851bcc4b6919873bd8
2022-12-01 16:43:54 +00:00
Gil Mizrahi
9ce6fe7197 Feature matrix standalone tester tool
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7120
Co-authored-by: Samir Talwar <47582+SamirTalwar@users.noreply.github.com>
GitOrigin-RevId: 5db7f8e24f22414805b10143248bfacfb5b7a03a
2022-12-01 14:48:55 +00:00
Anon Ray
b398001c87 server/gardening: drop an unnecessary field from 'ServeCtx'
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7108
GitOrigin-RevId: 50caff9e60d53261ce15870338bc3aa97c479190
2022-12-01 07:48:58 +00:00
David Overton
a18c6976f8 Map scalar types to GraphQL built-in types for parsing [GDC-587]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6818
GitOrigin-RevId: 6d1887fb7865fe8ec24a73c77da291c7ecf8dfd2
2022-12-01 00:08:40 +00:00
Jesse Hallett
d6b8f29383 server: implement codecs for table permissions
This PR implements the remaining codecs for table permissions. However the codec for boolean expressions delegates to Aeson instances because Autodocodec doesn't currently have the necessary feature to write a codec for boolean expressions that will reliably parse valid data.

Boolean expressions are objects with keys like `_and`, `_or`, `_exists`, or `<field name>`. The parsing rules for each value depend on the key, so we need to be able to select different codecs for each key. We could do that with an `object` codec, but that doesn't account for the arbitrary field name keys that can be provided. OpenAPI supports object types with "additional properties", but I don't know if we can declare a specific type for those properties. There might or might not be a reasonable path to extending Autodocodec to handle this case.

Ticket: [GDC-585](https://hasurahq.atlassian.net/browse/GDC-585)

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6978
GitOrigin-RevId: 0b0dcfd59ebd1d5022ff2ab86dd8d4c6f93bd039
2022-11-30 19:33:16 +00:00
Auke Booij
512340b864 Collect Metadata dependencies in a Sequence rather than a list
Dependencies seem to get concatenated very often, so let's use a data structure that supports efficient concatenation.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7050
GitOrigin-RevId: 6331963f99f17d1b908a6038318d8c4834cf4dd7
2022-11-30 18:13:31 +00:00
kodiakhq[bot]
26ce8ff768 server: -O1 + -funfolding-use-threshold=640, for memory residency
...and keep binary size down using `--gc-sections` (supported by all major linkers). Stripped size after this PR is `85.3 Mb` down from `121 Mb` on `main`.

This compliments Auke's recent work (though there is some overlap). See: https://github.com/hasura/graphql-engine-mono/pull/7033#issuecomment-1326673632

Old experiments, for reference:
https://github.com/hasura/graphql-engine-mono/pull/6605
https://github.com/hasura/graphql-engine-mono/pull/6606
https://github.com/hasura/graphql-engine-mono/pull/6589

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7079
GitOrigin-RevId: 0b0d86730fb959d6249bd17a87e39af12ef71261
2022-11-30 17:05:15 +00:00