Commit Graph

5850 Commits

Author SHA1 Message Date
pranshi06
8dab7df169 server: fix error in metadata APIs with inconsistency
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6669
Co-authored-by: Tirumarai Selvan <8663570+tirumaraiselvan@users.noreply.github.com>
Co-authored-by: Sean Park-Ross <94021366+seanparkross@users.noreply.github.com>
GitOrigin-RevId: 1b004074b41ccb6512123cdb1707b39792e97927
2022-12-08 16:04:55 +00:00
Samir Talwar
f430e5b599 dev.sh: Fix mssql_wait to use mssql-tools to run sqlcmd.
On macOS, we use the `azure-sql-edge` image, which doesn't contain
`sqlcmd`. To work around this, we can use another container which _does_
contain the `sqlcmd` binary to verify that MS SQL Server is up.

This was also broken on Linux anyway.

This also updates the PostgreSQL image, in line with _databases.yaml_.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7204
GitOrigin-RevId: ad974cd353c348ec055a153cbba5cb39d85e0967
2022-12-08 10:00:28 +00:00
Daniel Harvey
e37b91a25a [server/tests] limit capabilities in matrix tester using RTS opts
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7202
GitOrigin-RevId: c2b6ff37a0b5839726d02c04c69fadeff4465022
2022-12-08 08:45:34 +00:00
Samir Talwar
2ee0dbf8eb server: Fix permissions when running an update_table_many query.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6969
Co-authored-by: Gil Mizrahi <8547573+soupi@users.noreply.github.com>
GitOrigin-RevId: 4c6f6959dd69ff861a0333619f7ad90a013ed6ef
2022-12-08 08:04:08 +00:00
Lyndon Maydwell
e9dcbf6491 Adding custom scalar types to SQLite DC Agent - GDC-610
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6783
Co-authored-by: David Overton <7734777+dmoverton@users.noreply.github.com>
Co-authored-by: Daniel Chambers <1214352+daniel-chambers@users.noreply.github.com>
GitOrigin-RevId: ee53c20b0090b6b3c88792ddc98b8287872fe0f3
2022-12-08 06:50:08 +00:00
Daniel Chambers
c14fd3ba4c Add mutability properties to the Data Connector schema API [GDC-664]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7190
GitOrigin-RevId: ce602b5e5cc5aee8716ff3f7a036b18b3bf47188
2022-12-08 02:07:01 +00:00
Daniel Harvey
1dd9e19b69 [server/test] create new HGE per test
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7111
GitOrigin-RevId: 6d3e88cb29ca3e98cbd16141c9f924aa36fa6b43
2022-12-07 17:06:42 +00:00
Vijay Prasanna
e1064c34e0 feature (console): View browse rows relationships for GDC tables (WIP)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6875
GitOrigin-RevId: d905d84c3cf21790415ec22294e1efabaf93f185
2022-12-07 14:53:13 +00:00
Daniel Harvey
d3ee6d46a9 [tooling] add make py-tests command
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7196
GitOrigin-RevId: 1507285bb42734a04bfb130f0cd1a0c28be2f3ca
2022-12-07 14:13:22 +00:00
Gil Mizrahi
f5ea45060a docs: cockroachdb and citus support subscriptions
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7194
GitOrigin-RevId: 48550ec78541a0ac1c834cef2ca83195636951ce
2022-12-07 14:02:21 +00:00
Krushan Bauva
c540f803a6 server: add server_type information to version API
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7141
GitOrigin-RevId: e1d1c780e6b588f0868003eb6d40019960246aba
2022-12-07 11:30:20 +00:00
Daniel Harvey
a7c6fd65f5 [ci] fix matrix test setup
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7195
GitOrigin-RevId: 75494d1bb4cb9060f03a918626135cd6f6191744
2022-12-07 09:27:26 +00:00
kodiakhq[bot]
3f3b19c565 server: tune builder runners, for ByteStrings we're about to send
`toLazyByteString` is a little deficient in two ways:

- It allocates relatively large chunks (4KB + 32KB +32KB, etc…) which is wasteful for small ByteStrings
- It shrinks each chunk (Copying the data to a new chunk of exactly the right size) if it's not more than half filled. If we're running the builder right before we send it over the wire, this copy is totally extraneous (we simply end up with more work for the next GC)

part of the effort: https://github.com/hasura/graphql-engine-mono/issues/5518

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7187
GitOrigin-RevId: b499cd49c33da6cfee96be629a36b5c812486e39
2022-12-07 06:01:08 +00:00
Mohd Bilal
00f5df3e47 cli: expose hasura migrate delete command as an API
Closes https://hasurahq.atlassian.net/browse/PLAT-292

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6955
GitOrigin-RevId: 41d77ebbe830b29e0ba0cdf6c3a67527e5580bdf
2022-12-07 03:02:14 +00:00
Lyndon Maydwell
3d5fb984b0 Fix metadata defaults bug - Defaults serialised into metadata table - GDC-647
## Description

There is a bug in the metadata defaults code, see [the original PR](https://github.com/hasura/graphql-engine-mono/pull/6286).

Steps to reproduce this issue:

* Start a new HGE project
* Start HGE with a defaults argument: `HASURA_GRAPHQL_LOG_LEVEL=debug cabal run exe:graphql-engine -- serve --enable-console --console-assets-dir=./console/static/dist --metadata-defaults='{"backend_configs": {"dataconnector": {"mongo": {"display_name": "BONGOBB", "uri": "http://localhost:8123"}}}}'`
* Add a source (doesn't need to be related to the defaults)
* Export metadata
* See that the defaults are present in the exported metadata

## Related Issues

* Github Issue: https://github.com/hasura/graphql-engine/issues/9237
* Jira: https://hasurahq.atlassian.net/browse/GDC-647
* Original PR: https://github.com/hasura/graphql-engine-mono/pull/6286

## Solution

* The test for if defaults should be included for metadata api operations has been extended to check for updates
* Metadata inconsistencies have been hidden for `/capabilities` calls on startup

## TODO

* [x] Fix bug
* [x] Write tests
* [x] OSS Metadata Migration to correct persisted data - `server/src-rsr/migrations/47_to_48.sql`
* [x] Cloud Metadata Migration - `pro/server/res/cloud/migrations/6_to_7.sql`
* [x] Bump Catalog Version - `server/src-rsr/catalog_version.txt`
* [x] Update Catalog Versions - `server/src-rsr/catalog_versions.txt` (This will be done by Infra when creating a release)
* [x] Log connection error as it occurs *(Already being logged. Requires `--enabled-log-types startup,webhook-log,websocket-log,http-log,data-connector-log`)
* [x] Don't mark metadata inconsistencies for this call.

## Questions

* [ ] Does the `pro/server/res/cloud/migrations/6_to_7.sql` cover the cloud scenarios?
* [ ] Should we have `SET search_path` in migrations?
* [x] What should be in `server/src-rsr/catalog_versions.txt`?

## Testing

To test the solution locally run:

> docker compose up -d

and

> cabal run  -- exe:api-tests --skip BigQuery --skip SQLServer --skip '/Test.API.Explain/Postgres/'

## Solution

In `runMetadataQuery` in `server/src-lib/Hasura/Server/API/Metadata.hs`:

```diff
-        if (exportsMetadata _rqlMetadata)
+        if (exportsMetadata _rqlMetadata || queryModifiesMetadata _rqlMetadata)
```

This ensures that defaults aren't present in operations that serialise metadata.

Note: You might think that `X_add_source` would need the defaults to be present to add a source that references the defaults, but since the resolution occurs in the schema-cache building phase, the defaults can be excluded for the metadata modifications required for `X_add_source`.

In addition to the code-change, a metadata migration has been introduced in order to clean up serialised defaults.

The following scenarios need to be considered for both OSS and Cloud:

* The user has not had defaults serialised
* The user has had the defaults serialised and no other backends configured
* The user has had the defaults serialised and has also configured other backends

We want to remove as much of the metadata as possible without any user-specified data and this should be reflected in migration `server/src-rsr/migrations/47_to_48.sql`.

## Server checklist

### Catalog upgrade

Does this PR change Hasura Catalog version?
-  Yes

### Metadata
Does this PR add a new Metadata feature?
-  No

### GraphQL
-  No new GraphQL schema is generated

### Breaking changes
-  No Breaking changes

## Changelog

__Component__ : server
__Type__: bugfix
__Product__: community-edition

### Short Changelog

Fixes a metadata defaults serialization bug and introduces a metadata migration to correct data that has been persisted due to the bug.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7034
GitOrigin-RevId: ad7d4f748397a1a607f2c0c886bf0fbbc3f873f2
2022-12-06 22:35:19 +00:00
Nicolas Inchauspe
fec3707e2e GraphiQL style bugs
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7172
GitOrigin-RevId: 8381852fd0c9ac22dce22911d933b446f6e8de59
2022-12-06 19:30:22 +00:00
Rob Dominguez
6446c798ab docs: remove heroku one-click and references
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7164
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
GitOrigin-RevId: e0fb2af227f323f82a401c81135a18f813cb3ce3
2022-12-06 18:08:19 +00:00
Sameer Kolhar
dc065cb98d console: hotfix minor issues in Check Database Latency feature
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7176
GitOrigin-RevId: 8f86ee64d44c67bbea0ccfead14d0bc2cd92b03a
2022-12-06 17:28:08 +00:00
Gil Mizrahi
a948e3460a add ci scripts for produce-feature-matrix
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7144
Co-authored-by: Daniel Harvey <4729125+danieljharvey@users.noreply.github.com>
GitOrigin-RevId: ab4e2e62ee2e347dfbbe08c16688b2ddfa3a1b1c
2022-12-06 16:42:39 +00:00
Adron Hall
bb5c935e59 Adding snowflake
## Description

First draft of the Snowflake getting started documentation.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6936
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sean Park-Ross <94021366+seanparkross@users.noreply.github.com>
Co-authored-by: Brandon Martin <40686+codedmart@users.noreply.github.com>
Co-authored-by: Rob Dominguez <24390149+robertjdominguez@users.noreply.github.com>
GitOrigin-RevId: 8500e81111f3200634ee9183d710718f0463f419
2022-12-06 16:01:36 +00:00
paritosh-08
3d4c66041f server: add event_processing_time metric
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6987
GitOrigin-RevId: 4f5948a225b18e2db59572482d2a608117fbb7d0
2022-12-06 15:11:04 +00:00
Julian@Hasura
4aee57b174 console: Use custom display name for drivers in dropdown - GDC-653
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7098
Co-authored-by: Julian <843342+okjulian@users.noreply.github.com>
GitOrigin-RevId: cf92d6c9c5d5116ce5b0ce972e859308afe886c2
2022-12-06 12:59:05 +00:00
surendran82
6acb7152cb community sample apps: image broken link update for SEO
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7170
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
GitOrigin-RevId: 145831d1056922af70150f8193c7f868de756f8f
2022-12-06 11:04:21 +00:00
Varun Choudhary
d10591daf3 console: add negative badge to show inconsistent remote schema
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7067
GitOrigin-RevId: 6eeae6945639e98fda9ab90ff05a6af5fc668454
2022-12-06 10:57:46 +00:00
Rishichandra Wawhal
7f0e4b2092 console: hide onboarding for users who onboarded before onboarding was launched
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7168
GitOrigin-RevId: 8e119129b2a910fefd692e93a19007cdfd564bf5
2022-12-06 06:10:34 +00:00
Rob Dominguez
adf565fe3d docs: make cloud db guides agnostic
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7127
Co-authored-by: Sean Park-Ross <94021366+seanparkross@users.noreply.github.com>
GitOrigin-RevId: 7ea4fc8829216308878085577412b4a9b8c5942e
2022-12-06 00:13:02 +00:00
Solomon
599d9fd602 Dynamic Dispatch For Test Harness BackendType Metadata
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6982
GitOrigin-RevId: faeec9ba0dfcb641c13cdc3f88244237fb370c66
2022-12-06 00:06:53 +00:00
Luca Restagno
1ce631132a console: fix scenario in which data is not ready yet
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7161
GitOrigin-RevId: 4b03fba03b2c1b8a4d855d2db852e8ce4f82d579
2022-12-05 15:23:51 +00:00
Gil Mizrahi
f45c379866 add concurrent_bulk api for parallel processing of read-only queries
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7087
Co-authored-by: Tom Harding <6302310+i-am-tom@users.noreply.github.com>
Co-authored-by: Luca Restagno <59067245+lucarestagno@users.noreply.github.com>
GitOrigin-RevId: 0d7a212d004908bc014def9d3828647545c9e062
2022-12-05 13:50:19 +00:00
Daniel Harvey
ba800ec978 [pg-client] test cancelled queries actually stop running queries
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7091
GitOrigin-RevId: 71cc1291d45a90a2c4f3519e12e365e968368dca
2022-12-05 12:19:11 +00:00
Daniel Harvey
aa9499ee0e [server/tests] remove unnecessary cleanup
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7157
GitOrigin-RevId: 93dc285c8629586336e56bb1622d94af0faa3e0e
2022-12-05 11:41:38 +00:00
Philip Lykke Carlsen
392ecf72b3 server: Improve modelling of function metadata and schema diffing
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7140
GitOrigin-RevId: 83f54308d002391903d3c3d421e589949e7d4dc4
2022-12-05 10:22:15 +00:00
Gil Mizrahi
1970785970 fix api-tests warnings and enable -Wall
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7137
GitOrigin-RevId: 1c06f99ef94e1343d33c79e59e844c0470ba71aa
2022-12-05 08:31:00 +00:00
Vishnu Bharathi
73ad0275f3 ci: tag release v2.16.0
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7148
GitOrigin-RevId: 25adaa09c57471ed96d3d131abfa6c6c4225d6fd
2022-12-05 07:37:26 +00:00
Daniel Chambers
354ab2f7aa Use Sandwich test framework for Data Connector agent test suite
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7133
GitOrigin-RevId: 625dde70d1c594dcdfd377efac78710174a74efc
2022-12-05 02:16:01 +00:00
Lyndon Maydwell
9c2a2823ef Data Connectors Stable API Release
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7131
GitOrigin-RevId: 1ddce3aced0e022b71527098d88817824ca37ce9
2022-12-02 22:32:38 +00:00
Rob Dominguez
9e20658b9e docs: reintroduce regression tests for pro cli and restructure subdirectory
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7081
Co-authored-by: Sean Park-Ross <94021366+seanparkross@users.noreply.github.com>
GitOrigin-RevId: 29f8f061e86d2be6c2db0a9f297fe6c5a704404d
2022-12-02 18:37:11 +00:00
Matthew Goodwin
12d4de57c0 console: Revisions to #6931
For context: https://github.com/hasura/graphql-engine-mono/pull/6931

This PR makes some revisions that I had originally intended to be in the code, but due to some mixup, was refactored without discussion. @vijayprasanna13 and I synced up to try and make a decision about how to resolve it, but ultimately wanted to get more eyes on it to weigh in.

For context, when I refer to "premade selectors" I am talking about [this file](8a9264b14c/console/src/features/hasura-metadata-api/selectors.ts (L34))

And when I refer to "utility functions" (or metadata utility functions), I am talking about [this file](8a9264b14c/console/src/features/hasura-metadata-api/utils.ts (L38))

The changes I am suggesting here mostly about maintaining (IMHO) clearer separation of concern for selectors vs utility functions, and improving DX.

Here's how I see it high level:

![flows (light theme)](https://user-images.githubusercontent.com/49927862/204630900-aca2aad1-d760-409d-8ec6-446942b2d6ff.png#gh-light-mode-only)
![flows (dar theme)](https://user-images.githubusercontent.com/173663/204755227-e8850dbf-1012-4c7b-8087-dd6f16e5ca23.jpg#gh-dark-mode-only)

In other words, I want to avoid using pre-made selectors within a custom selector as it creates a pattern of multiple layers of curried functions and beings to (IMHO) look confusing and blur the separation of concern.

What I am proposing is that we have a convention of doing either

1) Developer wants the entire table object of a specific table. They can easily do this using one of the **premade** selectors and pass directly into `useMetdata()`:

```ts
const { isLoading, data: metadataTable } = useMetadata(
    MetadataSelectors.findTable(dataSourceName, table)
  );
```

2) Developer wants to get a specific piece of information on a specific table. They can leverage the `findMetadataTable()` utility function within a custom inline selector like this and avoid extra currying and double selector confusion:

  ```ts
const { isLoading, data: savedComment } = useMetadata(
    m => MetadataUtils.findMetadataTable(dataSourceName, table, m)?.configuration?.comment
  );
```

This is somewhat of a stylistic preference, but also one that I think makes a cleaner separation of concern in the way we are building the api and supportive pieces. In other words, premade selectors are _always_ a selector and not used to compose other selectors. It also makes available utility functions that are easy to pull of the shelf to do common metadata tasks whether in the scope of `useMetadata` or not.

## Simpler Approach?

We could also just go with **Flow # 2** and eliminate the premade selectors. The dev would then pass a custom selector into `useMetadata()` and if they prefer, use a utility function to support their task.

With that aside, whatever the team agrees makes more sense, I will gladly get on board with, but I wanted to make my case here : )

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7095
Co-authored-by: Vijay Prasanna <11921040+vijayprasanna13@users.noreply.github.com>
GitOrigin-RevId: 7ecc2db172def0d2f714cbbb2f95a4b56a528093
2022-12-02 18:28:40 +00:00
Philip Lykke Carlsen
886302fd7e server/tests: Introuce pytest-to-haskell porting script
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7054
GitOrigin-RevId: 2c4597465da28d26993b347693813ce7d42962d3
2022-12-02 15:59:19 +00:00
Rob Dominguez
0cc4693def docs: add ha multi region docs
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7096
Co-authored-by: Sean Park-Ross <94021366+seanparkross@users.noreply.github.com>
GitOrigin-RevId: bcb5e70dac234920c27dcf34e13f84fa2bc992d7
2022-12-02 14:26:45 +00:00
Puru Gupta
5c8bd8abd9 docs: update docs around webhook based authentication for websocket connection
## Description

This PR updates the doc around webhook based authentication for websocket connections explaining how exactly `Cache-Control` and `Expires` headers function.

## Changelog ✍️

__Component__ : server

__Type__: bugfix

__Product__: community-edition

### Short Changelog

update docs around webhook based authentication for websocket connection

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7135
Co-authored-by: Sean Park-Ross <94021366+seanparkross@users.noreply.github.com>
GitOrigin-RevId: ab873c394aad3a94d1a3affa483363f28ab77693
2022-12-02 13:25:06 +00:00
Gil Mizrahi
2f83899979 Fix mssql aggregation test
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7142
GitOrigin-RevId: 66447f0bd7859b3fc5203d87cba5ea2c6d5f054e
2022-12-02 12:53:32 +00:00
Daniel Harvey
ef5e983aee [server/tests] simplify TestEnvironment
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7123
GitOrigin-RevId: 0a7313dab729de08e273ba06e47f656d766ff82a
2022-12-02 11:36:57 +00:00
Erik Magnusson
ffdc9ceca8 console: fix clone value handling for the insert tab
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7097
Co-authored-by: Luca Restagno <59067245+lucarestagno@users.noreply.github.com>
GitOrigin-RevId: 483ce76f24234f76f3b557c892c62d891e542eb7
2022-12-02 10:55:56 +00:00
Daniele Cammareri
5f205a3f7d test: add snapshot check to action e2e test
## Description 🔖

This PR adds snapshot testing to e2e tests. The goal is to avoid regressions that involve metadata by checking:

- the payload of requests sent
- the metadata after modification (only relevant part of the metadata)

![image](https://user-images.githubusercontent.com/8408875/204244610-4486b689-d220-40ab-bc1d-9c7b1a1b232f.png)

We implemented this feature for one test, `actionWithTransform.e2e.test.ts` but this approach can be potentially used for other tests as well.

## Review checklist. 📋

- [ ] Run e2e tests

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7058
Co-authored-by: Varun Choudhary <68095256+Varun-Choudhary@users.noreply.github.com>
GitOrigin-RevId: 618147fad3f7d6849a0815b16c077efddef751ca
2022-12-02 10:03:39 +00:00
Daniel Harvey
16ecd63b01 [server/tests] don't do detailed teardown
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7128
GitOrigin-RevId: cbc52dadaca8ef7d4affb43528e8dcec0c8ec62e
2022-12-02 09:00:07 +00:00
Solomon
6c106c9e35 [GDC] Transform SourceConnConfig in runGetSourceTables
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7112
GitOrigin-RevId: d6ab09ba001fa8d4d33cc8f669b588459360f910
2022-12-02 08:02:20 +00:00
Gil Mizrahi
e752010a83 Gil/fix feature matrix test and apply css
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7125
GitOrigin-RevId: fa051a4942a882efe6e3d673eab084b94ec8dd90
2022-12-01 19:26:39 +00:00
Shahidh K Muhammed
5873eb025e github md pages: update email address for security team
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7113
GitOrigin-RevId: f29224d486016ea8ef8bb6ff51d1059950509e02
2022-12-01 17:19:51 +00:00
Abby Sassel
13eb1122ed server/tests: Table Computed Fields > Postgres
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7122
GitOrigin-RevId: e8c191a87465c1810908e8851bcc4b6919873bd8
2022-12-01 16:43:54 +00:00