`toLazyByteString` is a little deficient in two ways:
- It allocates relatively large chunks (4KB + 32KB +32KB, etc…) which is wasteful for small ByteStrings
- It shrinks each chunk (Copying the data to a new chunk of exactly the right size) if it's not more than half filled. If we're running the builder right before we send it over the wire, this copy is totally extraneous (we simply end up with more work for the next GC)
part of the effort: https://github.com/hasura/graphql-engine-mono/issues/5518
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7187
GitOrigin-RevId: b499cd49c33da6cfee96be629a36b5c812486e39
## Description
There is a bug in the metadata defaults code, see [the original PR](https://github.com/hasura/graphql-engine-mono/pull/6286).
Steps to reproduce this issue:
* Start a new HGE project
* Start HGE with a defaults argument: `HASURA_GRAPHQL_LOG_LEVEL=debug cabal run exe:graphql-engine -- serve --enable-console --console-assets-dir=./console/static/dist --metadata-defaults='{"backend_configs": {"dataconnector": {"mongo": {"display_name": "BONGOBB", "uri": "http://localhost:8123"}}}}'`
* Add a source (doesn't need to be related to the defaults)
* Export metadata
* See that the defaults are present in the exported metadata
## Related Issues
* Github Issue: https://github.com/hasura/graphql-engine/issues/9237
* Jira: https://hasurahq.atlassian.net/browse/GDC-647
* Original PR: https://github.com/hasura/graphql-engine-mono/pull/6286
## Solution
* The test for if defaults should be included for metadata api operations has been extended to check for updates
* Metadata inconsistencies have been hidden for `/capabilities` calls on startup
## TODO
* [x] Fix bug
* [x] Write tests
* [x] OSS Metadata Migration to correct persisted data - `server/src-rsr/migrations/47_to_48.sql`
* [x] Cloud Metadata Migration - `pro/server/res/cloud/migrations/6_to_7.sql`
* [x] Bump Catalog Version - `server/src-rsr/catalog_version.txt`
* [x] Update Catalog Versions - `server/src-rsr/catalog_versions.txt` (This will be done by Infra when creating a release)
* [x] Log connection error as it occurs *(Already being logged. Requires `--enabled-log-types startup,webhook-log,websocket-log,http-log,data-connector-log`)
* [x] Don't mark metadata inconsistencies for this call.
## Questions
* [ ] Does the `pro/server/res/cloud/migrations/6_to_7.sql` cover the cloud scenarios?
* [ ] Should we have `SET search_path` in migrations?
* [x] What should be in `server/src-rsr/catalog_versions.txt`?
## Testing
To test the solution locally run:
> docker compose up -d
and
> cabal run -- exe:api-tests --skip BigQuery --skip SQLServer --skip '/Test.API.Explain/Postgres/'
## Solution
In `runMetadataQuery` in `server/src-lib/Hasura/Server/API/Metadata.hs`:
```diff
- if (exportsMetadata _rqlMetadata)
+ if (exportsMetadata _rqlMetadata || queryModifiesMetadata _rqlMetadata)
```
This ensures that defaults aren't present in operations that serialise metadata.
Note: You might think that `X_add_source` would need the defaults to be present to add a source that references the defaults, but since the resolution occurs in the schema-cache building phase, the defaults can be excluded for the metadata modifications required for `X_add_source`.
In addition to the code-change, a metadata migration has been introduced in order to clean up serialised defaults.
The following scenarios need to be considered for both OSS and Cloud:
* The user has not had defaults serialised
* The user has had the defaults serialised and no other backends configured
* The user has had the defaults serialised and has also configured other backends
We want to remove as much of the metadata as possible without any user-specified data and this should be reflected in migration `server/src-rsr/migrations/47_to_48.sql`.
## Server checklist
### Catalog upgrade
Does this PR change Hasura Catalog version?
- ✅ Yes
### Metadata
Does this PR add a new Metadata feature?
- ✅ No
### GraphQL
- ✅ No new GraphQL schema is generated
### Breaking changes
- ✅ No Breaking changes
## Changelog
__Component__ : server
__Type__: bugfix
__Product__: community-edition
### Short Changelog
Fixes a metadata defaults serialization bug and introduces a metadata migration to correct data that has been persisted due to the bug.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7034
GitOrigin-RevId: ad7d4f748397a1a607f2c0c886bf0fbbc3f873f2
## Description
First draft of the Snowflake getting started documentation.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6936
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sean Park-Ross <94021366+seanparkross@users.noreply.github.com>
Co-authored-by: Brandon Martin <40686+codedmart@users.noreply.github.com>
Co-authored-by: Rob Dominguez <24390149+robertjdominguez@users.noreply.github.com>
GitOrigin-RevId: 8500e81111f3200634ee9183d710718f0463f419
For context: https://github.com/hasura/graphql-engine-mono/pull/6931
This PR makes some revisions that I had originally intended to be in the code, but due to some mixup, was refactored without discussion. @vijayprasanna13 and I synced up to try and make a decision about how to resolve it, but ultimately wanted to get more eyes on it to weigh in.
For context, when I refer to "premade selectors" I am talking about [this file](8a9264b14c/console/src/features/hasura-metadata-api/selectors.ts (L34))
And when I refer to "utility functions" (or metadata utility functions), I am talking about [this file](8a9264b14c/console/src/features/hasura-metadata-api/utils.ts (L38))
The changes I am suggesting here mostly about maintaining (IMHO) clearer separation of concern for selectors vs utility functions, and improving DX.
Here's how I see it high level:
In other words, I want to avoid using pre-made selectors within a custom selector as it creates a pattern of multiple layers of curried functions and beings to (IMHO) look confusing and blur the separation of concern.
What I am proposing is that we have a convention of doing either
1) Developer wants the entire table object of a specific table. They can easily do this using one of the **premade** selectors and pass directly into `useMetdata()`:
```ts
const { isLoading, data: metadataTable } = useMetadata(
MetadataSelectors.findTable(dataSourceName, table)
);
```
2) Developer wants to get a specific piece of information on a specific table. They can leverage the `findMetadataTable()` utility function within a custom inline selector like this and avoid extra currying and double selector confusion:
```ts
const { isLoading, data: savedComment } = useMetadata(
m => MetadataUtils.findMetadataTable(dataSourceName, table, m)?.configuration?.comment
);
```
This is somewhat of a stylistic preference, but also one that I think makes a cleaner separation of concern in the way we are building the api and supportive pieces. In other words, premade selectors are _always_ a selector and not used to compose other selectors. It also makes available utility functions that are easy to pull of the shelf to do common metadata tasks whether in the scope of `useMetadata` or not.
## Simpler Approach?
We could also just go with **Flow # 2** and eliminate the premade selectors. The dev would then pass a custom selector into `useMetadata()` and if they prefer, use a utility function to support their task.
With that aside, whatever the team agrees makes more sense, I will gladly get on board with, but I wanted to make my case here : )
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7095
Co-authored-by: Vijay Prasanna <11921040+vijayprasanna13@users.noreply.github.com>
GitOrigin-RevId: 7ecc2db172def0d2f714cbbb2f95a4b56a528093
## Description
This PR updates the doc around webhook based authentication for websocket connections explaining how exactly `Cache-Control` and `Expires` headers function.
## Changelog ✍️
__Component__ : server
__Type__: bugfix
__Product__: community-edition
### Short Changelog
update docs around webhook based authentication for websocket connection
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7135
Co-authored-by: Sean Park-Ross <94021366+seanparkross@users.noreply.github.com>
GitOrigin-RevId: ab873c394aad3a94d1a3affa483363f28ab77693
## Description 🔖
This PR adds snapshot testing to e2e tests. The goal is to avoid regressions that involve metadata by checking:
- the payload of requests sent
- the metadata after modification (only relevant part of the metadata)
We implemented this feature for one test, `actionWithTransform.e2e.test.ts` but this approach can be potentially used for other tests as well.
## Review checklist. 📋
- [ ] Run e2e tests
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7058
Co-authored-by: Varun Choudhary <68095256+Varun-Choudhary@users.noreply.github.com>
GitOrigin-RevId: 618147fad3f7d6849a0815b16c077efddef751ca
