## Description
There is a bug in the metadata defaults code, see [the original PR](https://github.com/hasura/graphql-engine-mono/pull/6286).
Steps to reproduce this issue:
* Start a new HGE project
* Start HGE with a defaults argument: `HASURA_GRAPHQL_LOG_LEVEL=debug cabal run exe:graphql-engine -- serve --enable-console --console-assets-dir=./console/static/dist --metadata-defaults='{"backend_configs": {"dataconnector": {"mongo": {"display_name": "BONGOBB", "uri": "http://localhost:8123"}}}}'`
* Add a source (doesn't need to be related to the defaults)
* Export metadata
* See that the defaults are present in the exported metadata
## Related Issues
* Github Issue: https://github.com/hasura/graphql-engine/issues/9237
* Jira: https://hasurahq.atlassian.net/browse/GDC-647
* Original PR: https://github.com/hasura/graphql-engine-mono/pull/6286
## Solution
* The test for if defaults should be included for metadata api operations has been extended to check for updates
* Metadata inconsistencies have been hidden for `/capabilities` calls on startup
## TODO
* [x] Fix bug
* [x] Write tests
* [x] OSS Metadata Migration to correct persisted data - `server/src-rsr/migrations/47_to_48.sql`
* [x] Cloud Metadata Migration - `pro/server/res/cloud/migrations/6_to_7.sql`
* [x] Bump Catalog Version - `server/src-rsr/catalog_version.txt`
* [x] Update Catalog Versions - `server/src-rsr/catalog_versions.txt` (This will be done by Infra when creating a release)
* [x] Log connection error as it occurs *(Already being logged. Requires `--enabled-log-types startup,webhook-log,websocket-log,http-log,data-connector-log`)
* [x] Don't mark metadata inconsistencies for this call.
## Questions
* [ ] Does the `pro/server/res/cloud/migrations/6_to_7.sql` cover the cloud scenarios?
* [ ] Should we have `SET search_path` in migrations?
* [x] What should be in `server/src-rsr/catalog_versions.txt`?
## Testing
To test the solution locally run:
> docker compose up -d
and
> cabal run -- exe:api-tests --skip BigQuery --skip SQLServer --skip '/Test.API.Explain/Postgres/'
## Solution
In `runMetadataQuery` in `server/src-lib/Hasura/Server/API/Metadata.hs`:
```diff
- if (exportsMetadata _rqlMetadata)
+ if (exportsMetadata _rqlMetadata || queryModifiesMetadata _rqlMetadata)
```
This ensures that defaults aren't present in operations that serialise metadata.
Note: You might think that `X_add_source` would need the defaults to be present to add a source that references the defaults, but since the resolution occurs in the schema-cache building phase, the defaults can be excluded for the metadata modifications required for `X_add_source`.
In addition to the code-change, a metadata migration has been introduced in order to clean up serialised defaults.
The following scenarios need to be considered for both OSS and Cloud:
* The user has not had defaults serialised
* The user has had the defaults serialised and no other backends configured
* The user has had the defaults serialised and has also configured other backends
We want to remove as much of the metadata as possible without any user-specified data and this should be reflected in migration `server/src-rsr/migrations/47_to_48.sql`.
## Server checklist
### Catalog upgrade
Does this PR change Hasura Catalog version?
- ✅ Yes
### Metadata
Does this PR add a new Metadata feature?
- ✅ No
### GraphQL
- ✅ No new GraphQL schema is generated
### Breaking changes
- ✅ No Breaking changes
## Changelog
__Component__ : server
__Type__: bugfix
__Product__: community-edition
### Short Changelog
Fixes a metadata defaults serialization bug and introduces a metadata migration to correct data that has been persisted due to the bug.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7034
GitOrigin-RevId: ad7d4f748397a1a607f2c0c886bf0fbbc3f873f2
With the vague idea that we can eventually avoid publishing ports at all, at least in CI, while still having the flexibility to publish ports locally.
If we can get there, it should help with the issues we're seeing on CI, where ports are already allocated from previous runs and containers aren't properly cleaned up.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6836
GitOrigin-RevId: 1d636c73ae889e45e80ad52042c56daa1b9d5838
>
## Description
->
This PR allows DC agents to define custom aggregate functions for their scalar types.
### Related Issues
->
GDC-189
### Solution and Design
>
We added a new property `aggregate_functions` to the scalar types capabilities. This allows the agent author to specify a set of aggregate functions supported by each scalar type, along with the function's result type.
During GraphQL schema generation, the custom aggregate functions are available via a new method `getCustomAggregateOperators` on the `Backend` type class.
Custom functions are merged with the builtin aggregate functions when building GraphQL schemas for table aggregate fields and for `order_by` operators on array relations.
### Steps to test and verify
>
• Codec tests for aggregate function capabilities have been added to the unit tests.
• Some custom aggregate operators have been added to the reference agent and are used in a new test in `api-tests`.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6199
GitOrigin-RevId: e9c0d1617af93847c1493671fdbb794f573bde0c
As I have been onboarding, in order to get a working sqlite agent/db running locally, I got some helpful steps from @mattshardman which I adapted into this testing setup.
Hopefully this should ease the pain for the next person needing to do this!
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6261
GitOrigin-RevId: 03a3f04e38f118898bfd1d27de9305739dee5276
## Description 🔖
This adds the ability to "customize & track" using the [new tracking ui](https://github.com/hasura/graphql-engine-mono/pull/5391).
A new button was added to implement this:
<img width="870" alt="Screen Shot 2022-09-16 at 12 37 14 PM" src="https://user-images.githubusercontent.com/49927862/190701948-1ad86717-f6be-4f67-8e0c-17b618790795.png">
## Solution and Design 🎨
This feature mostly makes use of components and hooks already created.
I was able to refactor some code to reduce code duplication and type duplication.
A few highlights on the refactor:
- `useTrackTable`, `useUntrackTable` and `useTrackSelectedTables` were all refactored into a single hook: `useTrackTable`. This hook has one main function but returns 4 wrapper functions: `trackTable`, `untrackTable`, `trackTables`, and `untrackTables`. This should make maintaining easier in the future.
- Synced up types between `MetadataTableConfig` and the customization form. Previously, the customization form had duplicated this same type, and there was some slight discrepancies between them.
- Modified `TableTrackingCustomizationModal` `onSubmit` return with a 2nd argument that's in the exact shape of `MetadataTableConfig` for convenience.
- Did some refactoring of the `DropDownMenu` component that should not interfere with anything. Exposed a few of the inner components for export and used these modular pieces in the current implementation. This ended up not being used in the feature, but left it in as it's a slight improvement.
I also added a function called `delayAsyncAction` [here](5e88262628/console/src/components/Common/utils/jsUtils.tsx (L416)) to create an artificial delay for `async` functions. This allows us to create a more confident UX when requests happen near-instantly. Introducing a tiny delay of around 300ms with good UI feedback (i.e. loading spinner) shows the user something is happening. I wanted to document this as I'm not sure it's something other will agree on. If it's against our UX philosophy, I can remove it, but I found it nice.
## Review Setup 💻
1. Run the code locally and go to `http://localhost:3000/data/v2/manage?database=YOUR_DATABASE_NAME_HERE`
2. You should see the new button as shown above to `Customize & Track`
3. Try it out with various field custom roots
4. Also try tracking and untracking tables both from the individual rows as well as the checkboxes as this logic was also modified in the refactor.
** I am not totally sure how much I need to put here for testing, I may want some help here from the team. **
## Review checklist. 📋
### Functionality
- [ ] Steps to verify console behaviour
### Tests
Going to open another PR for adding testing to the relevant areas. Will update this PR with a link once that's been done.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5922
GitOrigin-RevId: e7c14be5b9bcc255a7b9ecfd43f1f84aa8aabba2
