These tests are intended to catch issues in upgrading HGE. However:
* the tests are very convoluted and hard to understand,
* we can only run a small subset of Python tests that don't mutate any data or metadata, and
* I have never seen them fail for a legitimate reason, but I've seen a lot of flakes.
While we do believe it's important to test that upgrades don't break the stored introspection, these tests don't seem to be doing that any more. I humbly request that we delete them now and either (a) figure out how to test this properly, or (b) just wait for v3, which does away with reintrospecting on server startup entirely.
[NDAT-259]: https://hasurahq.atlassian.net/browse/NDAT-259?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8844
GitOrigin-RevId: 528bc632fce377b7eff2026b832bd58586ac5a0b
This brings it all in line. It seems to work fine for me.
I am doing this because I can't seem to cross-build with node.js v12 on macOS+arm64.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8713
GitOrigin-RevId: 06d7daa4176e99584ab88146a3d17200721ae8bf
## Description
Add multitenant integration tests for zero downtime env variable configuration on cloud
- [x] `HASURA_GRAPHQL_STRINGIFY_NUMERIC_TYPES`
- [x] `HASURA_GRAPHQL_DEV_MODE`
- [x] `HASURA_GRAPHQL_DEFAULT_NAMING_CONVENTION`
- [x] `HASURA_GRAPHQL_ENABLED_APIS`
- [x] `HASURA_GRAPHQL_ENABLE_ALLOWLIST`
- [x] `HASURA_GRAPHQL_ADMIN_INTERNAL_ERRORS`
- [ ] `HASURA_GRAPHQL_EXPERIMENTAL_FEATURES`
- [x] inherited_roles (GA tests not needed)
- [ ] optimise_permission_filters (pending)
- [x] naming_convention (done as part of `HASURA_GRAPHQL_DEFAULT_NAMING_CONVENTION`)
- [x] streaming_subscriptions (GA tests not needed)
- [x] apollo_federation
- [x] hide_update_many_fields
- [x] bigquery_string_numeric_input (done as part of `HASURA_GRAPHQL_STRINGIFY_NUMERIC_TYPES`)
- [x] hide_aggregation_predicates
- [x] hide_stream_fields
- [x] `HASURA_GRAPHQL_V1_BOOLEAN_NULL_COLLAPSE`
This PR also add the integration tests for changing auth mode dynamically. This test changes the auth mode from Admin Secret to JWT and finally to auth hook and checks that a sample query is run correctly.
```
Dynamic Auth mode changes are reflected properly
test multitenant works fine when auth mode changes from ADMIN SECRET -> JWT -> AUTH HOOK [ ]
Spock is running on port 9231
test multitenant works fine when auth mode changes from ADMIN SECRET -> JWT -> AUTH HOOK [✔]
```
We know that OSS python tests (`test-server.sh`) has a more intensive auth mode tests. And hence the efforts in this [branch](https://github.com/hasura/graphql-engine-mono/compare/gqls/zero-downtime-envvars...naveen/tests-auth-changes-dynamically) tests multitenant server on all the auth tests of the pytest suite on a single running multitenant server. I can confirm that all those tests passed and thus gives us high confidence that we did not break any code related to auth.
Note that, the test in this PR is a very simple test to see that AUTH mode is working correctly. We are okay with this simple test because we have already tested the multitenant server on all the auth mode tests present in OSS pytest suite.
Epic: https://hasurahq.atlassian.net/browse/GS-232
Jira Ticket: https://hasurahq.atlassian.net/browse/GS-387
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7721
Co-authored-by: Naveen Naidu <30195193+Naveenaidu@users.noreply.github.com>
Co-authored-by: Anon Ray <616387+ecthiender@users.noreply.github.com>
GitOrigin-RevId: 7fbe3d0d6e1dd2334b911c94f7f8bbaa170a8b59
This rewrites the JWT tests to generate and specify the secrets per test class, and to provide the server configuration to the HGE fixture.
It covers the tests in:
- *test_jwt.py*
- *test_jwt_claims_map.py*
- *test_config_api.py*
- *test_graphql_queries.py* (just a couple here)
This does reduce the number of code paths exercised with JWT, as we were previously running *all* tests with JWT tokens. However, this seems excessive; we don't need to tread every code path, just enough to ensure we handle the tokens appropriately. I believe that the test coverage in *test_jwt.py* does this well enough (though I'd prefer if we moved the coverage lower down in the stack as unit tests).
These tests were configured in multiple different ways by *test-server.sh*; this configuration is now moved to test subclasses within the various files. This results in a bit of duplication.
Unfortunately, the tests would ideally use parameterization rather than subclassing, but that doesn't work because of `hge_fixture_env`, which creates a "soft" dependency between the environment variables and `hge_server`. Parameterizing the former *should* force the latter to be recreated for each new set of environment variables, but `hge_server` isn't actually aware there's a dependency.
It currently looks like this adds lines of code; we'll more than make up for it when we delete the relevant lines from *test-server.sh*. I am not doing that here because I plan on deleting the whole file in a subsequent changeset.
[NDAT-538]: https://hasurahq.atlassian.net/browse/NDAT-538?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8803
GitOrigin-RevId: f7f2caa62de0b0a45e42964b69a8ae73d1575fe8
See this earlier iteration of this work for an example of the kind of report we're producing: #7664
And related work in this repo: github.com:hasura/graphql-bench-helper
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7923
GitOrigin-RevId: 99d2a55e2fb5b55f3f33e2570cfd0bc23e448e0c
