hasura/graphql-engine
1
0
Fork 0
mirror of https://github.com/hasura/graphql-engine.git synced 2024-12-15 17:31:56 +03:00
Code Issues Projects Releases Wiki Activity
9d2ac22365
graphql-engine/.circleci
History
Abby Sassel a091110364 server/tests: ephemeral BigQuery projects for CI test jobs 
_Problem_
We currently run teardown/`DELETE` statements on the same, shared `hasura_test` dataset. This is not ideal as parallel test runs can and do affect each other, resulting in nondeterministic CI failures.

Closes https://github.com/hasura/graphql-engine-mono/issues/2521

_Solution and design_
This PR introduces ephemeral, isolated projects for each test run _in CI only_. Projects are created within [the Google Cloud Platform `data-sources-test-bigquery` directory](https://console.cloud.google.com/iam-admin/settings?folder=704256416468&orgonly=true&supportedpurview=organizationId) on each test run, and destroyed afterwards.

I've only introduced this change in CI for the time being:
  1. this isn't as much of an issue locally because we're less likely to run bigquery tests in parallel.
  2. to more quickly unblock https://github.com/hasura/graphql-engine/issues/7929.
  3. to limit the number of new projects created until we have a better idea of our usage vs GCP quota/limits.

Also updated the [internal wiki here](https://github.com/hasura/graphql-engine-mono/wiki/Testing-BigQuery) with this info.

_To verify_
- CI: [this job](https://buildkite.com/hasura/graphql-engine-mono/builds/3770#89e5bac6-16fe-447e-bcda-85cd47ea1b77) successfully runs all tests on a temporary project & dataset
- local: follow [these steps](https://github.com/hasura/graphql-engine-mono/wiki/Testing-BigQuery#ci--optional-dedicated-gcp-project-for-tests) to try the same setup locally

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3240
GitOrigin-RevId: d88d9cb7922266bfa962cfcb481e0272b8929a5d
2022-01-25 08:28:06 +00:00
..
pgbouncer run graphql tests on both http and websocket; add parallelism (close #1868) (#1921) 2019-04-08 12:52:38 +05:30
postgres-12.0-alpine-postgis3 build: rename file to adhere to windows rules (close #4002) (#4003) 2020-03-03 12:33:21 +05:30
postgres-13.0-alpine-postgis3 server: hasura on PG v13 (#125) 2020-12-01 12:22:42 +00:00
server-upgrade-downgrade bump python test suite dependencies 2022-01-17 07:40:56 +00:00
.gitignore run graphql tests on both http and websocket; add parallelism (close #1868) (#1921) 2019-04-08 12:52:38 +05:30
build-docker-image.sh circleci integration (#74) 2018-07-10 10:01:02 +00:00
ciignore.sh server: split server test suite to run in parallel on circleci (#309) 2021-01-25 08:24:13 +00:00
cli-migrations-builder.dockerfile ci: add tests for cli-migrations image (#4396) 2020-04-15 07:26:00 +05:30
config.yml ci,server: upgrades to GHC 8.10.7 and reworks server builder image 2021-11-15 13:22:39 +00:00
install-cli-builder-deps.sh cli, ci: enables docker based tests 2021-05-17 00:30:08 +00:00
README.md circleci: Use backport of Matt Pickering's memory reclamation RTS patch in CI 2021-03-12 02:09:21 +00:00
server-test-names.txt tests: break out flag tests into separate step 2022-01-24 09:55:32 +00:00
test-console.sh build: use the debian buster version in the docker image 2021-05-27 07:50:15 +00:00
test-deprecated-server-flags.sh tests: run server test shell files through shfmt for consistent indentation 2022-01-11 11:22:42 +00:00
test-server-flags.sh tests: run server test shell files through shfmt for consistent indentation 2022-01-11 11:22:42 +00:00
test-server.sh server/tests: ephemeral BigQuery projects for CI test jobs 2022-01-25 08:28:06 +00:00

README.md

TODO(swann): write this out better

Any tests in test-server.sh should also have their names added to server-test-names.txt, otherwise the CI pipelines will not run them.