Abby Sassel
|
fc0352a995
|
server/tests: fix BigQuery test failure
This PR addresses a couple of recent issues with BigQuery tests in CI, plus some refactoring.
TL;DR: most issues were fixed by being [explicit about failures](http://redsymbol.net/articles/unofficial-bash-strict-mode/) and variables in scope. Thanks to @qrilka for addressing a couple of these in your last PR already, sharing my notes about them here too for posterity as I found the troubleshooting exercise useful.
_commands listed in execution order_
`generate_test_project`
* issue: this rarely fails AFAICT, but a failure in any of the `gcloud` commands will result in unclear failures further along the call stack
* fix: fail faster with `set -e`
`create_bigquery_dataset`
* error: `BigQuery error in mk operation: Not found: Project`. examples [[1]](https://github.com/hasura/graphql-engine-mono/issues/3600)[[2]](https://buildkite.com/hasura/graphql-engine-mono/builds/4857#f70990eb-3721-45c6-b11a-af7e4ebd9fe3)
* issue: a failure on a missing project, which I could verify was created successfully
* cause: missing `HASURA_BIGQUERY_PROJECT_ID` , most likely caused by the above failure and dodgy subshell scoping
* fix: pass `project_id` as an argument. This might not have been necessary since `generate_test_project` fails faster, but I thought it was clearer to pass explicitly.
`ensure_bigquery_db`
* issue: this could fail, but return a 0 exit code. [example](https://buildkite.com/hasura/graphql-engine-mono/builds/4836#e3dfbc1e-4034-40bb-beb1-181fb5d9489f)
* fix: add a `--fail` flag to `curl` call. Kirill addressed in https://github.com/hasura/graphql-engine-mono/pull/3435/files#diff-0525000dbb36f436dcc17570541378de51471200d294b468c4b288e0292441b6R94
`delete_temp_bigquery_project`
* error: `(gcloud.projects.delete) value for field [projectId] in collection [cloudresourcemanager.projects] is required but was not provided`. [example](https://buildkite.com/hasura/graphql-engine-mono/builds/4836#e3dfbc1e-4034-40bb-beb1-181fb5d9489f)
* issue: attempting to delete a tmp file that didn't exist due to earlier failures
* fix: Kirill addressed in https://github.com/hasura/graphql-engine-mono/pull/3435/files#diff-0525000dbb36f436dcc17570541378de51471200d294b468c4b288e0292441b6R106
to verify: check [bigquery](https://buildkite.com/hasura/graphql-engine-mono/builds/5115#41d059eb-329c-46fb-a745-b2b97fffd328) and [hspec](https://buildkite.com/hasura/graphql-engine-mono/builds/5115#5623a53d-c18d-478e-bf44-446e1287453b) jobs in CI
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3616
Co-authored-by: Divi <32202683+imperfect-fourth@users.noreply.github.com>
GitOrigin-RevId: b85420ef57036b4dbb05202b73ee9e954113bf9d
|
2022-02-21 11:45:03 +00:00 |
|
Kirill Zaborsky
|
960cf2d954
|
Use 'hasura' as dataset in python test to unify it with hspec
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3636
GitOrigin-RevId: f4114fded09cbdd9875488610c635537478e4ab5
|
2022-02-14 06:23:07 +00:00 |
|
Abby Sassel
|
a091110364
|
server/tests: ephemeral BigQuery projects for CI test jobs
_Problem_
We currently run teardown/`DELETE` statements on the same, shared `hasura_test` dataset. This is not ideal as parallel test runs can and do affect each other, resulting in nondeterministic CI failures.
Closes https://github.com/hasura/graphql-engine-mono/issues/2521
_Solution and design_
This PR introduces ephemeral, isolated projects for each test run _in CI only_. Projects are created within [the Google Cloud Platform `data-sources-test-bigquery` directory](https://console.cloud.google.com/iam-admin/settings?folder=704256416468&orgonly=true&supportedpurview=organizationId) on each test run, and destroyed afterwards.
I've only introduced this change in CI for the time being:
1. this isn't as much of an issue locally because we're less likely to run bigquery tests in parallel.
2. to more quickly unblock https://github.com/hasura/graphql-engine/issues/7929.
3. to limit the number of new projects created until we have a better idea of our usage vs GCP quota/limits.
Also updated the [internal wiki here](https://github.com/hasura/graphql-engine-mono/wiki/Testing-BigQuery) with this info.
_To verify_
- CI: [this job](https://buildkite.com/hasura/graphql-engine-mono/builds/3770#89e5bac6-16fe-447e-bcda-85cd47ea1b77) successfully runs all tests on a temporary project & dataset
- local: follow [these steps](https://github.com/hasura/graphql-engine-mono/wiki/Testing-BigQuery#ci--optional-dedicated-gcp-project-for-tests) to try the same setup locally
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3240
GitOrigin-RevId: d88d9cb7922266bfa962cfcb481e0272b8929a5d
|
2022-01-25 08:28:06 +00:00 |
|
Evie Ciobanu
|
db22401e4f
|
server: document python integration test suite
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3350
GitOrigin-RevId: 47dcebbfa84ed3681da40b3ea52e9b849b2fe8e4
|
2022-01-13 14:29:10 +00:00 |
|
Kirill Zaborsky
|
a2afe4116b
|
BigQuery remote joins
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2874
Co-authored-by: Abby Sassel <3883855+sassela@users.noreply.github.com>
GitOrigin-RevId: 24b0304716795a28038629775238996c28b312a3
|
2021-11-24 16:22:55 +00:00 |
|
Abby Sassel
|
a9e3a1b4bd
|
server/bigquery: improve test setup and documentation (close #2345)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2495
GitOrigin-RevId: ef250a0796200be6b75363b4275eb8a7ae780154
|
2021-10-12 17:59:44 +00:00 |
|
Abby Sassel
|
3a6f4e1737
|
server/bigquery: allow empty tables in replace_metadata . Default BigQueryField mode to Nullable
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2466
GitOrigin-RevId: 1f3599d1317235a31c98d7ed1ece2db92d82e916
|
2021-10-01 18:30:01 +00:00 |
|
Abby Sassel
|
16b09f7d52
|
server/mssql: support transactions
https://github.com/hasura/graphql-engine-mono/pull/2268
GitOrigin-RevId: b1bc2812cd403688228b3ecf143aa36b3a6af707
|
2021-09-09 07:59:55 +00:00 |
|
Abby Sassel
|
6b3db83737
|
server/bigquery: run tests in CI
server/bigquery: run tests in CI #1525
[Example of successful BigQuery test job](https://app.circleci.com/pipelines/github/hasura/graphql-engine-mono/15951/workflows/86deb614-ddbc-4df5-9af3-288eeff6504a/jobs/283568)
TODO:
- [ ] I'll make corresponding changes in the OSS repo once this is merged
- [ ] @scriptnull or anyone with permissions: to require in CI
https://github.com/hasura/graphql-engine-mono/pull/1741
GitOrigin-RevId: d4ba446e9b13d213de135c73bb72973478575a29
|
2021-08-12 16:48:29 +00:00 |
|
Abby Sassel
|
1cd9fc376d
|
server/mysql: integrate MySQL tests into dev.sh workflow
https://github.com/hasura/graphql-engine-mono/pull/1848
GitOrigin-RevId: 2bd7c0a18448f042a1e35d21aaf42232acf48a46
|
2021-07-21 17:22:56 +00:00 |
|
Abby Sassel
|
76c322589c
|
server/bigquery: Document BigQuery integration tests
This PR documents & streamlines how we run BigQuery integration tests locally as a first step to [running them in CI](https://github.com/hasura/graphql-engine-mono/issues/1525).
I've also created a hasura service account for internal use. [Internal docs here](https://github.com/hasura/graphql-engine-mono/wiki/Testing-BigQuery).
Thanks to FP Complete team for [the guidance here](https://docs.google.com/document/d/1dGDK0touUtsDxRQPonMxSoPbIfzBoSYo02tAjQEO7qA/edit?ts=60c0cf24#), which I've reused parts of.
https://github.com/hasura/graphql-engine-mono/pull/1732
GitOrigin-RevId: 303819d212aa073fbef685d077b1cfa583cd15fc
|
2021-07-06 11:13:06 +00:00 |
|
Abby Sassel
|
5b7d949ef4
|
server/citus: run tests in CI
https://github.com/hasura/graphql-engine-mono/pull/1526
GitOrigin-RevId: 4fd4214a799bc0d91cd2d29b1ea444f80714f6bc
|
2021-07-01 14:41:12 +00:00 |
|