hasura/graphql-engine
1
0
Fork 0
mirror of https://github.com/hasura/graphql-engine.git synced 2024-09-21 07:28:26 +03:00
Code Issues Projects Releases Wiki Activity

server: Fix up the instructions on testing against BigQuery.

Browse Source 
This improves the experience of testing against BigQuery in a few ways:

1.  There is an explicit instruction to create a dataset, which was not
    documented.
2.  The verification script uses the environment variables, just like the
    tests.
3.  The verification script passes the correct content type headers.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4706
GitOrigin-RevId: 15b5bd28b1762c130c0ec09d6babe8c08e05ab15
This commit is contained in:
Samir Talwar 2022-06-15 11:44:13 +02:00 committed by hasura-bot
parent 0f553cf1a3
commit 1f2f9a8f94
2 changed files with 52 additions and 20 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

33
scripts/verify-bigquery-creds.sh
View File

@ -1,12 +1,33 @@
#!/usr/bin/env bash
set -e
set -u
# Suggested usage:
# https://github.com/hasura/graphql-engine/blob/master/server/py-tests/README.md#running-bigquery-tests
# https://github.com/hasura/graphql-engine/blob/master/server/tests-hspec/README.md#required-setup-for-bigquery-tests
# https://cloud.google.com/iam/docs/creating-managing-service-accounts#iam-service-accounts-create-rest
project_id=${1}
api_key=${2}
service_account_email=${3} # eg. "<<SERVICE_ACCOUNT_NAME>>@<<PROJECT_NAME>>.iam.gserviceaccount.com"
if [[ -z "${HASURA_BIGQUERY_SERVICE_KEY:-}" ]]; then
echo >&2 'You must set the HASURA_BIGQUERY_SERVICE_KEY environment variable.'
echo >&2 'Please see the test README for details.'
exit 2
fi
curl "https://content-bigquery.googleapis.com/bigquery/v2/projects/$project_id/queries?alt=json&key=$api_key" \
--data-binary '{"query":"select 123"}' \
if [[ -z "${HASURA_BIGQUERY_PROJECT_ID:-}" ]]; then
echo >&2 'You must set the HASURA_BIGQUERY_PROJECT_ID environment variable.'
echo >&2 'Please see the test README for details.'
exit 2
fi
project_id="$(jq -r -n --argjson service_account "$HASURA_BIGQUERY_SERVICE_KEY" '$service_account.project_id')"
service_account_email="$(jq -r -n --argjson service_account "$HASURA_BIGQUERY_SERVICE_KEY" '$service_account.client_email')"
if [[ "$HASURA_BIGQUERY_PROJECT_ID" != "$project_id" ]]; then
echo >&2 'The HASURA_BIGQUERY_PROJECT_ID must be set to the same value as specified in the service key.'
exit 2
fi
curl --fail \
"https://content-bigquery.googleapis.com/bigquery/v2/projects/${project_id}/queries" \
--json '{"query": "select 123"}' \
-H "Authorization: Bearer $(gcloud auth print-access-token "$service_account_email" --project="$project_id")"

39
server/tests-hspec/README.md
View File

@ -30,6 +30,7 @@ For motivation, rationale, and more, see the [test suite rfc](../../rfcs/hspec-t
## Required setup for BigQuery tests
Running integration tests against a BigQuery data source is a more involved due to the necessary service account requirements:
```
HASURA_BIGQUERY_PROJECT_ID=# the project ID of the service account
HASURA_BIGQUERY_SERVICE_KEY=# the service account key
@ -38,24 +39,34 @@ HASURA_BIGQUERY_SERVICE_ACCOUNT_EMAIL=# eg. "<<SERVICE_ACCOUNT_NAME>>@<<PROJECT_
```
Before running the test suite:
1. Ensure you have access to a [Google Cloud Console service account](https://cloud.google.com/iam/docs/creating-managing-service-accounts#creating). Store the project ID and account email in `HASURA_BIGQUERY_PROJECT_ID` variable.
2. [Create and download a new service account key](https://cloud.google.com/iam/docs/creating-managing-service-account-keys). Store the contents of file in a `HASURA_BIGQUERY_SERVICE_KEY` variable.
1. Ensure you have access to a [Google Cloud Console service account](https://cloud.google.com/iam/docs/creating-managing-service-accounts#creating). Store the project ID and account email in `HASURA_BIGQUERY_PROJECT_ID` variable.
2. [Create and download a new service account key](https://cloud.google.com/iam/docs/creating-managing-service-account-keys). Store the contents of file in a `HASURA_BIGQUERY_SERVICE_KEY` variable.
```bash
export HASURA_BIGQUERY_SERVICE_KEY=$(cat /path/to/service/account)
```
3. [Login and activate the service account](https://cloud.google.com/sdk/gcloud/reference/auth/activate-service-account), if it is not already activated.
4. Verify the service account is accessible via the [BigQuery API](https://cloud.google.com/bigquery/docs/reference/rest):
1. Run the following command:
```bash
source scripts/verify-bigquery-creds.sh $HASURA_BIGQUERY_PROJECT_ID $HASURA_BIGQUERY_SERVICE_KEY $HASURA_BIGQUERY_SERVICE_ACCOUNT_EMAIL
```
If the query succeeds, the service account is setup correctly to run tests against BigQuery locally.
5. Finally, run the BigQuery tests once the `HASURA_BIGQUERY_SERVICE_KEY` and `HASURA_BIGQUERY_PROJECT_ID` environment variables set. For example:
```
cabal run tests-hspec -- -m "BigQuery"
```
*Note to Hasura team: a service account is already setup for internal use, please check the wiki for further details.*
3. [Login and activate the service account](https://cloud.google.com/sdk/gcloud/reference/auth/activate-service-account), if it is not already activated.
4. Verify the service account is accessible via the [BigQuery API](https://cloud.google.com/bigquery/docs/reference/rest), by running the following command:
```bash
./scripts/verify-bigquery-creds.sh
```
If the query succeeds, the service account is setup correctly to run tests against BigQuery locally.
5. If necessary, create a dataset called "hasura" in the [BigQuery workspace](https://console.cloud.google.com/bigquery).
6. Finally, run the BigQuery tests once the `HASURA_BIGQUERY_SERVICE_KEY` and `HASURA_BIGQUERY_PROJECT_ID` environment variables set. For example:
```
cabal run tests-hspec -- -m "BigQuery"
```
_Note to Hasura team: a service account is already setup for internal use, please check the wiki for further details._
## Running the test suite