d95435ad16
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9629 GitOrigin-RevId: 10584252fd97889c68813a21ac817238b0847a40 |
||
---|---|---|
.. | ||
src/Harness | ||
README.md | ||
test-harness.cabal |
Hspec API Test Harness
The test harness provides convenience functions and infrastructure for writing
tests at the API level of graphql-engine
. Specifically, it focuses on setup
and teardown for fixtures across our supported backends, and writing tests
against the GraphQL API.
Setup
The test harness supports running tests using an external GraphQL-Engine
process, as opposed to a server instance running in a thread via library
linking. For this purpose, tests that use this need to know the path to the
graphql-engine
executable via the GRAPHQL_ENGINE
environment variable.
To point a test suite using the harness to the graphql-engine
executable
produced in-tree by cabal, use e.g.:
$ export GRAPHQL_ENGINE=$(cabal list-bin exe:graphql-engine)
(Note that at the time of this writing, almost all of the test suite still tests the Graphql Engine via library linking rather than via external process. However, we expect the external process approach to become gradually more prominent because it is more flexible and true to integration testing.)
The harness depends on the values in src/Harness/Constants.hs
being correct.
Running docker-compose up
in the root of graphql-engine
should take care of
this for everything except BigQuery. Because BigQuery doesn't have a way of
installing locally for testing, we have to connect to an actual BigQuery
account. To test against BigQuery, the following environment variables must be
set:
HASURA_BIGQUERY_SERVICE_KEY
: the service account key.HASURA_BIGQUERY_PROJECT_ID
: the ID of a project on the service account.
Once the account has been verified, the service account email variable can be omitted from subsequent test runs. For the first run, however:
-
Ensure you have access to a Google Cloud Console service account. Store the project ID and account email in
HASURA_BIGQUERY_PROJECT_ID
variable.$ export HASURA_BIGQUERY_PROJECT_ID=hasura-project-identifier
https://cloud.google.com/iam/docs/creating-managing-service-accounts#creating
-
Create and download a new service account key. Store the contents of file in a
HASURA_BIGQUERY_SERVICE_KEY
variable.$ export HASURA_BIGQUERY_SERVICE_KEY=$(cat /path/to/service/account)
https://cloud.google.com/iam/docs/creating-managing-service-account-keys
-
Login and activate the service account if it is not already activated.
https://cloud.google.com/sdk/gcloud/reference/auth/activate-service-account
-
Verify the service account is accessible via the BigQuery API by setting the
HASURA_BIGQUERY_SERVICE_ACCOUNT_EMAIL
with the service account email and running the verification script.$ export HASURA_BIGQUERY_SERVICE_ACCOUNT_EMAIL=hasura@email.com $ ./scripts/verify-bigquery-creds.sh
If the query succeeds, the service account is setup correctly to run tests against BigQuery locally!
Note to Hasura team: a service account is already setup for internal use, please check the wiki for further details.
Logging
Currently, in order to enable logging, you have to edit the engineLogLevel
term in Harness/Constants.hs manually and specify the
level of logging you want to display.
engineLogLevel :: Maybe L.LogLevel
- engineLogLevel = Nothing
+ engineLogLevel = Just L.LevelDebug -- Enable all logs
Library Structure
Harness.Backend
Utilies for interacting with all our specific backends. This includes functions for checking liveness (is the database/backend running?), as well as setting up fixtures (creating tables, inserting data, tracking them in Hasura, setting permissions, and so on).
Harness.Quoter
QuasiQuoters for writing non-Haskell languages such as YAML and GraphQL.
Harness.Test
Specific functions for writing Hspec suites. This directory is probably best
understood starting with Harness.Test.Fixture
, which describes the method for
running tests against one or more backends.
Setup / teardown actions
To ensure things are cleaned up properly in the event of errors in setting up
and tearing down tests, test setup is defined in terms of SetupAction
s.
These look like this:
data SetupAction = forall a.
SetupAction
{ setupAction :: IO a,
teardownAction :: Maybe a -> IO ()
}
A SetupAction
encodes how to setup and tear down a single piece of test
system state. The value produced by a setupAction
is to be input into the
corresponding teardownAction
, if the setupAction
completed without throwing
an exception.
Therefore one SetupAction
could create the DB tables, and the matching
teardown removes them. Pairing setup / teardown in this way makes it easier to
remove everything in the right order.
Setup
The setup actions are responsible for creating the environment for the test. They need to, for example:
- Clear and reconfigure the metadata
- Setup tables and insert values
- Track tables, add relationships, permissions
These actions can be created by running POST requests against graphql-engine
using Harness.GraphqlEngine.post_
, or by running SQL requests against the
backend using Backend.<backend>.run_
.
Teardown
These actions are responsible for freeing acquired resources, and reverting all local modifications: dropping newly created tables, deleting custom functions, removing the changes made to the metadata, and so on.
These actions can be created by running POST requests against graphql-engine
using Harness.GraphqlEngine.post_
, or by running SQL requests against the
backend using Backend.<backend>.run_
.