graphql-engine/server/tests-hspec/Harness/Test/Feature.hs

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

96 lines
2.5 KiB
Haskell
Raw Normal View History

-- | Helper functions for easily testing features.
module Harness.Test.Feature
( feature,
Feature (..),
Backend (..),
BackendOptions (..),
defaultBackendOptions,
)
where
Auto-launch graphql-engine (close hasura/graphql-engine#7801, hasura/graphql-engine#7827 ) Dupe of https://github.com/hasura/graphql-engine-mono/pull/2853 with branch renamed so it doesn't break a tool. prev pr: https://github.com/hasura/graphql-engine-mono/pull/2911 next pr: https://github.com/hasura/graphql-engine-mono/pull/2922 This implements https://github.com/hasura/graphql-engine/issues/7801 Some points to keep in mind for review: * How state is passed to the tests. Do we like how this works? * I quite like it, with the opaque type [`State`](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Harness/State.hs#L17), we can avoid churn when adding things. * The [setup/teardown](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Spec.hs#L19-L31) seems clean. * By using hspec's own means to pass and denote context, we avoid "getting new ideas" for how to structure the tests -- we use a standard. Hopefully, that means the tests' structure rarely change. * The various flags passed in the [ServeOptions](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Harness/Constants.hs#L123) - if there are any causes for concern, raise them here. My thinking is that, there are lots. I've picked some "sane" defaults (mostly empty). The tests pass. I think as we add more complex tests, these flags will be scrutinised and updated as needed. I think it's valuable that all flags are explicitly listed here, though. PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2921 GitOrigin-RevId: 2c2e70bf784ef571a48509a7e5006fd0f48773b5
2021-11-23 21:15:17 +03:00
import Control.Exception
import Data.Foldable
Auto-launch graphql-engine (close hasura/graphql-engine#7801, hasura/graphql-engine#7827 ) Dupe of https://github.com/hasura/graphql-engine-mono/pull/2853 with branch renamed so it doesn't break a tool. prev pr: https://github.com/hasura/graphql-engine-mono/pull/2911 next pr: https://github.com/hasura/graphql-engine-mono/pull/2922 This implements https://github.com/hasura/graphql-engine/issues/7801 Some points to keep in mind for review: * How state is passed to the tests. Do we like how this works? * I quite like it, with the opaque type [`State`](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Harness/State.hs#L17), we can avoid churn when adding things. * The [setup/teardown](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Spec.hs#L19-L31) seems clean. * By using hspec's own means to pass and denote context, we avoid "getting new ideas" for how to structure the tests -- we use a standard. Hopefully, that means the tests' structure rarely change. * The various flags passed in the [ServeOptions](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Harness/Constants.hs#L123) - if there are any causes for concern, raise them here. My thinking is that, there are lots. I've picked some "sane" defaults (mostly empty). The tests pass. I think as we add more complex tests, these flags will be scrutinised and updated as needed. I think it's valuable that all flags are explicitly listed here, though. PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2921 GitOrigin-RevId: 2c2e70bf784ef571a48509a7e5006fd0f48773b5
2021-11-23 21:15:17 +03:00
import Harness.State (State)
import Test.Hspec
import Prelude
-- | Use this record to put together a test against a set of backends.
data Feature = Feature
{ backends :: [Backend],
tests :: BackendOptions -> SpecWith State
}
-- | A backend specification.
data Backend = Backend
{ -- | Can be any name you want (e.g. "PostgreSQL" or "MySQL v1.2")
-- or whatnot.
name :: String,
-- | To setup the test suite: Run SQL commands, run metadata track
-- tables calls.
Auto-launch graphql-engine (close hasura/graphql-engine#7801, hasura/graphql-engine#7827 ) Dupe of https://github.com/hasura/graphql-engine-mono/pull/2853 with branch renamed so it doesn't break a tool. prev pr: https://github.com/hasura/graphql-engine-mono/pull/2911 next pr: https://github.com/hasura/graphql-engine-mono/pull/2922 This implements https://github.com/hasura/graphql-engine/issues/7801 Some points to keep in mind for review: * How state is passed to the tests. Do we like how this works? * I quite like it, with the opaque type [`State`](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Harness/State.hs#L17), we can avoid churn when adding things. * The [setup/teardown](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Spec.hs#L19-L31) seems clean. * By using hspec's own means to pass and denote context, we avoid "getting new ideas" for how to structure the tests -- we use a standard. Hopefully, that means the tests' structure rarely change. * The various flags passed in the [ServeOptions](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Harness/Constants.hs#L123) - if there are any causes for concern, raise them here. My thinking is that, there are lots. I've picked some "sane" defaults (mostly empty). The tests pass. I think as we add more complex tests, these flags will be scrutinised and updated as needed. I think it's valuable that all flags are explicitly listed here, though. PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2921 GitOrigin-RevId: 2c2e70bf784ef571a48509a7e5006fd0f48773b5
2021-11-23 21:15:17 +03:00
setup :: State -> IO (),
-- | Clean up any resources you created in 'setup'.
teardown :: State -> IO (),
-- | Backend-specific details which should be taken into account in tests
backendOptions :: BackendOptions
}
data BackendOptions = BackendOptions
{ -- | Defines whether numeric values for the particular backend output as strings
stringifyNumbers :: Bool
}
defaultBackendOptions :: BackendOptions
defaultBackendOptions = BackendOptions {stringifyNumbers = False}
-- | Test the feature, running the setup before any tests are run, and
-- and ensuring teardown happens after all tests are run.
Auto-launch graphql-engine (close hasura/graphql-engine#7801, hasura/graphql-engine#7827 ) Dupe of https://github.com/hasura/graphql-engine-mono/pull/2853 with branch renamed so it doesn't break a tool. prev pr: https://github.com/hasura/graphql-engine-mono/pull/2911 next pr: https://github.com/hasura/graphql-engine-mono/pull/2922 This implements https://github.com/hasura/graphql-engine/issues/7801 Some points to keep in mind for review: * How state is passed to the tests. Do we like how this works? * I quite like it, with the opaque type [`State`](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Harness/State.hs#L17), we can avoid churn when adding things. * The [setup/teardown](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Spec.hs#L19-L31) seems clean. * By using hspec's own means to pass and denote context, we avoid "getting new ideas" for how to structure the tests -- we use a standard. Hopefully, that means the tests' structure rarely change. * The various flags passed in the [ServeOptions](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Harness/Constants.hs#L123) - if there are any causes for concern, raise them here. My thinking is that, there are lots. I've picked some "sane" defaults (mostly empty). The tests pass. I think as we add more complex tests, these flags will be scrutinised and updated as needed. I think it's valuable that all flags are explicitly listed here, though. PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2921 GitOrigin-RevId: 2c2e70bf784ef571a48509a7e5006fd0f48773b5
2021-11-23 21:15:17 +03:00
feature :: Feature -> SpecWith State
feature Feature {backends, tests} =
for_
backends
( \Backend {name, setup, teardown, backendOptions} ->
describe
name
Auto-launch graphql-engine (close hasura/graphql-engine#7801, hasura/graphql-engine#7827 ) Dupe of https://github.com/hasura/graphql-engine-mono/pull/2853 with branch renamed so it doesn't break a tool. prev pr: https://github.com/hasura/graphql-engine-mono/pull/2911 next pr: https://github.com/hasura/graphql-engine-mono/pull/2922 This implements https://github.com/hasura/graphql-engine/issues/7801 Some points to keep in mind for review: * How state is passed to the tests. Do we like how this works? * I quite like it, with the opaque type [`State`](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Harness/State.hs#L17), we can avoid churn when adding things. * The [setup/teardown](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Spec.hs#L19-L31) seems clean. * By using hspec's own means to pass and denote context, we avoid "getting new ideas" for how to structure the tests -- we use a standard. Hopefully, that means the tests' structure rarely change. * The various flags passed in the [ServeOptions](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Harness/Constants.hs#L123) - if there are any causes for concern, raise them here. My thinking is that, there are lots. I've picked some "sane" defaults (mostly empty). The tests pass. I think as we add more complex tests, these flags will be scrutinised and updated as needed. I think it's valuable that all flags are explicitly listed here, though. PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2921 GitOrigin-RevId: 2c2e70bf784ef571a48509a7e5006fd0f48773b5
2021-11-23 21:15:17 +03:00
( aroundAllWith
( \actionWith state -> do
finallyRethrow
( do
setup state
actionWith state
)
(teardown state)
Auto-launch graphql-engine (close hasura/graphql-engine#7801, hasura/graphql-engine#7827 ) Dupe of https://github.com/hasura/graphql-engine-mono/pull/2853 with branch renamed so it doesn't break a tool. prev pr: https://github.com/hasura/graphql-engine-mono/pull/2911 next pr: https://github.com/hasura/graphql-engine-mono/pull/2922 This implements https://github.com/hasura/graphql-engine/issues/7801 Some points to keep in mind for review: * How state is passed to the tests. Do we like how this works? * I quite like it, with the opaque type [`State`](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Harness/State.hs#L17), we can avoid churn when adding things. * The [setup/teardown](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Spec.hs#L19-L31) seems clean. * By using hspec's own means to pass and denote context, we avoid "getting new ideas" for how to structure the tests -- we use a standard. Hopefully, that means the tests' structure rarely change. * The various flags passed in the [ServeOptions](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Harness/Constants.hs#L123) - if there are any causes for concern, raise them here. My thinking is that, there are lots. I've picked some "sane" defaults (mostly empty). The tests pass. I think as we add more complex tests, these flags will be scrutinised and updated as needed. I think it's valuable that all flags are explicitly listed here, though. PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2921 GitOrigin-RevId: 2c2e70bf784ef571a48509a7e5006fd0f48773b5
2021-11-23 21:15:17 +03:00
)
(tests backendOptions)
Auto-launch graphql-engine (close hasura/graphql-engine#7801, hasura/graphql-engine#7827 ) Dupe of https://github.com/hasura/graphql-engine-mono/pull/2853 with branch renamed so it doesn't break a tool. prev pr: https://github.com/hasura/graphql-engine-mono/pull/2911 next pr: https://github.com/hasura/graphql-engine-mono/pull/2922 This implements https://github.com/hasura/graphql-engine/issues/7801 Some points to keep in mind for review: * How state is passed to the tests. Do we like how this works? * I quite like it, with the opaque type [`State`](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Harness/State.hs#L17), we can avoid churn when adding things. * The [setup/teardown](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Spec.hs#L19-L31) seems clean. * By using hspec's own means to pass and denote context, we avoid "getting new ideas" for how to structure the tests -- we use a standard. Hopefully, that means the tests' structure rarely change. * The various flags passed in the [ServeOptions](https://github.com/hasura/graphql-engine-mono/blob/68f33051ca6373c42988e78d069eea2a135dc190/server/tests-hspec/Harness/Constants.hs#L123) - if there are any causes for concern, raise them here. My thinking is that, there are lots. I've picked some "sane" defaults (mostly empty). The tests pass. I think as we add more complex tests, these flags will be scrutinised and updated as needed. I think it's valuable that all flags are explicitly listed here, though. PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2921 GitOrigin-RevId: 2c2e70bf784ef571a48509a7e5006fd0f48773b5
2021-11-23 21:15:17 +03:00
)
)
-- | A custom 'finally' which re-throws exceptions from both the main action and the sequel action.
--
-- The standard 'finally' only re-throws the @sequel@ exception.
finallyRethrow :: IO a -> IO b -> IO a
finallyRethrow a sequel =
mask $ \restore -> do
r <-
catch
(restore a)
( \restoreEx -> do
_ <- sequel `catch` (throwIO . Exceptions restoreEx)
(throwIO restoreEx)
)
_ <- sequel
pure r
-- | Two exceptions
data Exceptions
= Exceptions SomeException SomeException
instance Show Exceptions where
show (Exceptions e1 e2) =
unlines
[ "1. " <> show e1,
"",
"2. " <> show e2
]
instance Exception Exceptions