graphql-engine/server/tests-py/super_classes.py

76 lines
2.2 KiB
Python
Raw Normal View History

import pytest
from abc import ABC, abstractmethod
run graphql tests on both http and websocket; add parallelism (close #1868) (#1921) Examples 1) ` pytest --hge-urls "http://127.0.0.1:8080" --pg-urls "postgresql://admin@127.0.0.1:5432/hge_tests" -vv ` 2) `pytest --hge-urls "http://127.0.0.1:8080" "http://127.0.0.1:8081" --pg-urls "postgresql://admin@127.0.0.1:5432/hge_tests" "postgresql://admin@127.0.0.1:5432/hge_tests2" -vv ` ### Solution and Design <!-- How is this issue solved/fixed? What is the design? --> <!-- It's better if we elaborate --> #### Reducing execution time of tests - The Schema setup and teardown, which were earlier done per test method, usually takes around 1 sec. - For mutations, the model has now been changed to only do schema setup and teardown once per test class. - A data setup and teardown will be done once per test instead (usually takes ~10ms). - For the test class to get this behaviour, one can can extend the class `DefaultTestMutations`. - The function `dir()` should be define which returns the location of the configuration folder. - Inside the configuration folder, there should be - Files `<conf_dir>/schema_setup.yaml` and `<conf_dir>/schema_teardown.yaml`, which has the metadata query executed during schema setup and teardown respectively - Files named `<conf_dir>/values_setup.yaml` and `<conf_dir>/values_teardown.yaml`. These files are executed to setup and remove data from the tables respectively. #### Running Graphql queries on both http and websockets - Each GraphQL query/mutation is run on the both HTTP and websocket protocols - Pytests test parameterisation is used to achieve this - The errors over websockets are slightly different from that on HTTP - The code takes care of converting the errors in HTTP to errors in websockets #### Parallel executation of tests. - The plugin pytest-xdist helps in running tests on parallel workers. - We are using this plugin to group tests by file and run on different workers. - Parallel test worker processes operate on separate postgres databases(and separate graphql-engines connected to these databases). Thus tests on one worker will not affect the tests on the other worker. - With two workers, this decreases execution times by half, as the tests on event triggers usually takes a long time, but does not consume much CPU.
2019-04-08 10:22:38 +03:00
import os
class DefaultTestQueries(ABC):
def do_setup(self, setup_ctrl, hge_ctx):
if not setup_ctrl['setupDone']:
st_code, resp = hge_ctx.v1q_f(self.dir() + '/setup.yaml')
assert st_code == 200, resp
setup_ctrl['setupDone'] = True
def do_teardown(self, setup_ctrl, hge_ctx):
if setup_ctrl['setupDone'] and not hge_ctx.may_skip_test_teardown:
st_code, resp = hge_ctx.v1q_f(self.dir() + '/teardown.yaml')
assert st_code == 200, resp
setup_ctrl['setupDone'] = False
@pytest.fixture(autouse=True)
def transact(self, setup_ctrl, hge_ctx):
self.do_setup(setup_ctrl, hge_ctx)
yield
self.do_teardown(setup_ctrl, hge_ctx);
@abstractmethod
def dir(self):
pass
run graphql tests on both http and websocket; add parallelism (close #1868) (#1921) Examples 1) ` pytest --hge-urls "http://127.0.0.1:8080" --pg-urls "postgresql://admin@127.0.0.1:5432/hge_tests" -vv ` 2) `pytest --hge-urls "http://127.0.0.1:8080" "http://127.0.0.1:8081" --pg-urls "postgresql://admin@127.0.0.1:5432/hge_tests" "postgresql://admin@127.0.0.1:5432/hge_tests2" -vv ` ### Solution and Design <!-- How is this issue solved/fixed? What is the design? --> <!-- It's better if we elaborate --> #### Reducing execution time of tests - The Schema setup and teardown, which were earlier done per test method, usually takes around 1 sec. - For mutations, the model has now been changed to only do schema setup and teardown once per test class. - A data setup and teardown will be done once per test instead (usually takes ~10ms). - For the test class to get this behaviour, one can can extend the class `DefaultTestMutations`. - The function `dir()` should be define which returns the location of the configuration folder. - Inside the configuration folder, there should be - Files `<conf_dir>/schema_setup.yaml` and `<conf_dir>/schema_teardown.yaml`, which has the metadata query executed during schema setup and teardown respectively - Files named `<conf_dir>/values_setup.yaml` and `<conf_dir>/values_teardown.yaml`. These files are executed to setup and remove data from the tables respectively. #### Running Graphql queries on both http and websockets - Each GraphQL query/mutation is run on the both HTTP and websocket protocols - Pytests test parameterisation is used to achieve this - The errors over websockets are slightly different from that on HTTP - The code takes care of converting the errors in HTTP to errors in websockets #### Parallel executation of tests. - The plugin pytest-xdist helps in running tests on parallel workers. - We are using this plugin to group tests by file and run on different workers. - Parallel test worker processes operate on separate postgres databases(and separate graphql-engines connected to these databases). Thus tests on one worker will not affect the tests on the other worker. - With two workers, this decreases execution times by half, as the tests on event triggers usually takes a long time, but does not consume much CPU.
2019-04-08 10:22:38 +03:00
class DefaultTestMutations(ABC):
@pytest.fixture(scope='class')
def schema_transact(self, request, hge_ctx):
st_code, resp = hge_ctx.v1q_f(self.dir() + '/schema_setup.yaml')
assert st_code == 200, resp
yield
st_code, resp = hge_ctx.v1q_f(self.dir() + '/schema_teardown.yaml')
assert st_code == 200, resp
@pytest.fixture(autouse=True)
def init_values_transact(self, schema_transact, hge_ctx):
setupValFile = self.dir() + '/values_setup.yaml'
if os.path.isfile(setupValFile):
st_code, resp = hge_ctx.v1q_f(setupValFile)
assert st_code == 200, resp
yield
st_code, resp = hge_ctx.v1q_f(self.dir() + '/values_teardown.yaml')
assert st_code == 200, resp
@abstractmethod
def dir(self):
pass
# Any test which has a setup and a teardown
# Ideally, DefaultTestSelectQueries should just be this
class GraphQLEngineTest(ABC):
@pytest.fixture(scope='class')
def transact(self, request, hge_ctx):
st_code, resp = hge_ctx.v1q_f(self.dir() + '/setup.yaml')
assert st_code == 200, resp
yield
st_code, resp = hge_ctx.v1q_f(self.dir() + '/teardown.yaml')
assert st_code == 200, resp
@pytest.fixture(autouse=True)
def ensure_transact(self, transact):
pass
@abstractmethod
def dir(self):
pass
class DefaultTestSelectQueries(GraphQLEngineTest):
pass