graphql-engine/server/tests-py/test_horizontal_scale.py
Brandon Simmons 91aee7fdeb Test result ordering, add --accept test mode to automatically accept changed test cases
We add a new pytest flag `--accept` that will automatically write back
yaml files with updated responses. This makes it much easier and less
error-prone to update test cases when we expect output to change, or
when authoring new tests.

Second we make sure to test that we actually preserve the order of the
selection set when returning results. This is a "SHOULD" part of the
spec but seems pretty important and something that users will rely on.

To support both of the above we use ruamel.yaml which preserves a
certain amount of formatting and comments (so that --accept can work in
a failry ergonomic way), as well as ordering (so that when we write yaml
the order of keys has meaning that's preserved during parsing).

Use ruamel.yaml everywhere for consistency (since both libraries have
different quirks).

Quirks of ruamel.yaml:
- trailing whitespace in multiline strings in yaml files isn't written
  back out as we'd like: https://bitbucket.org/ruamel/yaml/issues/47/multiline-strings-being-changed-if-they
- formatting is only sort of preserved; ruamel e.g. normalizes
  indentation. Normally the diff is pretty clean though, and you can
  always just check in portions of your test file after --accept

fixup
2019-11-05 15:15:25 -06:00

59 lines
1.9 KiB
Python

import pytest
import ruamel.yaml as yaml
import time
import jsondiff
if not pytest.config.getoption("--test-hge-scale-url"):
pytest.skip("--test-hge-scale-url flag is missing, skipping tests", allow_module_level=True)
class TestHorizantalScaleBasic():
servers = {}
@pytest.fixture(autouse=True, scope='class')
def transact(self, hge_ctx):
self.servers['1'] = hge_ctx.hge_url
self.servers['2'] = hge_ctx.hge_scale_url
yield
# teardown
st_code, resp = hge_ctx.v1q_f(self.dir() + '/teardown.yaml')
assert st_code == 200, resp
def test_horizontal_scale_basic(self, hge_ctx):
with open(self.dir() + "/steps.yaml") as c:
conf = yaml.safe_load(c)
assert isinstance(conf, list) == True, 'Not an list'
for _, step in enumerate(conf):
# execute operation
response = hge_ctx.http.post(
self.servers[step['operation']['server']] + "/v1/query",
json=step['operation']['query']
)
st_code = response.status_code
resp = response.json()
assert st_code == 200, resp
# wait for 20 sec
time.sleep(20)
# validate data
response = hge_ctx.http.post(
self.servers[step['validate']['server']] + "/v1alpha1/graphql",
json=step['validate']['query']
)
st_code = response.status_code
resp = response.json()
assert st_code == 200, resp
if 'response' in step['validate']:
assert resp == step['validate']['response'], yaml.dump({
'response': resp,
'expected': step['validate']['response'],
'diff': jsondiff.diff(step['validate']['response'], resp)
})
@classmethod
def dir(cls):
return 'queries/horizontal_scale/basic'