We seem to be getting flakes where we try and use the same port for two different servers. This is because in certain cases we cannot simply allocate the port dynamically, but have to decide it in advance, leading to a race condition.
We resolve this by keeping track of the ports we allocate when using this method, making sure we never allocate them twice. We also make sure we allocate from a different pool of ports to the usual dynamic port pool (typically above port 32768, and often above port 49152).
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8903
GitOrigin-RevId: 375a23867591a4566493dddbc550c58cf88ea392
This adds the ability to capture logs to the HGE fixture, and uses this in test_logging.py to analyze the logs, instead of relying on a shell script redirecting the logs to a file.
We then inject the logs into the tests and parse the JSON. Because we're no longer reading a file, we need to do this in a separate thread, as we'll block on reading rather than the stream ending. (Once HGE stops, the stream will be closed.)
Some of the tests require a JWK server, so this has been extracted from test_jwk.py.
[NDAT-540]: https://hasurahq.atlassian.net/browse/NDAT-540?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8772
GitOrigin-RevId: 9413e714f1c42b8a0991d0d30c4358209fd30c0c
This rewrites the last couple of Python tests that were failing when run with a separate HGE binary per test class. The changes are as follows:
1. The event triggers tests, naming conventions tests, and subscriptions tests all generate a new source DB per test, so can run in parallel.
2. The scheduled triggers tests use the correct URL for the trigger service when the port is generated randomly.
3. Whitespace and trailing commas are added to the scheduled triggers tests.
4. Support for SQL Server is added to _hge.py_ so the naming conventions test that runs on SQL Server passes. (The other SQL Server tests do not pass and we're not going to bother with them for now.)
5. Container names are fixed in _run.sh_.
6. _run.sh_ and _run-new.sh_ don't pull images explicitly as it's annoying when running tests a lot. If you want to pull the latest versions, just run `docker compose pull` from the _server/tests-py_ directory, or the root directory. (If you don't have the images at all, they'll still be pulled automatically.)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7350
GitOrigin-RevId: db58f310f017b2a0884fcf61ccc56d15583f99bd
When running using the "new" style (with a HGE binary, not a URL), a new PostgreSQL metadata and source database are created for each test. When we get this into CI, this should drastically reduce the flakiness.
I have also enabled parallelization by default when using `run-new.sh`. It's much faster.
I had to basically rewrite _server/tests-py/test_graphql_read_only_source.py_ so that it does two different things depending on how it's run. It's unfortunate, but it should eventually go away.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6879
GitOrigin-RevId: a121b9035f8da3e61a3e36d8b1fbc6ccae918fad
When we run the HGE server inside the test harness, it needs to run with
an admin secret for some tests to make sense. This tags each test that
requires an admin secret with `pytest.mark.admin_secret`, which then
generates a UUID and injects that into both the server and the test case
(if required).
It also simplifies the way the test harness picks up an existing admin
secret, allowing it to use the environment variable instead of requiring
it via a parameter.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6120
GitOrigin-RevId: 55c5b9e8c99bdad9c8304098444ddb9516749a2c
This has two purposes:
* When running the Python integration tests against a running HGE instance, with `--hge-url`, it will check the environment variables available and actively skip the test if they aren't set. This replaces the previous ad-hoc skip behavior.
* More interestingly, when running against a binary with `--hge-bin`, the environment variables are passed through, which means different tests can run with different environment variables.
On top of this, the various services we use for testing now also provide their own environment variables, rather than expecting a test script to do it.
In order to make this work, I also had to invert the dependency between various services and `hge_ctx`. I extracted a `pg_version` fixture to provide the PostgreSQL version, and now pass the `hge_url` and `hge_key` explicitly to `ActionsWebhookServer`.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6028
GitOrigin-RevId: 16d866741dba5887da1adf4e1ade8182ccc9d344
This argument allows the user to specify how to run HGE, rather than starting it beforehand. The runner will start a new instance of HGE for each test class.
This does not provide isolation, as the database is still re-used, but it helps us get closer.
You can try it yourself by executing:
```
$ cabal build graphql-engine:exe:graphql-engine
$ ./server/tests-py/run-new.sh
```
This doesn't affect CI at all.
I also fixed a few warnings flagged by Pylance.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5881
GitOrigin-RevId: ea6f0fd631a2c278b2c6b50e9dbdd9d804ebc9d4
