Commit Graph

83 Commits

Author SHA1 Message Date
Samir Talwar
f430e5b599 dev.sh: Fix mssql_wait to use mssql-tools to run sqlcmd.
On macOS, we use the `azure-sql-edge` image, which doesn't contain
`sqlcmd`. To work around this, we can use another container which _does_
contain the `sqlcmd` binary to verify that MS SQL Server is up.

This was also broken on Linux anyway.

This also updates the PostgreSQL image, in line with _databases.yaml_.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7204
GitOrigin-RevId: ad974cd353c348ec055a153cbba5cb39d85e0967
2022-12-08 10:00:28 +00:00
Samir Talwar
e12923a1b3 Demand Python 3.9 to run the server's Python integration tests.
CI has already been upgraded.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5857
GitOrigin-RevId: 28a56c5fd61f425b76df478fd230e6944ce8122f
2022-09-13 20:20:19 +00:00
Samir Talwar
38c94d8cda server/tests-py: Fix some warnings.
They're getting in the way of real problems.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5541
GitOrigin-RevId: 9bc99a0826715f6df77055cca79e428ad49742a8
2022-08-24 08:32:07 +00:00
Samir Talwar
2757c2156b server/tests-py: Move installing node_modules to the makefile.
This means that if `remote_schemas/nodejs/package.json` changes, the
dependencies will be automatically reinstalled.

It also moves `package-lock.json` to the correct location (in the
directory in which we run `npm install`), and updates it.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5481
GitOrigin-RevId: f3fb431afd19de150f39ec2e4cb6572b896c870f
2022-08-16 09:27:04 +00:00
Samir Talwar
b690a3fc3a server/tests-py: Install the pytest-durations plugin.
[pytest-durations][] is an enhancement to pytest over the built-in
[`--durations` parameter][] which also shows the duration of setup,
teardown, and fixtures that take the longest.

While it's not strictly necessary now, I used it to figure out what was
taking so long when trying to run things in parallel, and found it very
helpful, so I think it's worth preserving.

[pytest-durations]: https://pypi.org/project/pytest-durations/
[`--durations` parameter]: https://docs.pytest.org/en/6.2.x/usage.html#profiling-test-execution-duration

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5480
GitOrigin-RevId: b789418dd4c971adb91e08b7837a962bede44452
2022-08-15 16:15:37 +00:00
Samir Talwar
b856d96989 server/tests-py: Factor out service URLs as environment variables.
Making it easier to inject different ones later.

I also included a change to _.prettierignore_ so Visual Studio Code doesn't keep trying to reformat the JavaScript or YAML files in `server/tests-py`, as it can cause diffs to balloon for no obvious benefit.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5456
GitOrigin-RevId: bc6d548708160a328e1e61a00e19be8e124da025
2022-08-15 14:59:05 +00:00
Samir Talwar
1e1a36a192 server/tests-py: Use environment variables for services in queries.
I'm trying to shore up the Python integration tests to make them more reliable. In doing so, I noticed this.

---

Rather than hard-coding hostnames and ports, we can (and already do) inject these into the HGE process using environment variables.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5255
GitOrigin-RevId: 6bb593999ece42cedef6619f31f9d9b2e39f30ef
2022-08-03 20:05:46 +00:00
Samir Talwar
34ec4d0278 Restore scripts/dev.sh.
Not sure how that snuck in. Apologies.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5307
GitOrigin-RevId: ffcccbddb86cd002fb14964fc6eb0120b29e1c37
2022-08-03 10:27:04 +00:00
Samir Talwar
349ccd3296 Use docker compose, not docker-compose.
Docker Compose is now a plugin for Docker, bundled by default in Docker Desktop and many Linux distribution packages. The standalone `docker-compose` binary has been deprecated since Docker Compose v2.

Using the new version directly allows us to write development scripts that do not require `docker-compose` to be installed.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5185
GitOrigin-RevId: c8542b8b2405d1aa32288991688c6fde4af96383
2022-08-02 10:30:57 +00:00
Samir Talwar
a471c5080b Nix: Upgrade Python to the latest version.
There's no reason to pin it to v3.7. This wasn't causing the Docker
Compose issues; it's a dependency version issue.

We actually don't need to install `docker-compose` at all. It's now a
Docker plugin (and is bundled in Docker Desktop) since Docker Compose
v2. You can run it with `docker compose`.

This also fixes a bug in _dev.sh_ due to a moved `make` target.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5197
GitOrigin-RevId: 638b2eb39c99e7a990e7a6c66388831438aed45f
2022-07-26 16:47:18 +00:00
Samir Talwar
adca95a78d server: Use make to simplify tests-py venv generation.
Rather than a homebrewed approach, we can use `make` to figure out when
it's necessary to regenerate our venv.

This Makefile will regenerate _requirements.txt_ from
_requirements-top-level.txt_ when the latter is changed.

It will also regenerate the venv when _requirements.txt_ is changed
(i.e. changes are pulled, or it's regenerated as described above).

`make` uses file/directory timestamps to figure out what to rebuild.
This is probably more reliable than expecting people to update a version
number whenever they change a file.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5152
GitOrigin-RevId: 24b27d49bf6c4ba1d57ac38ea38ae278216c6d66
2022-07-25 20:05:49 +00:00
paritosh-08
95adde4ce2 server: support for Apollo federation
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4584
Co-authored-by: Auke Booij <164426+abooij@users.noreply.github.com>
GitOrigin-RevId: 0f60c263efb5fbaa25620dd8159e8cfda25a61b2
2022-07-25 15:54:41 +00:00
Samir Talwar
89eafb1589 server: Re-freeze tests-py/requirements.txt.
I tried re-freezing _server/tests-py/requirements-top-level.txt_
recently, and discovered that it caused the tests to fail.

This pins a couple of dependencies so that we can safely re-freeze.
Specifically:

- `cryptography` is pinned at v3.*
- `graphene` is pinned at v2.*
- `PyJWT` is pinned at v2.3.*
- `websocket-client` is pinned at v0.56.0 (this was done in
  _requirements.txt_ already, but that file is supposed to be
  regenerated)

Upgrading `SQLAlchemy` required changing PostgreSQL URLs to use
"postgresql://" as the URL scheme, not "postgres://".

Updating `ruamel.yaml` caused a few tests to fail as we are passing
`ruamel.yaml.scalarstring.LiteralScalarString` values as header values.
This is fixed by explicitly converting header values to strings.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5120
GitOrigin-RevId: 9c12a3013c3d1f23dddbe781037663838b23f6f5
2022-07-22 10:25:33 +00:00
Evie Ciobanu
d9d193d0e6 Evie/nix/fix nix shell
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5089
GitOrigin-RevId: 584a42ea6d2af08b98d592ed9c31bb1d2df540ee
2022-07-18 21:39:48 +00:00
Samir Talwar
75c01223a8 server: Use plain assertions in pytest.
When pytest rewrites assertions to make them more useful, it also
truncates long assertion messages in the middle, often obscuring the
actual error.

Disabling this allows us to see the full message, which should hopefully
allow us to see the actual error.

In order to support this, we need to make sure that custom assertion
messages include the actual assertion information, as this will no
longer be rendered by pytest.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4835
GitOrigin-RevId: de6839a3b40d0edc7cb96e46625eebca0aaf0c95
2022-06-25 00:04:16 +00:00
Brandon Simmons
6e8da71ece server: migrate to aeson-2 in preparation for ghc 9.2 upgrade
(Work here originally done by awjchen, rebased and fixed up for merge by
jberryman)

This is part of a merge train towards GHC 9.2 compatibility. The main
issue is the use of the new abstract `KeyMap` in 2.0. See:
https://hackage.haskell.org/package/aeson-2.0.3.0/changelog

Alex's original work is here:
#4305

BEHAVIOR CHANGE NOTE: This change causes a different arbitrary ordering
of serialized Json, for example during metadata export. CLI users care
about this in particular, and so we need to call it out as a _behavior
change_ as we did in v2.5.0. The good news though is that after this
change ordering should be more stable (alphabetical key order).

See: https://hasurahq.slack.com/archives/C01M20G1YRW/p1654012632634389

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4611
Co-authored-by: awjchen <13142944+awjchen@users.noreply.github.com>
GitOrigin-RevId: 700265162c782739b2bb88300ee3cda3819b2e87
2022-06-08 15:32:27 +00:00
Puru Gupta
382e99f3cc Add ACTION_WEBHOOK_HANDLER env variable to dev.sh
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4602
GitOrigin-RevId: b1fc97dfee85aeba68272f91550ead53a0ed777a
2022-06-01 08:06:55 +00:00
paritosh-08
42da1dbc2e server/docs: follow up on the naming convention
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4563
GitOrigin-RevId: 0ae1f226a63dae34e6cb0d001b4915c05b0974cf
2022-05-27 05:56:52 +00:00
paritosh-08
fd30fb343b server: naming conventions for auto generated fields
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3982
Co-authored-by: Brandon Simmons <210815+jberryman@users.noreply.github.com>
GitOrigin-RevId: f90b2e8f394e7bd69780f003d2d980475f104f42
2022-05-26 11:55:29 +00:00
jkachmar
6720aef26f server: Clean up Cabal project files
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4105
GitOrigin-RevId: 423eecf04061ca8d5ee036a84ee6792036a149e4
2022-04-07 07:16:12 +00:00
Brandon Simmons
fdea752679 server/ci: rework version baking, and cache dist-newstyle in CI
UPDATE: After testing in CI it turns out that the compile time Improvement is better than expected: even though we always have to recompile the OSS lib (due to Version.hs), downstream packages like Pro and multi-tenant can still benefit from some caching and avoid full recompilation.  In the best case this takes us from 22 minutes to 13 minutes total.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4104
GitOrigin-RevId: 76cbfc157064b33856e30f4c2b2ab2366f9c6089
2022-04-05 15:59:09 +00:00
Philip Lykke Carlsen
e1918adb52 Replace "identity column" with "column mutability" data for all backends
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3373
GitOrigin-RevId: bf08cc9008a4b0b3ece4952528c15c45e57fc74c
2022-02-03 14:15:35 +00:00
Evie Ciobanu
ab829b4355 test: have dev.sh pass through CLI options for unit tests
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3420
GitOrigin-RevId: eba7ac0bb609cf83ac1e636c24907601ffcd3651
2022-01-20 15:40:34 +00:00
Auke Booij
a7dbe95666 Hlint all Haskell code
So far we've only used `hlint` to lint the OSS code. This moves some things around to lint the Pro code as well.

Note that the CI action only runs `hlint` on Haskell files that are _changed_ by a PR, relative to its merge-base (usually `main`).

As a reminder, you can use the `ignore-server-hlint-checks` label to prevent this from blocking a merge.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3383
GitOrigin-RevId: d5779c63d780f526a1d58ae4107f0d5262a23ec1
2022-01-18 04:06:15 +00:00
Vamshi Surabhi
2e1be1d2c5 bump python test suite dependencies
This PR upgrades some of the pinned dependencies do not build with python 3.10 - cffi, ruamel, py. Further, it upgrades other packages where the effort is minimal.

For the reviewers: Please review it commit by commit.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3367
GitOrigin-RevId: c5401fe289d3185a79c4d382297f86fbde139825
2022-01-17 07:40:56 +00:00
Evie Ciobanu
db22401e4f server: document python integration test suite
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3350
GitOrigin-RevId: 47dcebbfa84ed3681da40b3ea52e9b849b2fe8e4
2022-01-13 14:29:10 +00:00
Evie Ciobanu
e30d51ee38 server: bump pytest version
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3348
GitOrigin-RevId: b1fd23da3c8af59e28575438ebffe75f6fb79428
2022-01-13 10:30:55 +00:00
Antoine Leblanc
f9b64b7d7c Fix some dev.sh issues
### Description

This PR fixes two small issues with the `dev.sh` script:

- when running the script without a specific test flag (`./scripts/dev.sh test`), we would start some of the containers more than once; this PR fixes this by checking whether the container is already running before starting it
- we were still doing some bigquery setup in the main code, despite that setup being already done in `parse-pytest-backend`, which would result in biquery setup errors even when testing non-bigquery backends; this PR just deletes the redundant line.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3017
GitOrigin-RevId: 86a53b1fe974aae733a02bdf045389a0d7436cb7
2021-11-30 16:37:34 +00:00
Gil Mizrahi
6dae132fce Add instructions for compiling graphql-engine natively on M1
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2049
GitOrigin-RevId: 9f10c26eef5382fb7b8e750ad272a5797b4cd6d8
2021-11-24 09:59:10 +00:00
Abby Sassel
3a6f4e1737 server/bigquery: allow empty tables in replace_metadata. Default BigQueryField mode to Nullable
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2466
GitOrigin-RevId: 1f3599d1317235a31c98d7ed1ece2db92d82e916
2021-10-01 18:30:01 +00:00
Antoine Leblanc
9bae641ac4 Always enable inherited roles tests
### Description

The inherited roles integration tests were behind a flag, and its corresponding fixture, presumably to avoid enabling the option globally. However, #2288 introduced a new test using inherited roles that was not gated behind the flag, which fails when run with `dev.sh`. However, that test works on CI... because inherited roles are globally enabled there.

Consequently, this PR:
- globally enables inherited roles in dev.sh
- removes the flag and the associated fixture

https://github.com/hasura/graphql-engine-mono/pull/2358

Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
GitOrigin-RevId: ebfa6754873324bed15b2cc5e37ec2d8008e8f8d
2021-09-16 06:24:54 +00:00
Abby Sassel
16b09f7d52 server/mssql: support transactions
https://github.com/hasura/graphql-engine-mono/pull/2268

GitOrigin-RevId: b1bc2812cd403688228b3ecf143aa36b3a6af707
2021-09-09 07:59:55 +00:00
Antoine Leblanc
7720496b95 Avoid building the engine twice when only running unit tests.
### Description

When running the tests with `dev.sh`, we always build the engine, before attempting to run the tests. This builds the engine twice, due to the flag change. This PR changes this to only do this first build if we're planning to actually run the integration tests.

https://github.com/hasura/graphql-engine-mono/pull/1964

GitOrigin-RevId: 32e8a6568f3665ad99adaf42421a711241974638
2021-08-26 11:53:06 +00:00
pranshi06
d179a6f2ec server: support EdDSA keys for JWT
https://github.com/hasura/graphql-engine-mono/pull/1818

Co-authored-by: paritosh-08 <85472423+paritosh-08@users.noreply.github.com>
Co-authored-by: Puru Gupta <32328846+purugupta99@users.noreply.github.com>
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
GitOrigin-RevId: aae87d34cd19c97e66721a2bd7602d907aeb90b3
2021-08-12 01:54:06 +00:00
awjchen
046da3caca Combine oss and pro server configuration
https://github.com/hasura/graphql-engine-mono/pull/1983

GitOrigin-RevId: 91069bd5d1f9c5a166eae2317cf4435220b472b0
2021-08-11 04:19:27 +00:00
Abby Sassel
1cd9fc376d server/mysql: integrate MySQL tests into dev.sh workflow
https://github.com/hasura/graphql-engine-mono/pull/1848

GitOrigin-RevId: 2bd7c0a18448f042a1e35d21aaf42232acf48a46
2021-07-21 17:22:56 +00:00
Abby Sassel
76c322589c server/bigquery: Document BigQuery integration tests
This PR documents & streamlines how we run BigQuery integration tests locally as a first step to [running them in CI](https://github.com/hasura/graphql-engine-mono/issues/1525).

I've also created a hasura service account for internal use. [Internal docs here](https://github.com/hasura/graphql-engine-mono/wiki/Testing-BigQuery).

Thanks to FP Complete team for [the guidance here](https://docs.google.com/document/d/1dGDK0touUtsDxRQPonMxSoPbIfzBoSYo02tAjQEO7qA/edit?ts=60c0cf24#), which I've reused parts of.

https://github.com/hasura/graphql-engine-mono/pull/1732

GitOrigin-RevId: 303819d212aa073fbef685d077b1cfa583cd15fc
2021-07-06 11:13:06 +00:00
Abby Sassel
5b7d949ef4 server/citus: run tests in CI
https://github.com/hasura/graphql-engine-mono/pull/1526

GitOrigin-RevId: 4fd4214a799bc0d91cd2d29b1ea444f80714f6bc
2021-07-01 14:41:12 +00:00
Abby Sassel
dc950eda6a server/test: fix BACKEND variable in dev.sh script
GitOrigin-RevId: 0dae37c0aa50c8e4be59618d9d0a777f537fa4dc
2021-06-09 09:51:07 +00:00
kodiakhq[bot]
01d8a37706 server: fix asymptotics of event_log batch fetching
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
GitOrigin-RevId: 9b8afb502e94dd371a8596ccde97d023bd2344a9
2021-06-01 17:34:38 +00:00
Abby Sassel
41908936cf server: improve backend-specific test setup
GitOrigin-RevId: c40393bcdd78feaba7e9490ce4ed78de19b5bfc5
2021-05-25 13:55:11 +00:00
Antoine Leblanc
6e95f761f5 server: rewrite remote input parsers to deal with partial variable expansion (fix hasura/graphql-engine#6656)
GitOrigin-RevId: e0b197a0fd1e259d43e6152b726b350c4d527a4b
2021-05-24 20:13:47 +00:00
Abby Sassel
1afa4ac3cc server/citus: feature branch
Co-authored-by: Vladimir Ciobanu <1017953+vladciobanu@users.noreply.github.com>
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
Co-authored-by: Ikechukwu Eze <22247592+iykekings@users.noreply.github.com>
Co-authored-by: Philip Lykke Carlsen <358550+plcplc@users.noreply.github.com>
Co-authored-by: Phil Freeman <630306+paf31@users.noreply.github.com>
GitOrigin-RevId: 1b964fe5f5f50380172cb702b6a328fed782b6b7
2021-05-21 02:47:51 +00:00
Brandon Simmons
9c9bb43a53 server: restore proper batching behavior in event trigger processing (#1237)
This essentially restores the original code from c425b554b8
(https://github.com/hasura/graphql-engine/pull/4013). Prior to this
commit we would slurp messages as fast as possible from the database
(one thing c425b55 fixed).

Another thing broken as a consequence of the same logic was the
removeEventFromLockedEvents logic which unlocks in-flight events
(breaking at-least-once delivery)

Some archeology, post-c425b55:

- cc8e2ccc erroneously attempted to refactor using `bracket`, resulting
  in the same slurp-all-events behavior (since we don't ever wait for
  processEvent to complete)
- at some point event processing within a batch is made serial, this
  reported as a bug. See: https://github.com/hasura/graphql-engine/issues/5189
- in 0ef52292b5 (which I approved...) an `async` is added, again
  causing the same issue...

GitOrigin-RevId: d8cbaab385267a4c3f1f173e268a385265980fb1
2021-04-29 04:02:05 +00:00
kodiakhq[bot]
a935746e17 Integration test improvements (for speed/clarity), also increase polling interval for scheduled events
This claws back ~7min from integration tests (run serially, as with `dev.sh test --integration`
Further improvements would do well to focus on optimizing metadata operations, as `setup` dominates

GitOrigin-RevId: 76637d6fa953c2404627c4391447a05bf09355fa
2021-04-27 05:35:26 +00:00
Chris Done
53fc5617cf Feature/bigquery python tests
Co-authored-by: Aniket Deshpande <922486+aniketd@users.noreply.github.com>
GitOrigin-RevId: c425f84f1c9b8e38ebbfa509b6fa9298e023f386
2021-04-22 11:32:55 +00:00
Antoine Leblanc
84ed74aba1 server: fix empty remote input objects (fixes #6029, #6703)
GitOrigin-RevId: 3c474ee85b5d1271abfc8848e29ae1d3be28ff63
2021-04-21 01:17:01 +00:00
Antoine Leblanc
78abc5423a fix missing env variables in dev.sh
GitOrigin-RevId: ceeb73d304d2c2026081d5fbc7c68116a2628cb0
2021-04-20 13:43:06 +00:00
Brandon Simmons
a99fd32955 dev.sh: fix hpc after switch to dynamic executable for local dev, in …
GitOrigin-RevId: b31e5c839f3277f99a11208d315e756ddaf46ca6
2021-04-15 13:48:40 +00:00
kodiakhq[bot]
1b378ae6e8 Add a new /dev/rts_stats endpoint, enabled when '+RTS -T'
GitOrigin-RevId: e347bc2f66bad814516662e34bbd8322c214be25
2021-04-13 17:33:38 +00:00