Commit Graph

2139 Commits

Author SHA1 Message Date
Lyndon Maydwell
4d2e37b3e6 Raw Query Support for Data Connectors - GDW-394
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5890
Co-authored-by: Vijay Prasanna <11921040+vijayprasanna13@users.noreply.github.com>
GitOrigin-RevId: f6bd2ed5fe170bcce262564cf4f45c95c9bdff94
2022-09-22 21:09:06 +00:00
Samir Talwar
fef1caabce server/tests-py: Support MSSQL and Citus with run.sh and run-new.sh.
This adds support for running the Python integration tests for MSSQL and Citus just as in CI, as follows:

```
./server/tests-py/run.sh backend-mssql
./server/tests-py/run.sh backend-citus
```

These run the named CI jobs, providing the appropriate backend.

(In reality, all backends are always provided, which is much simpler.)

It also provides the various databases to _server/tests-py/run-new.sh_, though the tests fail as they don't properly initialize the sources. (This will be fixed in the future by provisioning sources in the test framework itself.)

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5997
GitOrigin-RevId: c276a4779a35bb538ef0dc02ac8b7cb2d5a8dec5
2022-09-22 19:03:44 +00:00
Brandon Simmons
d2b3ec8938 ci/benchmarks: fix copypaste bug affecting regression report highlighting in #5942
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6021
GitOrigin-RevId: eb77bf97a52e533c565bfb12a45c0851f832610d
2022-09-22 16:24:15 +00:00
Gil Mizrahi
dd37456949 replace SELECT FROM with SELECT 1 FROM to fix count aggregates for cockroachdb
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5787
GitOrigin-RevId: f5f01a64ccffa36c93a0a88228ac927fbd3aa168
2022-09-22 13:58:03 +00:00
kodiakhq[bot]
81df74852e ci/benchmarks: tweak what we present in regression report
- Make regression report more compact, remove `min` since it doesn't behave like a limit
- It looks like the low-load variants are probably just noisier with fewer samples, so remove them from the regression view:

![samples](https://user-images.githubusercontent.com/210815/191076137-f5c3a0c6-9586-4ea4-a5a7-66000e7a2540.png)

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5942
GitOrigin-RevId: 8a9ab00c251f04d7d5a542731696cb5f86ad2b78
2022-09-22 13:28:56 +00:00
Abby Sassel
37b846c28d server/postgres: aggregation predicate filters should respect aggregation queries permissions
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5936
GitOrigin-RevId: ecb6d677873676ec2bcb7d0bedd1ec707f8e05a1
2022-09-22 12:53:05 +00:00
Abby Sassel
b59008c880 server/postgres: implement execution of aggregation predicates (permissions)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5959
GitOrigin-RevId: eebfe36d217c333b393a308a1db19271809b6a87
2022-09-22 11:15:04 +00:00
Karthikeyan Chinnakonda
3b167b7df1 server: fix bug when streaming subscription cursor has custom name
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6009
GitOrigin-RevId: ba69bbd3a2c8f44e717cb64c8195041edc80a2d6
2022-09-22 09:52:36 +00:00
ananya-2410
c34d2504e8 ci: tag release v2.13.0-beta.1
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6008
GitOrigin-RevId: 3b877e49dcf5ce4116b8b68581a9189521ac81f9
2022-09-22 08:19:04 +00:00
Daniel Harvey
f80ec1d543 server: Rename AltJ to ViaJSON
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5911
GitOrigin-RevId: bd4dec9792f2137b2f4e3169e19109a3d3f63b50
2022-09-21 18:42:30 +00:00
Tom Harding
e71496efa5 Replace Hasura.RQL.Types.Numeric with refined
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5913
GitOrigin-RevId: 96e218229a08dfbc5a598d709be1ee2083d41ec6
2022-09-21 18:03:04 +00:00
paritosh-08
1c7e19c209 server/docs: event trigger log cleanup | remove default for clear_older_than
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5998
GitOrigin-RevId: 1376fd2192c7daaf73e8099cdb2f1aab4b8c3cd1
2022-09-21 17:29:55 +00:00
Samir Talwar
8b8d7ab96d server/tests-hspec: Simplifying the steps required to run these on macOS on aarch64.
This makes a few changes to the test scripts and makefiles in order to make things simpler for the average Apple user.

First of all, we change the `wait_for_mysql` function to use "localhost", not "127.0.0.1", as this fixed an issue on my system when attempting to connect to the MySQL server.

Secondly, we split the SQL Server test image into two:

* The first is the server itself, which now automatically uses `azure-sql-edge` as the image if you are on an aarch64 chip and using the `make` commands.
* The second is the initialization script. Because `sqlcmd` is not available in the `azure-sql-edge` image on aarch64, we use a separate container based on `mssql-tools` to initialize the server.

The README has been updated.

Tested on both macOS/aarch64 (with other changes) and Linux/x86_64.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5986
GitOrigin-RevId: b16e079861dcbcc66773295c47d715e443b67eea
2022-09-21 16:48:21 +00:00
Daniel Harvey
231de5166a server: bump CockroachDB image and un-skip fixed tests
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5967
GitOrigin-RevId: 28a2f8e94d091c8ddd272b6abf3828c40662ef38
2022-09-21 14:11:06 +00:00
Daniel Harvey
e2ced4011d server: import local Postgres modules as Postgres
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5962
GitOrigin-RevId: 862862c34b6c633c94ee8ae1f075afca2799fd2b
2022-09-21 11:35:50 +00:00
Daniel Chambers
3365a18516 Rename DataConnector cross_table capability to subquery
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5980
GitOrigin-RevId: c289b2a7afb4d07214aac2b4d89c9b87a75e98aa
2022-09-21 10:48:05 +00:00
paritosh-08
876c906660 server: event trigger log cleanup | rename start_event_trigger_cleanups API
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5976
GitOrigin-RevId: 752c905d31b3700aa49d816ec573df74715f6654
2022-09-21 06:00:39 +00:00
Daniel Chambers
dc9a86680c Gardening: Clean up Data Connector API types
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5977
GitOrigin-RevId: 1a6898d6416fff265a8add74d414c979f7fa3bc5
2022-09-21 05:13:03 +00:00
Brandon Simmons
fc9cc14943 ci/benchmarks: don't measure decompression in k6 (NOT REAL PERFORMANCE IMPROVEMENT)
See: https://github.com/grafana/k6/issues/2685

It might be interesting to think about taking into consideration decompression time when thinking about performance, but In general I think doing so is surprising and I wasted a lot of time trying to figure out why my optimizations to the compression codepath weren't improving things  to the degree I expected

The downside here is we lose error reporting, so you'll need to only set
discardResponseBodies: true after the query has been tested.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5940
GitOrigin-RevId: 82a589a59b93f10ffb5391e4a3190459fb6e613b
2022-09-21 02:07:54 +00:00
Auke Booij
4c8ea8e865 Import pg-client-hs as PG
Result of executing the following commands:
```shell
# replace "as Q" imports with "as PG" (in retrospect this didn't need a regex)
git grep -lE 'as Q($|[^a-zA-Z])' -- '*.hs' | xargs sed -i -E 's/as Q($|[^a-zA-Z])/as PG\1/'
# replace " Q." with " PG."
git grep -lE ' Q\.' -- '*.hs' | xargs sed -i 's/ Q\./ PG./g'
# replace "(Q." with "(PG."
git grep -lE '\(Q\.' -- '*.hs' | xargs sed -i 's/(Q\./(PG./g'
# ditto, but for [, |, { and !
git grep -lE '\[Q\.' -- '*.hs' | xargs sed -i 's/\[Q\./\[PG./g'
git grep -l '|Q\.' -- '*.hs' | xargs sed -i 's/|Q\./|PG./g'
git grep -l '{Q\.' -- '*.hs' | xargs sed -i 's/{Q\./{PG./g'
git grep -l '!Q\.' -- '*.hs' | xargs sed -i 's/!Q\./!PG./g'
```
(Doing the `grep -l` before the `sed`, instead of `sed` on the entire codebase, reduces the number of `mtime` updates, and so reduces how many times a file gets recompiled while checking intermediate results.)

Finally, I manually removed a broken and unused `Arbitrary` instance in `Hasura.RQL.Network`. (It used an `import Test.QuickCheck.Arbitrary as Q` statement, which was erroneously caught by the first find-replace command.)

After this PR, `Q` is no longer used as an import qualifier. That was not the goal of this PR, but perhaps it's a useful fact for future efforts.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5933
GitOrigin-RevId: 8c84c59d57789111d40f5d3322c5a885dcfbf40e
2022-09-20 19:55:51 +00:00
paritosh-08
f7115891f7 server: event trigger log cleanup | change metedata API field name
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5965
GitOrigin-RevId: aa50006946e79fe4d9c971568221dc436040b738
2022-09-20 16:33:11 +00:00
Tom Harding
8dcc8f8fca Implement aggregation tests for hspec
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5960
GitOrigin-RevId: bac9bae3ea39c005f0e7b9c5f7864f012d395ebb
2022-09-20 15:23:46 +00:00
paritosh-08
607b26362d server: event trigger cleanup | fix query_timeout field name
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5953
GitOrigin-RevId: b80b6672db5a73cafca77ba06cf25f71c7dc8c10
2022-09-20 12:47:22 +00:00
Samir Talwar
0b3c8ccaca server/tests-py: Support running BigQuery tests with run.sh.
This fixes a few issues so that we can run `./server/tests-py/run.sh backend-bigquery` to run the Python integration tests for BigQuery locally.

* We forward the relevant environment variables to the Docker container.
* We increase the HTTP timeout, as I'm seeing requests taking up to 90s locally.
* We rewrite the setup so that it avoids `INSERT INTO`, which is not available using the BigQuery free tier. Instead, we use `CREATE TABLE ... AS SELECT ...`. This is the same method used by the Haskell integration tests.

We also capture local server output in a volume so it's easier to figure out what went wrong later.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5921
GitOrigin-RevId: c628f8c08a84f2582958659ab6d6494832471f6f
2022-09-20 10:56:36 +00:00
Abby Sassel
89f9a5fdf2 server/tests: add permissionRows field and Permission constructor
Factoring out some minor cleanup from a PR which [tests aggregate filters and permissions](https://github.com/hasura/graphql-engine-mono/pull/5936). This PR adds a `permissionRows` field to the `Permission`s type, needed to test row permissions (https://github.com/hasura/graphql-engine-mono/pull/5936) and some default constructors.

Relates to https://github.com/hasura/graphql-engine-mono/issues/5888

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5949
GitOrigin-RevId: 89679a49496364473976cc73afa4d064a02bbf21
2022-09-20 09:17:32 +00:00
Daniel Chambers
5c06eb7a3e Remove duplicated Data Connector API types
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5930
GitOrigin-RevId: 8c11387fa8556c3cdf8c92a0924ae53d31b953a5
2022-09-20 06:20:25 +00:00
Daniel Chambers
04ae6abf78 Exists support in filter expressions for Data Connector queries [GDW-133]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5904
GitOrigin-RevId: 6bad4c29a7d14d3881f9c57fe983d14cc41bdc4b
2022-09-20 04:01:33 +00:00
awjchen
dcca0c6275 server: support 128-bit trace ids
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5905
GitOrigin-RevId: 48a981a432b2e979bc7b42b32f9c7f53ab9e2598
2022-09-20 02:50:06 +00:00
Samir Talwar
6c03eef990 server: Add test cases for inconsistencies upon reloading or replacing the metadata.
I am working on https://github.com/hasura/graphql-engine/issues/8807, and wanted to write a Haskell integration test case to reproduce it.

We have Python integration tests somewhat covering this behavior in *test_inconsistent_meta.py*, but no Haskell tests, so I thought I'd shore up the coverage here by adding a few test cases for working behavior.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5897
GitOrigin-RevId: 21500e530e413feaede5cbd8b4a94b07d25a6260
2022-09-19 18:49:31 +00:00
hasura-bot
71bba3e3aa Add X-Forwarded-Origin header
GITHUB_PR_NUMBER: 8728
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/8728

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5246
Co-authored-by: Wsevolod <31807671+kraftwerk28@users.noreply.github.com>
GitOrigin-RevId: b8229786a2b14a8be49303f9a6da10231c8fb331
2022-09-19 14:55:23 +00:00
Daniel Harvey
559cc78b2d server: optimise CockroachDB metadata fetch
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5910
GitOrigin-RevId: 09deef75576fc916138888819db6689b4b61da2b
2022-09-19 14:07:59 +00:00
Gil Mizrahi
a3036be77b tests-hspec readme: update m1 workaround fix
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5931
GitOrigin-RevId: 4aeefb430e8aa7c3a257b0cce440842c3ed156df
2022-09-19 09:54:38 +00:00
Abby Sassel
71597778c9 server/postgres: fix non-latin text elements in array literals (copy). close hasura/graphql-engine#8961
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5916
Co-authored-by: Gil Mizrahi <8547573+soupi@users.noreply.github.com>
GitOrigin-RevId: 9063f7ff39f684c4ab43f093706b872021b67b13
2022-09-18 16:08:15 +00:00
Philip Lykke Carlsen
26cdc03aa4 fix(#8904): Make BigQuery Computed Fields arguments optional
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5909
GitOrigin-RevId: 0d7c7de273c27bbd2338f42253754be0a06ea66e
2022-09-17 22:41:19 +00:00
Solomon
85cfd91908 allow exporting dc-agent test descriptions
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5925
GitOrigin-RevId: a508e942c61c775823dff440a37fd8769ca42e21
2022-09-17 02:20:31 +00:00
awjchen
c3840fe2d1 server: graphql-ws: stop sending pings in response to pongs
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5882
GitOrigin-RevId: 07fad6a24bb6eed9807e785e8201517126f7c302
2022-09-17 00:01:53 +00:00
Abby Sassel
3056678e04 server/postgres: Implement execution of aggregation predicates
PR for the translation / execution step of [aggregation predicate filters](https://github.com/hasura/graphql-engine-mono/issues/5174).

[`translateAVAggregationPredicates`](translateAVAggregationPredicates) is the main change of note, everything else is a supporting or helper function. Please note this doesn't yet include [tests relating to permissions](https://hasurahq.slack.com/archives/C01RZPEPF0W/p1662560092197769); I decided to raise a PR without them for slightly faster feedback. I may include them in this PR or a separately if it's not trivial.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5724
GitOrigin-RevId: fcac258b64066e2bd45108372165a16fd957f5ab
2022-09-16 15:01:03 +00:00
Daniel Harvey
4bb4c91c0c tooling: add coreutils to macos install docs
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5914
GitOrigin-RevId: 4a7383b7537987a5ab5f744ab49713c0f532a1b4
2022-09-16 14:16:21 +00:00
Karthikeyan Chinnakonda
cb2349d5a0 server, pro server, console: provide an option to not include rows count while fetching scheduled events
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5650
Co-authored-by: Varun Choudhary <68095256+Varun-Choudhary@users.noreply.github.com>
Co-authored-by: Daniele Cammareri <5709409+dancamma@users.noreply.github.com>
GitOrigin-RevId: 8a55f3ca52cfbda4d4e75372b813b7acd421eabb
2022-09-15 19:12:04 +00:00
Stefano Magni
a0f4f00bfd console: Add Sentry
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5699
Co-authored-by: Rishichandra Wawhal <27274869+wawhal@users.noreply.github.com>
Co-authored-by: Daniel Harvey <4729125+danieljharvey@users.noreply.github.com>
GitOrigin-RevId: 00f2c5d25012b21f4e8763ef578598a3a11896e4
2022-09-15 17:00:44 +00:00
Samir Talwar
a24cdd9b49 Prepare PostgreSQL for running tests in parallel.
This makes two changes to the Docker Compose files that we use for local testing:

1. We disable `fsync`. On my machine, this decreases the time taken to create a new database from ~5s to less than 0.1s. The trade-off is that you might lose data, which we don't care about, as this is for testing.
2. We increase the maximum number of connections from the default, 100, to 1000. This allows us to run more tests in parallel without hitting connection limits.

These changes won't have any meaningful effect for now; they simply allow us to parallelize tests against PostgreSQL in the future.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5892
GitOrigin-RevId: 5d0d0ab37fdfbf4c9e20084d3cbedf647f54a04e
2022-09-15 14:03:23 +00:00
Samir Talwar
1a5aaae9cf server/tests-py: Add a --hge-bin argument to the Pytest runner.
This argument allows the user to specify how to run HGE, rather than starting it beforehand. The runner will start a new instance of HGE for each test class.

This does not provide isolation, as the database is still re-used, but it helps us get closer.

You can try it yourself by executing:

```
$ cabal build graphql-engine:exe:graphql-engine
$ ./server/tests-py/run-new.sh
```

This doesn't affect CI at all.

I also fixed a few warnings flagged by Pylance.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5881
GitOrigin-RevId: ea6f0fd631a2c278b2c6b50e9dbdd9d804ebc9d4
2022-09-15 12:31:12 +00:00
paritosh-08
ab71adc3a0 server, pro: event trigger auto cleanup (increment 3)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5823
Co-authored-by: pranshi06 <85474619+pranshi06@users.noreply.github.com>
Co-authored-by: Puru Gupta <32328846+purugupta99@users.noreply.github.com>
Co-authored-by: Karthikeyan Chinnakonda <15602904+codingkarthik@users.noreply.github.com>
GitOrigin-RevId: 3d4af905d5df4c05107ad46ef29f4c0a567ff754
2022-09-15 11:46:22 +00:00
Lyndon Maydwell
ba0a685211 CI Support for Data Connector Agent Tests - GDW-336
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5808
GitOrigin-RevId: 817575185cdbd21985c647f22d5b643ce0852ef6
2022-09-15 09:47:24 +00:00
Anon Ray
1d9844ee4e server: use metadata DB ID as ProjectID for EE lite projects
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5874
GitOrigin-RevId: 63a4016227eeeceb0143cf84ed4c13b14f424d4b
2022-09-15 09:13:56 +00:00
Lyndon Maydwell
118b03e329 RMGetSourceKindCapabilities doesn't modify metadata
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5883
GitOrigin-RevId: dbf69fd3f6086eca0e9b2ed9b47d19ae56339e63
2022-09-15 07:35:09 +00:00
Samir Talwar
bff1309344 server/tests-py: Make HGECtxGQLServer a fixture.
Starting it and stopping it for the various tests that actually use it.

There are only a few.

This also removes some dead code and fixes warnings in _test_webhook_request_context.py_.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5846
GitOrigin-RevId: 7760467f9de7b1f9718e7482275c298eeaa3ad3a
2022-09-14 21:42:40 +00:00
kodiakhq[bot]
f2931a4d32 server: import pg-client-hs with history
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5735
Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
Co-authored-by: Alexis King <759911+lexi-lambda@users.noreply.github.com>
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Auke Booij <164426+abooij@users.noreply.github.com>
Co-authored-by: Brandon Simmons <210815+jberryman@users.noreply.github.com>
Co-authored-by: Lyndon Maydwell <92299+sordina@users.noreply.github.com>
Co-authored-by: Anon Ray <616387+ecthiender@users.noreply.github.com>
Co-authored-by: Evie Ciobanu <1017953+eviefp@users.noreply.github.com>
Co-authored-by: Swann Moreau <62569634+evertedsphere@users.noreply.github.com>
Co-authored-by: jkachmar <8461423+jkachmar@users.noreply.github.com>
Co-authored-by: Robert <132113+robx@users.noreply.github.com>
Co-authored-by: awjchen <13142944+awjchen@users.noreply.github.com>
Co-authored-by: Karthikeyan Chinnakonda <15602904+codingkarthik@users.noreply.github.com>
Co-authored-by: Tom Harding <6302310+i-am-tom@users.noreply.github.com>
GitOrigin-RevId: 6a3940b2596fc178379b85d5fa79bd9ac83457e2
2022-09-14 14:51:34 +00:00
David Overton
cbddcce26b Data connector cache invalidations
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5820
GitOrigin-RevId: ded80e378901cc0c0ca12eb67d9264756a8d6ff4
2022-09-14 13:00:47 +00:00
Solomon
4a0446852c Adds configSchema to get_source_kind_capabilities
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5860
GitOrigin-RevId: 9bc13af5104251c509194d28ece81066643ab2a2
2022-09-14 07:07:04 +00:00