1
0
mirror of https://github.com/hasura/graphql-engine.git synced 2024-12-15 17:31:56 +03:00
Commit Graph

389 Commits

Author SHA1 Message Date
Samir Talwar
1e1a36a192 server/tests-py: Use environment variables for services in queries.
I'm trying to shore up the Python integration tests to make them more reliable. In doing so, I noticed this.

---

Rather than hard-coding hostnames and ports, we can (and already do) inject these into the HGE process using environment variables.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5255
GitOrigin-RevId: 6bb593999ece42cedef6619f31f9d9b2e39f30ef
2022-08-03 20:05:46 +00:00
Daniel Harvey
0c9a97040e tooling: Add make ghcid-test-x commands
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5274
GitOrigin-RevId: e9096812b1cd97f45fd89bf7deeca4d45fd9d422
2022-08-03 11:07:05 +00:00
Samir Talwar
34ec4d0278 Restore scripts/dev.sh.
Not sure how that snuck in. Apologies.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5307
GitOrigin-RevId: ffcccbddb86cd002fb14964fc6eb0120b29e1c37
2022-08-03 10:27:04 +00:00
Samir Talwar
349ccd3296 Use docker compose, not docker-compose.
Docker Compose is now a plugin for Docker, bundled by default in Docker Desktop and many Linux distribution packages. The standalone `docker-compose` binary has been deprecated since Docker Compose v2.

Using the new version directly allows us to write development scripts that do not require `docker-compose` to be installed.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5185
GitOrigin-RevId: c8542b8b2405d1aa32288991688c6fde4af96383
2022-08-02 10:30:57 +00:00
Samir Talwar
07328fd9fc server: Automate generation and verification of Cabal files from hpack.
We currently use `hpack` to generate the Cabal files from _package.yaml_
files for the two small libraries in _server/lib_. While this is more
convenient, we also check the Cabal files into the repository to avoid
needing an extra step upon pulling changes.

In order to ensure that the Cabal files do not get out of sync with the
hpack files, this introduces a few improvements:

1.  Makefile targets to automatically generate the Cabal files without
    needing to know the correct incantation. These targets are a
    dependency of all build targets, so you can simply run
    `make build-all` and it will work.
2.  An extra comment at the top of all generated Cabal files that
    explains how to regenerate it.
3.  A `lint-hpack` Makefile target that verifies that the Cabal files
    are up-to-date.
4.  A CI job that runs `make lint-hpack`, to stop inconsistencies
    getting merged into trunk.

Most of these changes are ported from .

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5217
GitOrigin-RevId: d3dfbe19ec00528368d357b6d0215a7ba4062f68
2022-07-29 16:22:12 +00:00
Samir Talwar
4f85976155 server: Get make test-backends working and somewhat speedy.
* Fix `make start-sqlserver` so it can find `sqlcmd`.
* Start the DC reference agent when testing all backends.
* Start backends in parallel so it doesn't take so long to spin them up. This is achieved by splitting all `start-*` tasks into `spawn-*` and `wait-for-*`.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5176
GitOrigin-RevId: ea483a755daa3fa6a8701f9d3dd0f393bc56510f
2022-07-27 13:30:14 +00:00
Samir Talwar
a471c5080b Nix: Upgrade Python to the latest version.
There's no reason to pin it to v3.7. This wasn't causing the Docker
Compose issues; it's a dependency version issue.

We actually don't need to install `docker-compose` at all. It's now a
Docker plugin (and is bundled in Docker Desktop) since Docker Compose
v2. You can run it with `docker compose`.

This also fixes a bug in _dev.sh_ due to a moved `make` target.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5197
GitOrigin-RevId: 638b2eb39c99e7a990e7a6c66388831438aed45f
2022-07-26 16:47:18 +00:00
Samir Talwar
adca95a78d server: Use make to simplify tests-py venv generation.
Rather than a homebrewed approach, we can use `make` to figure out when
it's necessary to regenerate our venv.

This Makefile will regenerate _requirements.txt_ from
_requirements-top-level.txt_ when the latter is changed.

It will also regenerate the venv when _requirements.txt_ is changed
(i.e. changes are pulled, or it's regenerated as described above).

`make` uses file/directory timestamps to figure out what to rebuild.
This is probably more reliable than expecting people to update a version
number whenever they change a file.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5152
GitOrigin-RevId: 24b27d49bf6c4ba1d57ac38ea38ae278216c6d66
2022-07-25 20:05:49 +00:00
Samir Talwar
e1c7767d5e Run both integration test suites against PostgresSQL v14 with Postgis.
Let's be consistent about this.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5177
GitOrigin-RevId: 75a48452984a0762e52ee251a98f086f2ae1b150
2022-07-25 16:37:54 +00:00
paritosh-08
95adde4ce2 server: support for Apollo federation
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4584
Co-authored-by: Auke Booij <164426+abooij@users.noreply.github.com>
GitOrigin-RevId: 0f60c263efb5fbaa25620dd8159e8cfda25a61b2
2022-07-25 15:54:41 +00:00
Samir Talwar
0e4d709285 Factor tool versions out into server/VERSIONS.json.
This helps us use the same versions locally as in CI. If you're using the Nix setup, it guarantees it.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5111
GitOrigin-RevId: 6e00cd7a78593df1e60fac37cc1195aba60e488f
2022-07-22 12:23:46 +00:00
Samir Talwar
778b1bae2b Fun improvements to Makefiles, with the intent of re-using test-infrastructure.mk.
I intend to use the Make targets around starting and stopping backend databases in order to wrap the Python integration tests in targets.

To that end, I have:

1. Factored out test infrastructure targets into _test-infrastructure.mk_.
2. Actually run the functions from _util.sh_; previously we were just running a bash script with no explicit commands. _util.sh_ must be `source`d and then you invoke the relevant function.
3. Run `docker-compose` through Nix if possible.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5136
GitOrigin-RevId: d7a01958045a82199c969ec3f88387d7d76d254b
2022-07-22 10:57:23 +00:00
Samir Talwar
89eafb1589 server: Re-freeze tests-py/requirements.txt.
I tried re-freezing _server/tests-py/requirements-top-level.txt_
recently, and discovered that it caused the tests to fail.

This pins a couple of dependencies so that we can safely re-freeze.
Specifically:

- `cryptography` is pinned at v3.*
- `graphene` is pinned at v2.*
- `PyJWT` is pinned at v2.3.*
- `websocket-client` is pinned at v0.56.0 (this was done in
  _requirements.txt_ already, but that file is supposed to be
  regenerated)

Upgrading `SQLAlchemy` required changing PostgreSQL URLs to use
"postgresql://" as the URL scheme, not "postgres://".

Updating `ruamel.yaml` caused a few tests to fail as we are passing
`ruamel.yaml.scalarstring.LiteralScalarString` values as header values.
This is fixed by explicitly converting header values to strings.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5120
GitOrigin-RevId: 9c12a3013c3d1f23dddbe781037663838b23f6f5
2022-07-22 10:25:33 +00:00
Daniel Harvey
8acb952f0b tests: Remove tests-hspec.tix before running tests
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5109
GitOrigin-RevId: fc5721f2b577e3892ea3dada3842338cc19704bb
2022-07-20 11:21:37 +00:00
Daniel Harvey
1375a77fd7 tooling: add make ghcid commands
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5097
GitOrigin-RevId: 2463c61afbffedaaf21bdd3a014f79f1cc4b930e
2022-07-19 12:20:49 +00:00
Evie Ciobanu
d9d193d0e6 Evie/nix/fix nix shell
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5089
GitOrigin-RevId: 584a42ea6d2af08b98d592ed9c31bb1d2df540ee
2022-07-18 21:39:48 +00:00
Daniel Harvey
37fb40ddec tooling: Add make build... commands
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4824
GitOrigin-RevId: eeee7c47fb8b8160d85aee2d9d07fdf9ed14a703
2022-07-04 11:45:21 +00:00
Lyndon Maydwell
3c33996e7d Data-Connectors SDK
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4332
GitOrigin-RevId: aa93b65e266d73c3b7b80fafd4e36da74428add4
2022-06-27 09:32:12 +00:00
Samir Talwar
75c01223a8 server: Use plain assertions in pytest.
When pytest rewrites assertions to make them more useful, it also
truncates long assertion messages in the middle, often obscuring the
actual error.

Disabling this allows us to see the full message, which should hopefully
allow us to see the actual error.

In order to support this, we need to make sure that custom assertion
messages include the actual assertion information, as this will no
longer be rendered by pytest.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4835
GitOrigin-RevId: de6839a3b40d0edc7cb96e46625eebca0aaf0c95
2022-06-25 00:04:16 +00:00
Samir Talwar
1f2f9a8f94 server: Fix up the instructions on testing against BigQuery.
This improves the experience of testing against BigQuery in a few ways:

1.  There is an explicit instruction to create a dataset, which was not
    documented.
2.  The verification script uses the environment variables, just like the
    tests.
3.  The verification script passes the correct content type headers.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4706
GitOrigin-RevId: 15b5bd28b1762c130c0ec09d6babe8c08e05ab15
2022-06-15 09:45:19 +00:00
Brandon Simmons
bf3657d0a2 Fix up for scripts/cabal-freeze-update.sh
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4685
GitOrigin-RevId: abae6aed0ae56a67d4c6bbb59598920bfcdc5e8e
2022-06-09 19:52:38 +00:00
Brandon Simmons
45af456fb3 dev: add new freeze file helper script: scripts/cabal-freeze-update.sh
This was reworked from an earlier version based on feedback from @solomon-b : https://github.com/hasura/graphql-engine-mono/pull/4618

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4646
GitOrigin-RevId: c54469a2d8466471e6d5c25a0cc80e26d94ce7c7
2022-06-08 21:14:10 +00:00
Brandon Simmons
6e8da71ece server: migrate to aeson-2 in preparation for ghc 9.2 upgrade
(Work here originally done by awjchen, rebased and fixed up for merge by
jberryman)

This is part of a merge train towards GHC 9.2 compatibility. The main
issue is the use of the new abstract `KeyMap` in 2.0. See:
https://hackage.haskell.org/package/aeson-2.0.3.0/changelog

Alex's original work is here:


BEHAVIOR CHANGE NOTE: This change causes a different arbitrary ordering
of serialized Json, for example during metadata export. CLI users care
about this in particular, and so we need to call it out as a _behavior
change_ as we did in v2.5.0. The good news though is that after this
change ordering should be more stable (alphabetical key order).

See: https://hasurahq.slack.com/archives/C01M20G1YRW/p1654012632634389

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4611
Co-authored-by: awjchen <13142944+awjchen@users.noreply.github.com>
GitOrigin-RevId: 700265162c782739b2bb88300ee3cda3819b2e87
2022-06-08 15:32:27 +00:00
Puru Gupta
382e99f3cc Add ACTION_WEBHOOK_HANDLER env variable to dev.sh
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4602
GitOrigin-RevId: b1fc97dfee85aeba68272f91550ead53a0ed777a
2022-06-01 08:06:55 +00:00
paritosh-08
42da1dbc2e server/docs: follow up on the naming convention
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4563
GitOrigin-RevId: 0ae1f226a63dae34e6cb0d001b4915c05b0974cf
2022-05-27 05:56:52 +00:00
paritosh-08
fd30fb343b server: naming conventions for auto generated fields
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3982
Co-authored-by: Brandon Simmons <210815+jberryman@users.noreply.github.com>
GitOrigin-RevId: f90b2e8f394e7bd69780f003d2d980475f104f42
2022-05-26 11:55:29 +00:00
Vishnu Bharathi
78b94ce765 ci: avoid build failure when short commit SHA contains only numbers
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4538
GitOrigin-RevId: 63700682a1b3b6fd68c6be22cde210f5e1e8f559
2022-05-23 20:10:51 +00:00
Brandon Simmons
f8fd61539e ci: finer-grained dependencies on build-servers artifacts
…p CI ()

Trying out a new convention here for jobs that generate multiple
artifacts at different points in time, and where downstream jobs depend
on just a particular subset of artifacts and don't need to wait for the
entire job. We accomplish this with separate `Input` steps, which we
unblock manually via calls to the buildkite API once particular sets of
artifacts are complete.

We could accomplish the same thing by having multiple build server jobs
focused on particular targets ( this is in fact how it used to work)
but that is quite wasteful of CI resources;  in order to build any of
the binaries quickly, we need significant number of cores;  but if we
only build particular targets, we don't take advantage I've all those
cores relative to the parallelism a full cabal build gives us. That
would likely be more difficult to maintain as well.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4256
Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
GitOrigin-RevId: 5f5603da8a85f9a491877d51a7c08764c7199602
2022-05-06 12:59:53 +00:00
Divi
08b878bc9f ci: use shorter version string to allow long JIRA branch names
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4183
GitOrigin-RevId: 8ceab85eca6122fcc51ab9351ee4818b4d8ae4c1
2022-04-14 09:01:17 +00:00
jkachmar
6720aef26f server: Clean up Cabal project files
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4105
GitOrigin-RevId: 423eecf04061ca8d5ee036a84ee6792036a149e4
2022-04-07 07:16:12 +00:00
Brandon Simmons
fdea752679 server/ci: rework version baking, and cache dist-newstyle in CI
UPDATE: After testing in CI it turns out that the compile time Improvement is better than expected: even though we always have to recompile the OSS lib (due to Version.hs), downstream packages like Pro and multi-tenant can still benefit from some caching and avoid full recompilation.  In the best case this takes us from 22 minutes to 13 minutes total.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4104
GitOrigin-RevId: 76cbfc157064b33856e30f4c2b2ab2366f9c6089
2022-04-05 15:59:09 +00:00
Divi
0baef156df ci: create a server test mode for PR builds
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3812
Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
GitOrigin-RevId: 5ea09770984c20af5e443f7be127ca89df513456
2022-03-31 05:48:54 +00:00
Vishnu Bharathi
fac5e0b2c3 ci: produce ubuntu and centos flavoured graphql-engine images
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3597
GitOrigin-RevId: f1fded72524f28f41433eea6819276932ee0b5cb
2022-03-22 12:03:25 +00:00
Rakesh Emmadi
d7b6585454 server/gardening: move local hoogle setup from Makefile to bash script
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3996
GitOrigin-RevId: ace7f4c13cab6e549aedf4520334c435acac9ab1
2022-03-16 10:44:01 +00:00
Rakesh Emmadi
b844c5d732 server/tests: unify HASURA_BIGQUERY_SERVICE_KEY and HASURA_BIGQUERY_SERVICE_ACCOUNT env vars
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3914
GitOrigin-RevId: 66f75420504d1b864b91599c2bdaa832784bb956
2022-03-14 07:50:35 +00:00
Divi
50d9c112e7 ci: fix issues faced during release
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3767
GitOrigin-RevId: 67c6536826f451161b8dda8efcf1e7d37469cc42
2022-02-23 06:27:48 +00:00
Abby Sassel
fc0352a995 server/tests: fix BigQuery test failure
This PR addresses a couple of recent issues with BigQuery tests in CI, plus some refactoring.

TL;DR: most issues were fixed by being [explicit about failures](http://redsymbol.net/articles/unofficial-bash-strict-mode/) and variables in scope. Thanks to @qrilka for addressing a couple of these in your last PR already, sharing my notes about them here too for posterity as I found the troubleshooting exercise useful.

_commands listed in execution order_

`generate_test_project`
* issue: this rarely fails AFAICT, but a failure in any of the `gcloud` commands will result in unclear failures further along the call stack
* fix: fail faster with `set -e`

`create_bigquery_dataset`
* error: `BigQuery error in mk operation: Not found: Project`. examples [[1]](https://github.com/hasura/graphql-engine-mono/issues/3600)[[2]](https://buildkite.com/hasura/graphql-engine-mono/builds/4857#f70990eb-3721-45c6-b11a-af7e4ebd9fe3)
* issue: a failure on a missing project, which I could verify was created successfully
* cause: missing `HASURA_BIGQUERY_PROJECT_ID` , most likely caused by the above failure and dodgy subshell scoping
* fix: pass `project_id` as an argument. This might not have been necessary since `generate_test_project` fails faster, but I thought it was clearer to pass explicitly.

`ensure_bigquery_db`
* issue: this could fail, but return a 0 exit code. [example](https://buildkite.com/hasura/graphql-engine-mono/builds/4836#e3dfbc1e-4034-40bb-beb1-181fb5d9489f)
* fix: add a `--fail` flag to `curl` call. Kirill addressed in https://github.com/hasura/graphql-engine-mono/pull/3435/files#diff-0525000dbb36f436dcc17570541378de51471200d294b468c4b288e0292441b6R94

`delete_temp_bigquery_project`
* error: `(gcloud.projects.delete) value for field [projectId] in collection [cloudresourcemanager.projects] is required but was not provided`. [example](https://buildkite.com/hasura/graphql-engine-mono/builds/4836#e3dfbc1e-4034-40bb-beb1-181fb5d9489f)
* issue: attempting to delete a tmp file that didn't exist due to earlier failures
* fix: Kirill addressed in https://github.com/hasura/graphql-engine-mono/pull/3435/files#diff-0525000dbb36f436dcc17570541378de51471200d294b468c4b288e0292441b6R106

to verify: check [bigquery](https://buildkite.com/hasura/graphql-engine-mono/builds/5115#41d059eb-329c-46fb-a745-b2b97fffd328) and [hspec](https://buildkite.com/hasura/graphql-engine-mono/builds/5115#5623a53d-c18d-478e-bf44-446e1287453b) jobs in CI

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3616
Co-authored-by: Divi <32202683+imperfect-fourth@users.noreply.github.com>
GitOrigin-RevId: b85420ef57036b4dbb05202b73ee9e954113bf9d
2022-02-21 11:45:03 +00:00
Divi
9e56d69599 ci: fix bugs faced during v2.2.0 release
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3659
Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
GitOrigin-RevId: 8ddc98a2c06f64ae86f3a011f79f93e7c15161a0
2022-02-16 08:21:35 +00:00
Kirill Zaborsky
960cf2d954 Use 'hasura' as dataset in python test to unify it with hspec
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3636
GitOrigin-RevId: f4114fded09cbdd9875488610c635537478e4ab5
2022-02-14 06:23:07 +00:00
Kirill Zaborsky
773870f443 BigQuery hspec tests
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3435
Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
GitOrigin-RevId: 78f698becb83bd9732eddbe419361aff2799dd2c
2022-02-09 15:30:13 +00:00
Philip Lykke Carlsen
e1918adb52 Replace "identity column" with "column mutability" data for all backends
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3373
GitOrigin-RevId: bf08cc9008a4b0b3ece4952528c15c45e57fc74c
2022-02-03 14:15:35 +00:00
Antoine Leblanc
bd2fce3691 Stable sort of extracted notes
## Description

This PR updates the notes-extracting script, to allow for a stable sort across runs:
- files are now treated in alphabetical order of the full file name:
  - within a file, notes are treated in order
  - that order is reflected in the generated index file
- within a note, references are sorted by file, then by line number

Additionally it fixes(?) a note whose format was unexpected.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3352
GitOrigin-RevId: 2b2b57ec0aa2657d75a88e4951e6b19bb2d665e3
2022-02-03 13:33:09 +00:00
Antoine Leblanc
3525e60021 Deploy server documentation to github page in CI
_(This PR is on top of #3352.)_

## Description

This PR overhauls our documentation CI steps to push all generated server documentation to the `gh-pages` branch of the OSS repo. The goal of this PR is to arrive in the situation where `https://hasura.github.io/graphql-engine/server/` is automatically populated to contain the following:
  - all the markdown files from `server/documentation`, copied verbatim, no transformation applied
  - all the notes, collected from the code by the `extract-notes.sh` script, in `server/notes`
  - the generated haddock documentation for each major release or branch in `server/haddock`.

To do so, this PR does the following:
  - it includes the script to extract notes from ,
  - it rewrites the documentation checking CI step, to generate the notes and publish the resulting "server/documentation" folder,
  - it includes a new CI step to deploy the documentation to the `gh-pages` branch

Of note:
  - we will generate a different haddock folder for each main branch and release; in practice, that means the _main_, _stable_, _alpha_, _beta_ branches, and every build tagged with a version number
  - the step that builds the haddock documentation checks that ALL projects in the repo build, including pro, but the deploy only deploys the graphql-engine documentation, as it pushes it to a publicly-accessible place

## Required work

**DO NOT MERGE THIS PR IT IS NOT READY**. Some work needs to go into this PR before it is ready.

First of all: the `gh-pages` branch of the OSS repo does NOT yet contain the documentation scaffolding that this new process assumes. At the bare minimum, it should be a orphan branch that contains a top-level README.md file, and a _server_ folder. An example of the bare minimum required can be previewed [on my fork](https://nicuveo.github.io/graphql-engine/server/).

The content of the `server/documentation` folder needs to be adjusted to reflect this; at the very least, a `README.md` file needs to be added to do the indexing (again, see the placeholder [on my fork](https://nicuveo.github.io/graphql-engine/server/) for an example).

This way of publishing documentation must be validated against [proposed changes to the documentation](https://github.com/hasura/graphql-engine-mono/pull/3294). @marionschleifer what do you think?

~~The buildkite code in this branch is currently untested, and I am not sure how to test it.~~

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3380
GitOrigin-RevId: b24f6759c64ae29886c1f1b481b172febc512032
2022-01-31 13:16:29 +00:00
Abby Sassel
a091110364 server/tests: ephemeral BigQuery projects for CI test jobs
_Problem_
We currently run teardown/`DELETE` statements on the same, shared `hasura_test` dataset. This is not ideal as parallel test runs can and do affect each other, resulting in nondeterministic CI failures.

Closes https://github.com/hasura/graphql-engine-mono/issues/2521

_Solution and design_
This PR introduces ephemeral, isolated projects for each test run _in CI only_. Projects are created within [the Google Cloud Platform `data-sources-test-bigquery` directory](https://console.cloud.google.com/iam-admin/settings?folder=704256416468&orgonly=true&supportedpurview=organizationId) on each test run, and destroyed afterwards.

I've only introduced this change in CI for the time being:
  1. this isn't as much of an issue locally because we're less likely to run bigquery tests in parallel.
  2. to more quickly unblock https://github.com/hasura/graphql-engine/issues/7929.
  3. to limit the number of new projects created until we have a better idea of our usage vs GCP quota/limits.

Also updated the [internal wiki here](https://github.com/hasura/graphql-engine-mono/wiki/Testing-BigQuery) with this info.

_To verify_
- CI: [this job](https://buildkite.com/hasura/graphql-engine-mono/builds/3770#89e5bac6-16fe-447e-bcda-85cd47ea1b77) successfully runs all tests on a temporary project & dataset
- local: follow [these steps](https://github.com/hasura/graphql-engine-mono/wiki/Testing-BigQuery#ci--optional-dedicated-gcp-project-for-tests) to try the same setup locally

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3240
GitOrigin-RevId: d88d9cb7922266bfa962cfcb481e0272b8929a5d
2022-01-25 08:28:06 +00:00
Robert
5df40cc65b multitenant tests: reduce reliance on env vars, use fake cabal run
There's two major parts to this change:

1. cut down on environment variables needed to run the tests:
  - lux parameters that don't change are now in code
  - most remaining parameters have reasonable defaults
  - the only variable that is still required to be set is HASURA_MULTITENANT_SPEC_DB_URL
  - env.sh is no longer needed
2. find a better work-around for the problems running graphql-engine-multitenant
  via cabal run (https://github.com/haskell/cabal/issues/7914), by adding a shell
  script that implements a more correct version of cabal run.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3461
GitOrigin-RevId: 0939a79cc45cd3c1c103719552b12099678850dd
2022-01-24 18:15:25 +00:00
Evie Ciobanu
ab829b4355 test: have dev.sh pass through CLI options for unit tests
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3420
GitOrigin-RevId: eba7ac0bb609cf83ac1e636c24907601ffcd3651
2022-01-20 15:40:34 +00:00
Divi
432e94939e ci: use curlimages/curl docker image instead of appropriate/curl
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3409
GitOrigin-RevId: 22c7b7fba3f1b20a082530ed4ace364da1c8fd67
2022-01-20 07:30:41 +00:00
Auke Booij
a7dbe95666 Hlint all Haskell code
So far we've only used `hlint` to lint the OSS code. This moves some things around to lint the Pro code as well.

Note that the CI action only runs `hlint` on Haskell files that are _changed_ by a PR, relative to its merge-base (usually `main`).

As a reminder, you can use the `ignore-server-hlint-checks` label to prevent this from blocking a merge.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3383
GitOrigin-RevId: d5779c63d780f526a1d58ae4107f0d5262a23ec1
2022-01-18 04:06:15 +00:00
Vamshi Surabhi
2e1be1d2c5 bump python test suite dependencies
This PR upgrades some of the pinned dependencies do not build with python 3.10 - cffi, ruamel, py. Further, it upgrades other packages where the effort is minimal.

For the reviewers: Please review it commit by commit.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3367
GitOrigin-RevId: c5401fe289d3185a79c4d382297f86fbde139825
2022-01-17 07:40:56 +00:00
Evie Ciobanu
db22401e4f server: document python integration test suite
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3350
GitOrigin-RevId: 47dcebbfa84ed3681da40b3ea52e9b849b2fe8e4
2022-01-13 14:29:10 +00:00
Evie Ciobanu
e30d51ee38 server: bump pytest version
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3348
GitOrigin-RevId: b1fd23da3c8af59e28575438ebffe75f6fb79428
2022-01-13 10:30:55 +00:00
Vishnu Bharathi
214191ab5c ci: add hotfix oss console pipeline
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3221
GitOrigin-RevId: 8dc60862eb75e3988a7add72d9b7d33f20a62048
2022-01-12 04:38:21 +00:00
Brandon Simmons
736236a904 ci: combine pro/oss build jobs
The motivation here is:
- save some CI resources
- make it cleaner/more sensible to do library caching

Since OSS is a dependency of pro, and at the same time the various
executables can be built in parallel, doing this all in the same job
probably is more sensible.

Future improvements: there are separate pro and oss pipelines and one or
the other of those will need to wait unnecessarily while everything
finishes. In practice, currently this isn't a big deal, but we could use
the BuildKite blocking job API to get more fine-grained downstream
triggering.

GitOrigin-RevId: aa917996cbfc678becaaafca490d6edc7de89b1f
2022-01-03 06:26:26 +00:00
Brandon Simmons
fbaa0f75de utils: fix scripts/performance_report.sh
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3159
GitOrigin-RevId: d615bf8d1f8a78c4c157b613f0d17cbcb8aef538
2021-12-20 12:03:33 +00:00
Vishnu Bharathi
9b7495c4ae ci: add build and test job for arm64 oss cli migrations
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3128
GitOrigin-RevId: a8ab8fc8e022166e27fcc68199311b1626930759
2021-12-14 14:51:21 +00:00
Antoine Leblanc
f9b64b7d7c Fix some dev.sh issues
### Description

This PR fixes two small issues with the `dev.sh` script:

- when running the script without a specific test flag (`./scripts/dev.sh test`), we would start some of the containers more than once; this PR fixes this by checking whether the container is already running before starting it
- we were still doing some bigquery setup in the main code, despite that setup being already done in `parse-pytest-backend`, which would result in biquery setup errors even when testing non-bigquery backends; this PR just deletes the redundant line.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3017
GitOrigin-RevId: 86a53b1fe974aae733a02bdf045389a0d7436cb7
2021-11-30 16:37:34 +00:00
Gil Mizrahi
f1cbe4e72b pass MSSQL_SA_PASSWORD to mssql container, and add MSSQL_HOST
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2969
GitOrigin-RevId: 42e4cdf36306148ea34ea19fd42ce5048a38dbbc
2021-11-25 10:23:53 +00:00
Kirill Zaborsky
a2afe4116b BigQuery remote joins
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2874
Co-authored-by: Abby Sassel <3883855+sassela@users.noreply.github.com>
GitOrigin-RevId: 24b0304716795a28038629775238996c28b312a3
2021-11-24 16:22:55 +00:00
Gil Mizrahi
6dae132fce Add instructions for compiling graphql-engine natively on M1
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2049
GitOrigin-RevId: 9f10c26eef5382fb7b8e750ad272a5797b4cd6d8
2021-11-24 09:59:10 +00:00
Kali Vara Purushotham Santhati
1e5a48286f cli: split remote schema permissions metadata into seperate files
closes https://github.com/hasura/graphql-engine/issues/7033

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/1877
Co-authored-by: Aravind K P <8335904+scriptonist@users.noreply.github.com>
GitOrigin-RevId: 3764db9102d41f67a43f76aa79d28eb5485579ca
2021-10-14 06:52:57 +00:00
Abby Sassel
a9e3a1b4bd server/bigquery: improve test setup and documentation (close )
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2495
GitOrigin-RevId: ef250a0796200be6b75363b4275eb8a7ae780154
2021-10-12 17:59:44 +00:00
Abby Sassel
3a6f4e1737 server/bigquery: allow empty tables in replace_metadata. Default BigQueryField mode to Nullable
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2466
GitOrigin-RevId: 1f3599d1317235a31c98d7ed1ece2db92d82e916
2021-10-01 18:30:01 +00:00
Vishnu Bharathi
80cb68332c ci: add pro server jobs to buildkite
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2463
Co-authored-by: Divi <32202683+imperfect-fourth@users.noreply.github.com>
GitOrigin-RevId: e267657d47dc25e49787f87ff46ce16f285d9565
2021-10-01 14:24:19 +00:00
Antoine Leblanc
9bae641ac4 Always enable inherited roles tests
### Description

The inherited roles integration tests were behind a flag, and its corresponding fixture, presumably to avoid enabling the option globally. However,  introduced a new test using inherited roles that was not gated behind the flag, which fails when run with `dev.sh`. However, that test works on CI... because inherited roles are globally enabled there.

Consequently, this PR:
- globally enables inherited roles in dev.sh
- removes the flag and the associated fixture

https://github.com/hasura/graphql-engine-mono/pull/2358

Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
GitOrigin-RevId: ebfa6754873324bed15b2cc5e37ec2d8008e8f8d
2021-09-16 06:24:54 +00:00
Divi
2d8daf20dd ci: create non-root user hasura in graphql engine docker images
https://github.com/hasura/graphql-engine-mono/pull/2307

GitOrigin-RevId: 30aafbc5645b7528ab23482a07deca7ed3c97f9d
2021-09-14 08:18:20 +00:00
Abby Sassel
16b09f7d52 server/mssql: support transactions
https://github.com/hasura/graphql-engine-mono/pull/2268

GitOrigin-RevId: b1bc2812cd403688228b3ecf143aa36b3a6af707
2021-09-09 07:59:55 +00:00
Vishnu Bharathi
4b7a035012 ci: introduce docker image scans
https://github.com/hasura/graphql-engine-mono/pull/2177

GitOrigin-RevId: 03cbcbd0cdf1ec20941b7eed8842e5075ccee94d
2021-09-03 06:51:46 +00:00
Antoine Leblanc
7720496b95 Avoid building the engine twice when only running unit tests.
### Description

When running the tests with `dev.sh`, we always build the engine, before attempting to run the tests. This builds the engine twice, due to the flag change. This PR changes this to only do this first build if we're planning to actually run the integration tests.

https://github.com/hasura/graphql-engine-mono/pull/1964

GitOrigin-RevId: 32e8a6568f3665ad99adaf42421a711241974638
2021-08-26 11:53:06 +00:00
Brandon Simmons
c3300534b0 benchmarks: add a utility for viewing performance of a branch over time
https://github.com/hasura/graphql-engine-mono/pull/2111

GitOrigin-RevId: 98df2fb1ff67ce8482925dc068e285bcdf4e2bd4
2021-08-17 07:56:41 +00:00
Abby Sassel
6b3db83737 server/bigquery: run tests in CI
server/bigquery: run tests in CI 

[Example of successful BigQuery test job](https://app.circleci.com/pipelines/github/hasura/graphql-engine-mono/15951/workflows/86deb614-ddbc-4df5-9af3-288eeff6504a/jobs/283568)

TODO:
- [ ] I'll make corresponding changes in the OSS repo once this is merged
- [ ] @scriptnull or anyone with permissions: to require in CI

https://github.com/hasura/graphql-engine-mono/pull/1741

GitOrigin-RevId: d4ba446e9b13d213de135c73bb72973478575a29
2021-08-12 16:48:29 +00:00
pranshi06
d179a6f2ec server: support EdDSA keys for JWT
https://github.com/hasura/graphql-engine-mono/pull/1818

Co-authored-by: paritosh-08 <85472423+paritosh-08@users.noreply.github.com>
Co-authored-by: Puru Gupta <32328846+purugupta99@users.noreply.github.com>
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
GitOrigin-RevId: aae87d34cd19c97e66721a2bd7602d907aeb90b3
2021-08-12 01:54:06 +00:00
awjchen
046da3caca Combine oss and pro server configuration
https://github.com/hasura/graphql-engine-mono/pull/1983

GitOrigin-RevId: 91069bd5d1f9c5a166eae2317cf4435220b472b0
2021-08-11 04:19:27 +00:00
awjchen
3aa0027c40 server: add support for tagged and dynamic metrics
https://github.com/hasura/graphql-engine-mono/pull/1680

GitOrigin-RevId: c92d4f977b5b7fe9a0b71d3841e960a95f722299
2021-08-05 21:08:17 +00:00
Vishnu Bharathi
28c7a1d0d0 scripts: avoid premature release tagging
https://github.com/hasura/graphql-engine-mono/pull/2021

GitOrigin-RevId: fd40b14e99aba0a8624f91aac11083f52545fef1
2021-08-05 10:05:25 +00:00
Aravind K P
2898db17af cli-migrations-v2: use env variable to start temporary hasura instance
closes https://github.com/hasura/graphql-engine/issues/7319

https://github.com/hasura/graphql-engine-mono/pull/1950

GitOrigin-RevId: a2c798b5825423f9c2d37d436a3852c2262644d2
2021-08-04 06:11:06 +00:00
Divi
28788f4b0b cli: add VERSION env to pin cli version
https://github.com/hasura/graphql-engine-mono/pull/1845

Co-authored-by: Aravind K P <8335904+scriptonist@users.noreply.github.com>
Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
GitOrigin-RevId: f12ab6ef5a3aa1ee5181f5ddbf16f7232ef2874e
2021-07-29 09:21:57 +00:00
Abby Sassel
178fce0893 server/citus: pin docker image tag
Relates to https://github.com/orgs/hasura/projects/142#card-65243091 and https://github.com/hasura/graphql-engine-mono/pull/1862.

Issue: Citus test failing on [error message diff](https://app.circleci.com/pipelines/github/hasura/graphql-engine-mono/14388/workflows/4081512d-b443-4862-859b-2f081dd5f6e6/jobs/256507).

Cause: I forgot to specify a tag when running the Citus Docker image 🤦 thanks to @codingkarthik for updating the error message, this PR follows that one to pin the tag to avoid failures like this in future.

https://github.com/hasura/graphql-engine-mono/pull/1863

Co-authored-by: David Overton <7734777+dmoverton@users.noreply.github.com>
GitOrigin-RevId: 7f5180f8ecdffca3dbcb0eba57037fb15a2b5980
2021-07-22 15:46:18 +00:00
Abby Sassel
1cd9fc376d server/mysql: integrate MySQL tests into dev.sh workflow
https://github.com/hasura/graphql-engine-mono/pull/1848

GitOrigin-RevId: 2bd7c0a18448f042a1e35d21aaf42232acf48a46
2021-07-21 17:22:56 +00:00
Abby Sassel
76c322589c server/bigquery: Document BigQuery integration tests
This PR documents & streamlines how we run BigQuery integration tests locally as a first step to [running them in CI](https://github.com/hasura/graphql-engine-mono/issues/1525).

I've also created a hasura service account for internal use. [Internal docs here](https://github.com/hasura/graphql-engine-mono/wiki/Testing-BigQuery).

Thanks to FP Complete team for [the guidance here](https://docs.google.com/document/d/1dGDK0touUtsDxRQPonMxSoPbIfzBoSYo02tAjQEO7qA/edit?ts=60c0cf24#), which I've reused parts of.

https://github.com/hasura/graphql-engine-mono/pull/1732

GitOrigin-RevId: 303819d212aa073fbef685d077b1cfa583cd15fc
2021-07-06 11:13:06 +00:00
Abby Sassel
5b7d949ef4 server/citus: run tests in CI
https://github.com/hasura/graphql-engine-mono/pull/1526

GitOrigin-RevId: 4fd4214a799bc0d91cd2d29b1ea444f80714f6bc
2021-07-01 14:41:12 +00:00
Abby Sassel
b6b1354a2b server/mssql: Fix sqlcmd symlink Dockerfile
https://github.com/hasura/graphql-engine-mono/pull/1578

GitOrigin-RevId: 97d793b42b029cd51a8a96699ca4a67f1463d349
2021-06-15 19:45:08 +00:00
Abby Sassel
34de64bcdf server/mssql: run tests in CI ()
https://github.com/hasura/graphql-engine-mono/pull/1452

GitOrigin-RevId: 9fd345f7e5c24ebc3f47bdd0b15f84c9b8766f0c
2021-06-14 07:31:42 +00:00
Abby Sassel
dc950eda6a server/test: fix BACKEND variable in dev.sh script
GitOrigin-RevId: 0dae37c0aa50c8e4be59618d9d0a777f537fa4dc
2021-06-09 09:51:07 +00:00
kodiakhq[bot]
01d8a37706 server: fix asymptotics of event_log batch fetching
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
GitOrigin-RevId: 9b8afb502e94dd371a8596ccde97d023bd2344a9
2021-06-01 17:34:38 +00:00
Abby Sassel
41908936cf server: improve backend-specific test setup
GitOrigin-RevId: c40393bcdd78feaba7e9490ce4ed78de19b5bfc5
2021-05-25 13:55:11 +00:00
Antoine Leblanc
6e95f761f5 server: rewrite remote input parsers to deal with partial variable expansion (fix )
GitOrigin-RevId: e0b197a0fd1e259d43e6152b726b350c4d527a4b
2021-05-24 20:13:47 +00:00
Abby Sassel
1afa4ac3cc server/citus: feature branch
Co-authored-by: Vladimir Ciobanu <1017953+vladciobanu@users.noreply.github.com>
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
Co-authored-by: Ikechukwu Eze <22247592+iykekings@users.noreply.github.com>
Co-authored-by: Philip Lykke Carlsen <358550+plcplc@users.noreply.github.com>
Co-authored-by: Phil Freeman <630306+paf31@users.noreply.github.com>
GitOrigin-RevId: 1b964fe5f5f50380172cb702b6a328fed782b6b7
2021-05-21 02:47:51 +00:00
Aravind K P
9dfed5da57 cli: fix applying migrations in a different environment after config v3 update
GitOrigin-RevId: 2f5d554dba50da1d45ea9689afca07f4d0f2c1e5
2021-05-17 15:20:08 +00:00
Brandon Simmons
9c9bb43a53 server: restore proper batching behavior in event trigger processing ()
This essentially restores the original code from c425b554b8
(https://github.com/hasura/graphql-engine/pull/4013). Prior to this
commit we would slurp messages as fast as possible from the database
(one thing c425b55 fixed).

Another thing broken as a consequence of the same logic was the
removeEventFromLockedEvents logic which unlocks in-flight events
(breaking at-least-once delivery)

Some archeology, post-c425b55:

- cc8e2ccc erroneously attempted to refactor using `bracket`, resulting
  in the same slurp-all-events behavior (since we don't ever wait for
  processEvent to complete)
- at some point event processing within a batch is made serial, this
  reported as a bug. See: https://github.com/hasura/graphql-engine/issues/5189
- in 0ef52292b5 (which I approved...) an `async` is added, again
  causing the same issue...

GitOrigin-RevId: d8cbaab385267a4c3f1f173e268a385265980fb1
2021-04-29 04:02:05 +00:00
Karthikeyan Chinnakonda
aca8964fdc server: make postgres related ENV vars source specific
Co-authored-by: Tirumarai Selvan <8663570+tirumaraiselvan@users.noreply.github.com>
Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
GitOrigin-RevId: 8ec3db00f00e9c28bf2dc0f47bd312a656c61a69
2021-04-28 16:50:14 +00:00
kodiakhq[bot]
a935746e17 Integration test improvements (for speed/clarity), also increase polling interval for scheduled events
This claws back ~7min from integration tests (run serially, as with `dev.sh test --integration`
Further improvements would do well to focus on optimizing metadata operations, as `setup` dominates

GitOrigin-RevId: 76637d6fa953c2404627c4391447a05bf09355fa
2021-04-27 05:35:26 +00:00
Chris Done
53fc5617cf Feature/bigquery python tests
Co-authored-by: Aniket Deshpande <922486+aniketd@users.noreply.github.com>
GitOrigin-RevId: c425f84f1c9b8e38ebbfa509b6fa9298e023f386
2021-04-22 11:32:55 +00:00
Antoine Leblanc
84ed74aba1 server: fix empty remote input objects (fixes , )
GitOrigin-RevId: 3c474ee85b5d1271abfc8848e29ae1d3be28ff63
2021-04-21 01:17:01 +00:00
Antoine Leblanc
78abc5423a fix missing env variables in dev.sh
GitOrigin-RevId: ceeb73d304d2c2026081d5fbc7c68116a2628cb0
2021-04-20 13:43:06 +00:00
Brandon Simmons
a99fd32955 dev.sh: fix hpc after switch to dynamic executable for local dev, in …
GitOrigin-RevId: b31e5c839f3277f99a11208d315e756ddaf46ca6
2021-04-15 13:48:40 +00:00
Rakesh Emmadi
f7c37b007a server: use_prepared_statements option in add_pg_source metadata API
GitOrigin-RevId: 7b5af6d1cafc0b95fc86354293b3c3a4669e8bd2
2021-04-14 17:52:17 +00:00
kodiakhq[bot]
1b378ae6e8 Add a new /dev/rts_stats endpoint, enabled when '+RTS -T'
GitOrigin-RevId: e347bc2f66bad814516662e34bbd8322c214be25
2021-04-13 17:33:38 +00:00
Antoine Leblanc
3775fa419c ci: add citus support in dev.sh
GitOrigin-RevId: e0d4e5164f6ea1f0ab6d2566687797584732e515
2021-04-08 15:03:49 +00:00
Aravind K P
1cc3a66fef cli-migrations,ci: add config v3 image
GitOrigin-RevId: ee04d046fd70355495fb18976ebfcca00454c0da
2021-04-06 09:59:16 +00:00
David Overton
ad8df00c48 Tests/multisource
GitOrigin-RevId: 603d51d65dc4ff8a64c8d1abd19845bf6a316bde
2021-03-24 06:45:33 +00:00
hasura-bot
2099208e4b cli: merges cli-ext with cli
GITHUB_PR_NUMBER: 6640
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6640

Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
Co-authored-by: Aravind K P <8335904+scriptonist@users.noreply.github.com>
GitOrigin-RevId: 362d82d8ee18afd7efa63e9c8912f6c5444f82dd
2021-03-17 15:10:03 +00:00
Abby Sassel
64d52f5fa3 server: generalize integration tests
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
GitOrigin-RevId: 647edb8d293bf783c504b7e15ce02c56858b3b72
2021-03-11 18:18:41 +00:00
Abby Sassel
660b442943 server: misc. dev.sh improvements
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
GitOrigin-RevId: 4c93b49a370afb98747aabc1c977494297432402
2021-03-10 22:19:14 +00:00
Rikin Kachhia
3696d92743 add github workflow to compress new images in PRs
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
GitOrigin-RevId: cfef3585b120d82cafe5343cfceddbcd05969a15
2021-03-10 20:55:02 +00:00
Vishnu Bharathi
0c8dce35e4 ci: avoids doing backport release commit
GitOrigin-RevId: 5aa89862f0427bdc3ab1aa84ba8041286786fcb0
2021-03-04 12:45:08 +00:00
Vishnu Bharathi
bf76de3ab6 ci: exits in case of an error in Makefile, refactor cli-migrations-v2 image in accordance with hge base image
Co-authored-by: Aravind K P <8335904+scriptonist@users.noreply.github.com>
GitOrigin-RevId: 484147cb31becf6742620219d1c2b96a66bdb667
2021-03-02 07:26:30 +00:00
Abby Sassel
01cb59919f docs: changelog & server documentation improvements
GitOrigin-RevId: e6a5e464acb5492ab7a592362fff2bac6528dfaa
2021-03-01 20:53:43 +00:00
Praveen Durairaju
0ff26100bf update docs link to avoid redirects
GitOrigin-RevId: 1f2a1d21bfb9b2908d56305fa2dfb69270deafdf
2021-03-01 18:51:18 +00:00
Vladimir Ciobanu
281cb771ff server: add MSSQL support
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
Co-authored-by: Aravind K P <8335904+scriptonist@users.noreply.github.com>
GitOrigin-RevId: 699c453b9692e1b822f393f23ff5e6db4e010d57
2021-02-23 17:38:36 +00:00
Aravind K P
a0995d1b30 cli: add client package, statestore implementations, deprecate config v1
GitOrigin-RevId: 5a1d9cb772ac62603f2543bfe6c01a95c0a035c6
2021-02-17 04:21:11 +00:00
Aravind K P
3d5b30ce01 cli: fix metadata export behavior of config v2 with a v3 metadata server
GitOrigin-RevId: 76d1d01cee55f950cba3184dd57655c922a8d011
2021-02-15 15:15:44 +00:00
Aravind K P
64d776c9de cli: fix cli migrations tests
https://github.com/hasura/graphql-engine-mono/pull/437

* cli: fix cli migrations tests

* dummy change to trigger tests

* update validation metadata files

* fix test files

* dummy code change

GitOrigin-RevId: e9ad86a80756e94b33df670885f8abb96f8f2e38
2021-01-29 17:41:25 +00:00
Tirumarai Selvan A
88c3d7c787 tag release v1.4.0-alpha.1
GitOrigin-RevId: f09f265175226f96faba53ccfe68349377523f98
2021-01-29 12:31:13 +00:00
Swann Moreau
5bc0355bdd [server] coalesce multiple run_sql calls in tests ()
GitOrigin-RevId: abd7303aaf8e7a8739fd10574249aec450082ef8
2021-01-06 16:07:22 +00:00
Tirumarai Selvan
b544b87b9b Merge pull request from hasura/jberryman/5863-prep-refactoring
GitOrigin-RevId: 71b1453edf4b93ffc16a15ea3c6057bb865b6606
2020-12-20 06:53:38 +00:00
Brandon Simmons
d4d5c53468 dev.sh: additional postgres container logging, to support pgBadger
GitOrigin-RevId: dabd4099269121d4c1cb967017861e94201727c8
2020-12-01 16:04:58 +00:00
Tirumarai Selvan A
ba45d0454d tag release v1.3.3
GitOrigin-RevId: e7ded3aaa7677e8baa5a30217625f32b68f7fb0b
2020-11-17 15:05:44 +00:00
Vishnu Bharathi P
58c44f55dd Merge oss/master onto mono/main
GitOrigin-RevId: 1c8c4d60e033c8a0bc8b2beed24c5bceb7d4bcc8
2020-11-12 22:37:19 +05:30
Vishnu Bharathi P
666058ab7f oss: renames dot files and folders
GitOrigin-RevId: 540aeec3be091e1cfb7b05a988f50445534ed663
2020-11-12 22:37:19 +05:30
Aravind K P
4e4e3f3fd3
cli (cli-migrations image): fix problems running cli-migrations-v2 image as a non root user (close close ) ()
https://github.com/hasura/graphql-engine/pull/5306
2020-11-06 14:09:53 +00:00
Rakesh Emmadi
8bf84dff17
Avoid hdb_views schema for storing event trigger procedures ()
https://github.com/hasura/graphql-engine/pull/6135
2020-11-05 12:12:13 +00:00
Auke Booij
0540b279db
server: make more use of hlint ()
https://github.com/hasura/graphql-engine/pull/6059
2020-10-28 16:40:33 +00:00
Auke Booij
a4113eb9a6
add hlint config, run hlint through a github action, add to dev.sh ()
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2020-10-16 13:55:18 +02:00
Karthikeyan Chinnakonda
92ef504c9e
Server: add URL templating for event triggers and remote schemas ()
* add url templating for event triggers


Co-authored-by: Brandon Simmons <brandon.m.simmons@gmail.com>
2020-09-10 15:00:34 +05:30
Tirumarai Selvan
bbc27437d8
v1.3.2 assets () 2020-09-09 13:19:32 +05:30
Tirumarai Selvan
bcda0cc927
tag release v1.3.1 () 2020-08-21 18:51:36 +05:30
Tirumarai Selvan
e56515934a
tag release v1.3.1-beta.1 ()
backport from beta branch
2020-08-17 17:10:27 +05:30
Tirumarai Selvan
f0e1195e42
add log kinds in cli-migrations image ()
* add log kinds in cli-migrations image

* give hint to resolve timeout error

* minor changes and CHANGELOG
2020-08-06 21:18:55 +05:30
Aravind
d8481c3a1c
tag release v1.3.0 () 2020-07-20 20:38:00 +05:30
Aravind
7ae3e55a88
tag release v1.3.0-beta.4 () 2020-07-04 10:15:37 +05:30
Brandon Simmons
2b0e3774a3
5087 libpq pool leak ()
Shrink libpq buffers to 1MB before returning connection to pool. Closes 

See: https://github.com/hasura/pg-client-hs/pull/19

Also related:  
2020-07-01 09:23:10 +05:30
Aravind
a8cc3605b9
tag release v1.3.0-beta.3 () 2020-06-26 06:58:01 +05:30
Aravind Shankar
8092e04838
tag release v1.3.0-beta.2 () 2020-06-25 11:54:16 +05:30
Aravind
6486a4f0ec
cli: when applying metadata, assume empty metadata plugin files if they don't exist (close ) () 2020-06-24 16:08:48 +05:30
Aravind
5a648a9bb2
cli: fix cron triggers not being exported on a metadata export (close ) () 2020-06-16 14:38:36 +05:30
Aravind Shankar
1f8748c7b3
tag release v1.3.0-beta.1 ()
Co-authored-by: shahidhk <shahidh@hasura.io>
2020-06-08 20:33:47 +05:30
Shahidh K Muhammed
b9ed3dfe96
tag release v1.2.2 ()
Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
2020-06-02 13:52:16 +05:30
Tirumarai Selvan
c0d2bc6653
Remote Joins: Create relationships across database and remote schemas ()
add remote joins: Create relationships across database and remote schemas ()

Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>

Co-authored-by: Chris Done <chrisdone@gmail.com>
Co-authored-by: Chris Done <github@chrisdone.com>
Co-authored-by: wawhal <rishichandra.wawhal@gmail.com>
Co-authored-by: Aravind Shankar <aravind@hasura.io>
Co-authored-by: Brandon Simmons <brandon.m.simmons@gmail.com>
Co-authored-by: Rishichandra Wawhal <rishi@hasura.io>
Co-authored-by: Brandon Simmons <brandon@hasura.io>
Co-authored-by: nizar-m <19857260+nizar-m@users.noreply.github.com>
Co-authored-by: Praveen Durairaju <praveend.web@gmail.com>
Co-authored-by: rakeshkky <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Anon Ray <rayanon004@gmail.com>
Co-authored-by: Shahidh K Muhammed <shahidh@hasura.io>
Co-authored-by: soorajshankar <soorajshankar@users.noreply.github.com>
Co-authored-by: Sooraj Sanker <sooraj@Soorajs-MacBook-Pro.local>
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>
2020-05-27 20:32:58 +05:30
Brandon Simmons
ff62d5e0bf Migrate to GHC 8.10, upgrade dependencies. Closes
This also seems to squash a stubborn space leak we see with
subscriptions (linking to canonical  for reference).

This may also fix some of the "Unexpected exception" websockets
exceptions we are now surfacing (see e.g. )

Also: dev.sh: fix hpc reporting

Initial work on this done by Vamshi.
2020-05-13 19:13:02 -04:00
Brandon Simmons
69f24d47b3 dev.sh: bump version due to addition of croniter python dependency 2020-05-13 17:40:01 -04:00
Tirumarai Selvan
cc8e2ccc78
Scheduled triggers (close ) ()
server: add scheduled triggers 

Co-authored-by: Alexis King <lexi.lambda@gmail.com>
Co-authored-by: Marion Schleifer <marion@hasura.io>
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>
2020-05-13 18:03:16 +05:30
Shahidh K Muhammed
e14352eb72
tag release v1.2.1 () 2020-05-05 19:23:29 -05:00
Aravind Shankar
1ee3f40ad4
cli(migrations): load correct server image (fix ) () 2020-04-30 10:52:11 +05:30
Shahidh K Muhammed
fb35576984
tag release v1.2.0 () 2020-04-29 17:13:42 +05:30
Rikin Kachhia
c2df714e5d
docs: update migrations docs () 2020-04-29 13:30:26 +05:30
Shahidh K Muhammed
41d3595aab
tag release v1.2.0-beta.5 () 2020-04-24 19:56:51 +05:30
Aravind Shankar
aca11f0944
cli: spinner bug, add cli_environment in global config ()
close 
close 
close 
2020-04-22 14:45:42 +05:30
Shahidh K Muhammed
da3220f3fa
tag release v1.2.0-beta.4 ()
Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
2020-04-17 13:05:18 +05:30
Aravind Shankar
1ada6627b0
ci: add tests for cli-migrations image ()
- Created new job test_and_build_cli_migrations which runs after test_and_build_cli
- Build the cli-migrations and cli-migrations-v2 and save the images as tar image.
- Run the test defined in each workflow v1 and v2.
- Load the image that was built earlier in deploy step
2020-04-15 07:26:00 +05:30
Brandon Simmons
1fbf647c0d dev.sh: generate coverage report after dev.sh graphql-engine exit 2020-04-10 12:55:43 +05:30
Aravind Shankar
029c77416f
ci: fix cli-migrations-v2 build process () 2020-04-10 08:25:14 +05:30
Aravind Shankar
863910fed2
ci: fix console hot-fix releases () 2020-04-01 18:26:54 +05:30
Shahidh K Muhammed
1c4f203e4a
update manifests to v1.1.1 () 2020-03-28 16:45:43 +05:30
Aravind Shankar
1453516767 update readme 2020-03-19 17:53:11 +05:30
Aravind Shankar
315f19738c add support for cli-migrations-v2 2020-03-19 17:51:53 +05:30
Brandon Simmons
2d3313e643
Try again to fix flaky TestEventFlood test ()
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
2020-03-18 22:45:41 +05:30
Vamshi Surabhi
c52bfc540d
More robust forking, exception safety. Closes ()
This is the result of a general audit of how we fork threads, with a
detour into how we're using mutable state especially in websocket
codepaths, making more robust to async exceptions and exceptions
resulting from bugs.

Some highlights:
- use a wrapper around 'immortal' so threads that die due to bugs are
  restarted, and log the error
- use 'withAsync' some places
- use bracket a few places where we might break invariants
- log some codepaths that represent bugs
- export UnstructuredLog for ad hoc logging (the alternative is we
  continue not logging useful stuff)

I had to timebox this. There are a few TODOs I didn't want to address.
And we'll wait until this is merged to attempt  for
Control.Concurrent.Extended
2020-03-05 23:29:26 +05:30
Shahidh K Muhammed
4015e512db
tag release v1.2.0-beta.2 () 2020-03-05 22:49:17 +05:30
Shahidh K Muhammed
cb75660b3e
cli(migrations-img): add env to skip update prompts (fix ) () 2020-03-05 11:31:22 +05:30
Aravind Shankar
45bcb6b536
cli, server: use prerelease tag as channel for console assets cdn ()
Co-authored-by: Shahidh K Muhammed <muhammedshahid.k@gmail.com>
2020-03-04 20:10:47 +05:30
Shahidh K Muhammed
da5e4498d3
tag release v1.2.0-beta.1 () 2020-02-27 16:55:12 +05:30
Praveen Durairaju
865b150c82
noop: replace subdomain links with subpath ()
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
2020-02-27 15:43:07 +05:30
Shahidh K Muhammed
8844459fb8
fix regression on tag release script () 2020-02-25 12:29:55 +05:30
Phil Freeman
8df0151f99
Maintain downgrade commands in a text file, update when tagging () 2020-02-24 20:03:56 +05:30
Shahidh K Muhammed
88afa2ef70
update manifests to v1.1.0 () 2020-02-19 19:21:40 +05:30
Brandon Simmons
b863dc25c8 dev.sh: exit and report if graphql-engine fails to start up during integration tests 2020-02-03 18:50:10 -06:00
Brandon Simmons
87cbe68cab dev.sh: restore code coverage functionality
After https://github.com/hasura/graphql-engine/pull/3558
2020-01-23 18:35:18 -05:00
Brandon Simmons
1dd63a9386 Switch from stack to cabal-install for building server code (close ) ()
Co-authored-by: Alexis King <lexi.lambda@gmail.com>
2020-01-17 16:07:15 -06:00
Simon
7f1cc633ee cli(migrations-docker): take a specific env var for database url ()
Co-Authored-By: Shahidh K Muhammed <shahidh@hasura.io>
2019-12-27 10:00:06 +05:30
Tirumarai Selvan
c2829c662b cli(migrations-docker): randomize server port (fix ) ()
Co-authored-by: Shahidh K Muhammed <muhammedshahid.k@gmail.com>
2019-12-27 09:35:29 +05:30
Shahidh K Muhammed
7abe5dd744 merge branch 'release-v1.0' to master after stable release 2019-12-17 19:23:34 +05:30
Shahidh K Muhammed
1d482c02dc update manifests to v1.0.0 2019-12-16 13:06:38 +05:30
Rakesh Emmadi
421a182f64 export metadata without nulls, empty arrays & default values ()
* export metadata without nulls, empty arrays
* property tests for 'ReplaceMetadata' using QuickCheck
-> Derive Arbitrary class for 'ReplaceMetadata' dependant types

* reduce property test cases number to 30
QuickCheck generates the `ReplaceMetadata` value really large
for higher number test cases. Encoded JSON for such values is large and
consumes more memory. Thus, CI is giving up while running property
tests.

* circle-ci: Add property tests as saperate job
* add no command mode to tests
* add yaml.v2 to go mod
* remove indirect comment for yaml.v2 dependency
2019-12-14 00:47:38 -06:00
Brandon Simmons
db126dbea5 dev.sh: fix a bug affecting first time users () 2019-12-03 16:36:38 -06:00
Shahidh K Muhammed
e43be51dc6 update manifests to v1.0.0-rc.1 2019-12-02 13:02:46 +05:30
Brandon Simmons
84cfe18b3c dev.sh: combine tix from unit/haskell tests and integration tests in final report 2019-11-14 18:53:33 -05:00
Brandon Simmons
505ea4deef dev.sh: support running only integration tests or haskell/unit tests individually 2019-11-14 18:53:33 -05:00
Brandon Simmons
256be84dba dev.sh script: build before waiting for PG 2019-11-14 18:53:33 -05:00
Brandon Simmons
5827d652f6 In dev.sh tests: use postgres:11.5-alpine-postgis image which boots 3 sec faster 2019-11-14 18:53:33 -05:00
Brandon Simmons
da78a77fb2 In tests/dev.sh: upgrade ruamel to support python 3.7. Closes
Tested on python 3.5 and 3.7

We make light use of pyenv to set an appropriate python version if
installed. We could easily install a correct version too if we wanted
but that seemed invasive.

The newer ruamel was an annoying upgrade but also offers some
improvements that exposed some test suite issues (fixed later).
2019-11-14 18:53:33 -05:00
Shahidh K Muhammed
98911df54e update manifests to v1.0.0-beta.10 2019-11-14 11:20:09 +05:30
cheater
637c2cb555 fix server dev script (close ) ()
Signed-off-by: cheater <195390+cheater@users.noreply.github.com>
2019-11-06 13:10:46 +05:30
Shahidh K Muhammed
84eb09c1d2 update manifests to v1.0.0-beta.9 2019-10-31 14:29:32 +05:30
Shahidh K Muhammed
713d689dc1 update manifests to v1.0.0-beta.8 2019-10-22 12:54:35 +05:30
Rakesh Emmadi
6d92e4f9db save permissions, relationships and collections in catalog with 'is_system_defined' explicitly ()
* save permissions, relationships and collections in catalog with 'is_system_defined'
* Use common stanzas in the .cabal file
* Refactor migration code into lib instead of exe
* Add new server test suite that exercises migrations
* Make graphql-engine clean succeed even if the schema does not exist
2019-10-21 11:01:05 -05:00
Rikin Kachhia
1369876699
Revert "update manifests to v1.0.0-beta.8" () 2019-10-17 17:48:29 +05:30
Shahidh K Muhammed
7c1a964819 update manifests to v1.0.0-beta.8 2019-10-17 15:30:07 +05:30
Shahidh K Muhammed
b7fb4eb13e
cli-migrations: enable metadata api while applying migrations () 2019-10-17 15:28:27 +05:30
Shahidh K Muhammed
624ce0c6aa Revert "update manifests to v1.0.0-beta.7"
This reverts commit ae145c587f.
2019-10-11 11:14:25 +05:30
Shahidh K Muhammed
ae145c587f update manifests to v1.0.0-beta.7 2019-10-10 10:37:33 +05:30
Arun Babu Neelicattu
927a48753e disable auto-update checks in cli-migrations container entrypoint () 2019-09-05 16:57:10 +05:30
Shahidh K Muhammed
f9daaf40d3 update manifests to v1.0.0-beta.6 2019-08-29 18:38:18 +05:30
Shahidh K Muhammed
b8b71c1000 update manifests to v1.0.0-beta.5 2019-08-23 14:57:40 +05:30
Vamshi Surabhi
a5a07634a5
Merge branch 'master' into dev-scripts-and-doc-tweaks 2019-08-07 12:08:46 +05:30
Shahidh K Muhammed
bfc6fc8a5e update manifests to v1.0.0-beta.4 2019-08-01 16:26:32 +05:30
Arun Babu Neelicattu
dc52c0e507 migrations image: use nc zero-i/o mode when waiting for po… ()
This change adds the `-z` option to the nc command when waiting for
a port to be ready. This ensures that we exit correctly reporting
connection status.
2019-07-31 05:18:03 +00:00
Brandon Simmons
24409c7cbd Tweak local dev ekg console for longer timespan graphs 2019-07-26 01:17:56 -04:00
Brandon Simmons
0ed7891c0a Add fork of 'ekg' console for local development
These are assets forked from https://github.com/tibbe/ekg at revision fe5e5a1e67
Only monitor.js was modified slightly to take a local port from the URL
fragment.

I think we should feel free to continue modifying this, or replace with
our own tooling later on.
2019-07-26 01:17:56 -04:00
Brandon Simmons
bc4456eccc Add local development swiss army knife script scripts/dev.sh
At the moment we can...

...run tests in isolation, generating coverage report:

    $ dev.sh test

You can pass args to pytest as well. e.g. to run a specific test:

    $ dev.sh test -k "test_jsonb_has_all"

Launch a postgres container with useful dev defaults, with PostGIS,
cleaning up afterwards:

    $ dev.sh postgres

Build and launch graphql-engine in dev mode, connecting with a
`postgres` launched above

    $ dev.sh graphql-engine
2019-07-26 01:17:35 -04:00
Shahidh K Muhammed
694ba2cb6d update manifests to v1.0.0-beta.3 2019-07-12 11:00:10 +05:30