`spawn-dc-sqlite-agent` was incorrectly named, and the `start-dc-sqlite-agent` target was missing. In addition, we usually only make the `start-xyz` target public in `make help`, and so only that needs a comment.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5804
GitOrigin-RevId: 8064ce148d004ef73dbd8570ef30435423ede0a1
This means that if `remote_schemas/nodejs/package.json` changes, the
dependencies will be automatically reinstalled.
It also moves `package-lock.json` to the correct location (in the
directory in which we run `npm install`), and updates it.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5481
GitOrigin-RevId: f3fb431afd19de150f39ec2e4cb6572b896c870f
Making it easier to inject different ones later.
I also included a change to _.prettierignore_ so Visual Studio Code doesn't keep trying to reformat the JavaScript or YAML files in `server/tests-py`, as it can cause diffs to balloon for no obvious benefit.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5456
GitOrigin-RevId: bc6d548708160a328e1e61a00e19be8e124da025
It's about time.
To do this I had to check a few more boxes.
* I copied the flags from `graphql-engine.cabal` to the libraries in `server/lib`.
* I moved `Cacheable` instances of schema parser types beside the typeclass declaration.
* I removed imports of `Hasura.Prelude` from the tests, and rewrote them accordingly.
* I copied the `TestMonad` parse monad into `server/src-test/Hasura/GraphQL/Schema/RemoteTest.hs`, which was using it. I think this could be done with the real thing, but I tried replacing it with constraints and it messed with my head somewhat.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5311
GitOrigin-RevId: ebebcc50a16f2d517b7f730fe72410827ca3e86c
I'm trying to shore up the Python integration tests to make them more reliable. In doing so, I noticed this.
---
This ensures that the data directories are always in a mutable file system, and not going through overlayfs.
In my testing, this sped up PostgreSQL database initialization a lot. In particular, `CREATE DATABASE` commands went from > 5s to < 0.1s.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5248
GitOrigin-RevId: 5104aea32c92559c323803020d727bebfdafb8e6
I'm trying to shore up the Python integration tests to make them more reliable. In doing so, I noticed this.
---
Rather than hard-coding hostnames and ports, we can (and already do) inject these into the HGE process using environment variables.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5255
GitOrigin-RevId: 6bb593999ece42cedef6619f31f9d9b2e39f30ef
Docker Compose is now a plugin for Docker, bundled by default in Docker Desktop and many Linux distribution packages. The standalone `docker-compose` binary has been deprecated since Docker Compose v2.
Using the new version directly allows us to write development scripts that do not require `docker-compose` to be installed.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5185
GitOrigin-RevId: c8542b8b2405d1aa32288991688c6fde4af96383
We currently use `hpack` to generate the Cabal files from _package.yaml_
files for the two small libraries in _server/lib_. While this is more
convenient, we also check the Cabal files into the repository to avoid
needing an extra step upon pulling changes.
In order to ensure that the Cabal files do not get out of sync with the
hpack files, this introduces a few improvements:
1. Makefile targets to automatically generate the Cabal files without
needing to know the correct incantation. These targets are a
dependency of all build targets, so you can simply run
`make build-all` and it will work.
2. An extra comment at the top of all generated Cabal files that
explains how to regenerate it.
3. A `lint-hpack` Makefile target that verifies that the Cabal files
are up-to-date.
4. A CI job that runs `make lint-hpack`, to stop inconsistencies
getting merged into trunk.
Most of these changes are ported from #4794.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5217
GitOrigin-RevId: d3dfbe19ec00528368d357b6d0215a7ba4062f68
* Fix `make start-sqlserver` so it can find `sqlcmd`.
* Start the DC reference agent when testing all backends.
* Start backends in parallel so it doesn't take so long to spin them up. This is achieved by splitting all `start-*` tasks into `spawn-*` and `wait-for-*`.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5176
GitOrigin-RevId: ea483a755daa3fa6a8701f9d3dd0f393bc56510f
There's no reason to pin it to v3.7. This wasn't causing the Docker
Compose issues; it's a dependency version issue.
We actually don't need to install `docker-compose` at all. It's now a
Docker plugin (and is bundled in Docker Desktop) since Docker Compose
v2. You can run it with `docker compose`.
This also fixes a bug in _dev.sh_ due to a moved `make` target.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5197
GitOrigin-RevId: 638b2eb39c99e7a990e7a6c66388831438aed45f
Rather than a homebrewed approach, we can use `make` to figure out when
it's necessary to regenerate our venv.
This Makefile will regenerate _requirements.txt_ from
_requirements-top-level.txt_ when the latter is changed.
It will also regenerate the venv when _requirements.txt_ is changed
(i.e. changes are pulled, or it's regenerated as described above).
`make` uses file/directory timestamps to figure out what to rebuild.
This is probably more reliable than expecting people to update a version
number whenever they change a file.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5152
GitOrigin-RevId: 24b27d49bf6c4ba1d57ac38ea38ae278216c6d66
This helps us use the same versions locally as in CI. If you're using the Nix setup, it guarantees it.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5111
GitOrigin-RevId: 6e00cd7a78593df1e60fac37cc1195aba60e488f
I intend to use the Make targets around starting and stopping backend databases in order to wrap the Python integration tests in targets.
To that end, I have:
1. Factored out test infrastructure targets into _test-infrastructure.mk_.
2. Actually run the functions from _util.sh_; previously we were just running a bash script with no explicit commands. _util.sh_ must be `source`d and then you invoke the relevant function.
3. Run `docker-compose` through Nix if possible.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5136
GitOrigin-RevId: d7a01958045a82199c969ec3f88387d7d76d254b
I tried re-freezing _server/tests-py/requirements-top-level.txt_
recently, and discovered that it caused the tests to fail.
This pins a couple of dependencies so that we can safely re-freeze.
Specifically:
- `cryptography` is pinned at v3.*
- `graphene` is pinned at v2.*
- `PyJWT` is pinned at v2.3.*
- `websocket-client` is pinned at v0.56.0 (this was done in
_requirements.txt_ already, but that file is supposed to be
regenerated)
Upgrading `SQLAlchemy` required changing PostgreSQL URLs to use
"postgresql://" as the URL scheme, not "postgres://".
Updating `ruamel.yaml` caused a few tests to fail as we are passing
`ruamel.yaml.scalarstring.LiteralScalarString` values as header values.
This is fixed by explicitly converting header values to strings.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5120
GitOrigin-RevId: 9c12a3013c3d1f23dddbe781037663838b23f6f5
When pytest rewrites assertions to make them more useful, it also
truncates long assertion messages in the middle, often obscuring the
actual error.
Disabling this allows us to see the full message, which should hopefully
allow us to see the actual error.
In order to support this, we need to make sure that custom assertion
messages include the actual assertion information, as this will no
longer be rendered by pytest.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4835
GitOrigin-RevId: de6839a3b40d0edc7cb96e46625eebca0aaf0c95
This improves the experience of testing against BigQuery in a few ways:
1. There is an explicit instruction to create a dataset, which was not
documented.
2. The verification script uses the environment variables, just like the
tests.
3. The verification script passes the correct content type headers.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4706
GitOrigin-RevId: 15b5bd28b1762c130c0ec09d6babe8c08e05ab15
(Work here originally done by awjchen, rebased and fixed up for merge by
jberryman)
This is part of a merge train towards GHC 9.2 compatibility. The main
issue is the use of the new abstract `KeyMap` in 2.0. See:
https://hackage.haskell.org/package/aeson-2.0.3.0/changelog
Alex's original work is here:
#4305
BEHAVIOR CHANGE NOTE: This change causes a different arbitrary ordering
of serialized Json, for example during metadata export. CLI users care
about this in particular, and so we need to call it out as a _behavior
change_ as we did in v2.5.0. The good news though is that after this
change ordering should be more stable (alphabetical key order).
See: https://hasurahq.slack.com/archives/C01M20G1YRW/p1654012632634389
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4611
Co-authored-by: awjchen <13142944+awjchen@users.noreply.github.com>
GitOrigin-RevId: 700265162c782739b2bb88300ee3cda3819b2e87
…p CI (#3975)
Trying out a new convention here for jobs that generate multiple
artifacts at different points in time, and where downstream jobs depend
on just a particular subset of artifacts and don't need to wait for the
entire job. We accomplish this with separate `Input` steps, which we
unblock manually via calls to the buildkite API once particular sets of
artifacts are complete.
We could accomplish the same thing by having multiple build server jobs
focused on particular targets ( this is in fact how it used to work)
but that is quite wasteful of CI resources; in order to build any of
the binaries quickly, we need significant number of cores; but if we
only build particular targets, we don't take advantage I've all those
cores relative to the parallelism a full cabal build gives us. That
would likely be more difficult to maintain as well.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4256
Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
GitOrigin-RevId: 5f5603da8a85f9a491877d51a7c08764c7199602
UPDATE: After testing in CI it turns out that the compile time Improvement is better than expected: even though we always have to recompile the OSS lib (due to Version.hs), downstream packages like Pro and multi-tenant can still benefit from some caching and avoid full recompilation. In the best case this takes us from 22 minutes to 13 minutes total.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4104
GitOrigin-RevId: 76cbfc157064b33856e30f4c2b2ab2366f9c6089
