This installs the ODBC Driver 18 for SQL Server in all our shipped Docker images, and update our tests and documentation accordingly.
This version supports arm64, and therefore can run natively (or via Docker) on macOS on aarch64.
`msodbcsql17` is still installed in production-targeted Docker images so that users do not _have_ to migrate to the new driver.
Nix expressions are packaged for the new driver, as it is not yet available in nixpkgs.
In this version, [the default encryption setting was changed from "no" to "yes"](https://techcommunity.microsoft.com/t5/sql-server-blog/odbc-driver-18-0-for-sql-server-released/ba-p/3169228). In addition, "mandatory" and "optional" were added as synonyms for "yes" and "no" respectively.
I have therefore modified all connection strings in tests to specify `Encrypt=optional` (and changed some from `Encrypt=no`). I chose "optional" rather than "no" because I feel it's more honest; these connection strings will work with or without an encrypted connection.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6241
GitOrigin-RevId: 959f88dd1f271ef06a3616bc46b358f364f6cdfd
This upgrades CI and anyone using Nix to HLint v3.4.1.
If you're not using Nix, this doesn't actually _do_ anything on your
local machine; it's just a suggestion.
It also applies a bunch of simple HLint refactors, using
`make lint-hs-fix`.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6324
GitOrigin-RevId: de8267e4909d6dcd3f83543188517f3aaeebc5f3
Now that we use the mssql-tools image for running `sqlcmd`, we no longer need this (or indeed use it very much).
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6152
GitOrigin-RevId: be8852a93889553c3ffd2ec00c0dda81b36c3d05
Improvements to the Nix configuration so that macOS is supported.
Microsoft SQL Server is still not supported (yet; I have something in mind there), but the rest works. You can still use Homebrew to install the SQL Server drivers.
I had to make the following changes:
* I updated nixpkgs, because it's been a while.
* I made `ODBCINSTINI` optional so that it's not loaded on macOS, as it depends on `msodbcsql17`, which is broken on macOS.
* I upgraded OpenSSL.
* I set `DYLD_LIBRARY_PATH` on macOS so GHC finds OpenSSL; otherwise, it uses the wrong version of `libcrypto`, and fails with a fun error:
> WARNING: ghc is loading libcrypto in an unsafe way
* I patched GHC to fix compilation on macOS (copied from https://github.com/NixOS/nixpkgs/pull/149942).
To test this out, you can run `nix develop` (or install [direnv][] and add `use flake` to _.envrc.local_), and then try building HGE in the shell provided.
Fair warning: GHC needs to be built, and takes _aaages_ the first time. If this becomes useful to others, we can set up a shared cache.
[direnv]: https://direnv.net/
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5989
GitOrigin-RevId: 7130738d47709c37778b31c134061758ce23e959
* Add `source=` directives.
* Quote variables.
* Don't export variables that can fail to assign; instead, export afterwards.
* Write HGE stderr to the log file.
* Fix warnings around the PIDs.
* Disable a couple of false positives.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6097
GitOrigin-RevId: ed1c696b8735cd5d63ba30b3105040a0a9eca63c
This makes a few changes to the test scripts and makefiles in order to make things simpler for the average Apple user.
First of all, we change the `wait_for_mysql` function to use "localhost", not "127.0.0.1", as this fixed an issue on my system when attempting to connect to the MySQL server.
Secondly, we split the SQL Server test image into two:
* The first is the server itself, which now automatically uses `azure-sql-edge` as the image if you are on an aarch64 chip and using the `make` commands.
* The second is the initialization script. Because `sqlcmd` is not available in the `azure-sql-edge` image on aarch64, we use a separate container based on `mssql-tools` to initialize the server.
The README has been updated.
Tested on both macOS/aarch64 (with other changes) and Linux/x86_64.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5986
GitOrigin-RevId: b16e079861dcbcc66773295c47d715e443b67eea
`spawn-dc-sqlite-agent` was incorrectly named, and the `start-dc-sqlite-agent` target was missing. In addition, we usually only make the `start-xyz` target public in `make help`, and so only that needs a comment.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5804
GitOrigin-RevId: 8064ce148d004ef73dbd8570ef30435423ede0a1
This means that if `remote_schemas/nodejs/package.json` changes, the
dependencies will be automatically reinstalled.
It also moves `package-lock.json` to the correct location (in the
directory in which we run `npm install`), and updates it.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5481
GitOrigin-RevId: f3fb431afd19de150f39ec2e4cb6572b896c870f
Making it easier to inject different ones later.
I also included a change to _.prettierignore_ so Visual Studio Code doesn't keep trying to reformat the JavaScript or YAML files in `server/tests-py`, as it can cause diffs to balloon for no obvious benefit.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5456
GitOrigin-RevId: bc6d548708160a328e1e61a00e19be8e124da025
It's about time.
To do this I had to check a few more boxes.
* I copied the flags from `graphql-engine.cabal` to the libraries in `server/lib`.
* I moved `Cacheable` instances of schema parser types beside the typeclass declaration.
* I removed imports of `Hasura.Prelude` from the tests, and rewrote them accordingly.
* I copied the `TestMonad` parse monad into `server/src-test/Hasura/GraphQL/Schema/RemoteTest.hs`, which was using it. I think this could be done with the real thing, but I tried replacing it with constraints and it messed with my head somewhat.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5311
GitOrigin-RevId: ebebcc50a16f2d517b7f730fe72410827ca3e86c
I'm trying to shore up the Python integration tests to make them more reliable. In doing so, I noticed this.
---
This ensures that the data directories are always in a mutable file system, and not going through overlayfs.
In my testing, this sped up PostgreSQL database initialization a lot. In particular, `CREATE DATABASE` commands went from > 5s to < 0.1s.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5248
GitOrigin-RevId: 5104aea32c92559c323803020d727bebfdafb8e6
I'm trying to shore up the Python integration tests to make them more reliable. In doing so, I noticed this.
---
Rather than hard-coding hostnames and ports, we can (and already do) inject these into the HGE process using environment variables.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5255
GitOrigin-RevId: 6bb593999ece42cedef6619f31f9d9b2e39f30ef
Docker Compose is now a plugin for Docker, bundled by default in Docker Desktop and many Linux distribution packages. The standalone `docker-compose` binary has been deprecated since Docker Compose v2.
Using the new version directly allows us to write development scripts that do not require `docker-compose` to be installed.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5185
GitOrigin-RevId: c8542b8b2405d1aa32288991688c6fde4af96383
We currently use `hpack` to generate the Cabal files from _package.yaml_
files for the two small libraries in _server/lib_. While this is more
convenient, we also check the Cabal files into the repository to avoid
needing an extra step upon pulling changes.
In order to ensure that the Cabal files do not get out of sync with the
hpack files, this introduces a few improvements:
1. Makefile targets to automatically generate the Cabal files without
needing to know the correct incantation. These targets are a
dependency of all build targets, so you can simply run
`make build-all` and it will work.
2. An extra comment at the top of all generated Cabal files that
explains how to regenerate it.
3. A `lint-hpack` Makefile target that verifies that the Cabal files
are up-to-date.
4. A CI job that runs `make lint-hpack`, to stop inconsistencies
getting merged into trunk.
Most of these changes are ported from #4794.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5217
GitOrigin-RevId: d3dfbe19ec00528368d357b6d0215a7ba4062f68
* Fix `make start-sqlserver` so it can find `sqlcmd`.
* Start the DC reference agent when testing all backends.
* Start backends in parallel so it doesn't take so long to spin them up. This is achieved by splitting all `start-*` tasks into `spawn-*` and `wait-for-*`.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5176
GitOrigin-RevId: ea483a755daa3fa6a8701f9d3dd0f393bc56510f
There's no reason to pin it to v3.7. This wasn't causing the Docker
Compose issues; it's a dependency version issue.
We actually don't need to install `docker-compose` at all. It's now a
Docker plugin (and is bundled in Docker Desktop) since Docker Compose
v2. You can run it with `docker compose`.
This also fixes a bug in _dev.sh_ due to a moved `make` target.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5197
GitOrigin-RevId: 638b2eb39c99e7a990e7a6c66388831438aed45f
Rather than a homebrewed approach, we can use `make` to figure out when
it's necessary to regenerate our venv.
This Makefile will regenerate _requirements.txt_ from
_requirements-top-level.txt_ when the latter is changed.
It will also regenerate the venv when _requirements.txt_ is changed
(i.e. changes are pulled, or it's regenerated as described above).
`make` uses file/directory timestamps to figure out what to rebuild.
This is probably more reliable than expecting people to update a version
number whenever they change a file.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5152
GitOrigin-RevId: 24b27d49bf6c4ba1d57ac38ea38ae278216c6d66
This helps us use the same versions locally as in CI. If you're using the Nix setup, it guarantees it.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5111
GitOrigin-RevId: 6e00cd7a78593df1e60fac37cc1195aba60e488f
I intend to use the Make targets around starting and stopping backend databases in order to wrap the Python integration tests in targets.
To that end, I have:
1. Factored out test infrastructure targets into _test-infrastructure.mk_.
2. Actually run the functions from _util.sh_; previously we were just running a bash script with no explicit commands. _util.sh_ must be `source`d and then you invoke the relevant function.
3. Run `docker-compose` through Nix if possible.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5136
GitOrigin-RevId: d7a01958045a82199c969ec3f88387d7d76d254b
I tried re-freezing _server/tests-py/requirements-top-level.txt_
recently, and discovered that it caused the tests to fail.
This pins a couple of dependencies so that we can safely re-freeze.
Specifically:
- `cryptography` is pinned at v3.*
- `graphene` is pinned at v2.*
- `PyJWT` is pinned at v2.3.*
- `websocket-client` is pinned at v0.56.0 (this was done in
_requirements.txt_ already, but that file is supposed to be
regenerated)
Upgrading `SQLAlchemy` required changing PostgreSQL URLs to use
"postgresql://" as the URL scheme, not "postgres://".
Updating `ruamel.yaml` caused a few tests to fail as we are passing
`ruamel.yaml.scalarstring.LiteralScalarString` values as header values.
This is fixed by explicitly converting header values to strings.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5120
GitOrigin-RevId: 9c12a3013c3d1f23dddbe781037663838b23f6f5
When pytest rewrites assertions to make them more useful, it also
truncates long assertion messages in the middle, often obscuring the
actual error.
Disabling this allows us to see the full message, which should hopefully
allow us to see the actual error.
In order to support this, we need to make sure that custom assertion
messages include the actual assertion information, as this will no
longer be rendered by pytest.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4835
GitOrigin-RevId: de6839a3b40d0edc7cb96e46625eebca0aaf0c95
This improves the experience of testing against BigQuery in a few ways:
1. There is an explicit instruction to create a dataset, which was not
documented.
2. The verification script uses the environment variables, just like the
tests.
3. The verification script passes the correct content type headers.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4706
GitOrigin-RevId: 15b5bd28b1762c130c0ec09d6babe8c08e05ab15
(Work here originally done by awjchen, rebased and fixed up for merge by
jberryman)
This is part of a merge train towards GHC 9.2 compatibility. The main
issue is the use of the new abstract `KeyMap` in 2.0. See:
https://hackage.haskell.org/package/aeson-2.0.3.0/changelog
Alex's original work is here:
#4305
BEHAVIOR CHANGE NOTE: This change causes a different arbitrary ordering
of serialized Json, for example during metadata export. CLI users care
about this in particular, and so we need to call it out as a _behavior
change_ as we did in v2.5.0. The good news though is that after this
change ordering should be more stable (alphabetical key order).
See: https://hasurahq.slack.com/archives/C01M20G1YRW/p1654012632634389
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4611
Co-authored-by: awjchen <13142944+awjchen@users.noreply.github.com>
GitOrigin-RevId: 700265162c782739b2bb88300ee3cda3819b2e87
