server/tests-hspec: Simplifying the steps required to run these on macOS on aarch64.

This makes a few changes to the test scripts and makefiles in order to make things simpler for the average Apple user.

First of all, we change the `wait_for_mysql` function to use "localhost", not "127.0.0.1", as this fixed an issue on my system when attempting to connect to the MySQL server.

Secondly, we split the SQL Server test image into two:

* The first is the server itself, which now automatically uses `azure-sql-edge` as the image if you are on an aarch64 chip and using the `make` commands.
* The second is the initialization script. Because `sqlcmd` is not available in the `azure-sql-edge` image on aarch64, we use a separate container based on `mssql-tools` to initialize the server.

The README has been updated.

Tested on both macOS/aarch64 (with other changes) and Linux/x86_64.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5986
GitOrigin-RevId: b16e079861dcbcc66773295c47d715e443b67eea

docker-compose.yml

@@ -59,7 +59,7 @@ services:
       - ./docker-compose/postgres/init.sh:/docker-entrypoint-initdb.d/init-hasura.sh:ro
 
   sqlserver:
-    build: ./docker-compose/sqlserver/
+    image: ${MSSQL_IMAGE:-mcr.microsoft.com/mssql/server:2019-latest@sha256:a098c9ff6fbb8e1c9608ad7511fa42dba8d22e0d50b48302761717840ccc26af}
     ports:
       - "65003:1433"
     environment:
@@ -69,6 +69,18 @@ services:
     volumes:
       - mssql-data:/var/opt/mssql
 
+  # Separated from the `sqlserver` container so we can use the `mssql-tools` image to run `sqlcmd`.
+  # `sqlcmd` is not available in the `azure-sql-edge` image on arm64, which we use for testing on macOS.
+  sqlserver-init:
+    image: mcr.microsoft.com/mssql-tools
+    command:
+      - /init.sh
+    environment:
+      SERVER_HOST: sqlserver
+    volumes:
+      - ./docker-compose/sqlserver-init/init.sh:/init.sh
+      - ./docker-compose/sqlserver-init/init.sql:/init.sql
+
   citus:
     image: citusdata/citus:10.1@sha256:7e497e5ca18d7f2ae2a66c1d5d676b548e9221b7e6294adfb03006adad85502c
     ports:

docker-compose/sqlserver-init/init.sh

@@ -0,0 +1,27 @@
+#!/bin/bash
+
+set -e
+set -u
+set -o pipefail
+
+SERVER_PORT="${1:-1433}"
+SLEEP=3
+MAX_ATTEMPTS=10
+
+if [[ -z "${SERVER_HOST-}" ]]; then
+  echo 'The server host must be set.'
+fi
+
+echo "Initializing ${SERVER_HOST}:${SERVER_PORT}..."
+
+sleep "${SLEEP}"
+for i in $(seq 1 $MAX_ATTEMPTS); do
+  echo "Attempt #${i} / ${MAX_ATTEMPTS}:"
+  if /opt/mssql-tools/bin/sqlcmd -S "${SERVER_HOST},${SERVER_PORT}" -U SA -P "DockerComposePassword!" -i /init.sql; then
+    break
+  fi
+  echo "Waiting ${SLEEP} seconds..."
+  sleep "${SLEEP}"
+done
+
+echo Finished attempts.

docker-compose/sqlserver/Dockerfile

@@ -1,9 +0,0 @@
-FROM mcr.microsoft.com/mssql/server:2019-latest@sha256:a098c9ff6fbb8e1c9608ad7511fa42dba8d22e0d50b48302761717840ccc26af
-
-COPY init.sql /init.sql
-COPY run-init.sh /run-init.sh
-COPY entrypoint.sh /entrypoint.sh
-
-CMD /bin/bash ./entrypoint.sh
-
-ENV PATH="/opt/mssql-tools/bin:${PATH}"

docker-compose/sqlserver/entrypoint.sh

@@ -1,7 +0,0 @@
-#!/bin/bash
-
-# Run init-script with long timeout - and make it run in the background
-bash /run-init.sh &
-
-# Start SQL server, quitely
-/opt/mssql/bin/sqlservr 2>&1 > /dev/null

docker-compose/sqlserver/run-init.sh

@@ -1,13 +0,0 @@
-#!/bin/bash
-
-echo 'Attempting to run script (10 times max)'
-PORT="${1:-1433}"
-SLEEP="3"
-
-sleep "${SLEEP}"
-for i in 1 2 3 4 5 6 7 8 9 10; do
-  echo Attempt "#$i" - will wait "${SLEEP}" seconds if this fails ...
-  sqlcmd -S 127.0.0.1,"${PORT}" -U SA -P "DockerComposePassword!" -i init.sql && break || sleep "${SLEEP}";
-done
-
-echo Finished attempts.

scripts/make/test-infrastructure.mk

@@ -27,19 +27,18 @@ COCKROACH_DBUSER = root
 # function from util.sh (or anywhere else).
 DB_UTILS = source ./.buildkite/scripts/util/util.sh;
 
-ifneq ($(shell command -v sqlcmd),)
-MSSQL_SQLCMD = sqlcmd
-MSSQL_SQLCMD_PORT = $(MSSQL_PORT)
+# Use the Azure SQL Edge image instead of the SQL Server image on arm64.
+# The latter doesn't work yet.
+ifeq ($(shell uname -m),arm64)
+MSSQL_IMAGE=mcr.microsoft.com/azure-sql-edge
 else
-ifneq ($(shell [[ -e /opt/mssql-tools/bin/sqlcmd ]] && echo true),)
-MSSQL_SQLCMD = /opt/mssql-tools/bin/sqlcmd
-MSSQL_SQLCMD_PORT = $(MSSQL_PORT)
-else
-MSSQL_SQLCMD = docker compose exec --no-TTY sqlserver sqlcmd
-MSSQL_SQLCMD_PORT = 1433
-endif
+MSSQL_IMAGE=  # allow the Docker Compose file to set the image
 endif
 
+# Run `sqlcmd` in a separate image when waiting for SQL Server to start.
+MSSQL_SQLCMD = docker run --rm --platform=linux/amd64 --net=host mcr.microsoft.com/mssql-tools /opt/mssql-tools/bin/sqlcmd
+
+export MSSQL_IMAGE
 export MSSQL_SQLCMD
 
 define stop_after
@@ -99,11 +98,12 @@ start-sqlserver: spawn-sqlserver wait-for-sqlserver
 .PHONY: spawn-sqlserver
 spawn-sqlserver:
 	docker compose up -d sqlserver
+	docker compose run sqlserver-init
 
 .PHONY: wait-for-sqlserver
 wait-for-sqlserver:
 	$(DB_UTILS) wait_for_mssql $(MSSQL_PORT)
-	$(DB_UTILS) wait_for_mssql_db $(MSSQL_SQLCMD_PORT) "$(MSSQL_DBNAME)" "$(MSSQL_DBUSER)" "$(MSSQL_DBPASSWORD)"
+	$(DB_UTILS) wait_for_mssql_db $(MSSQL_PORT) "$(MSSQL_DBNAME)" "$(MSSQL_DBUSER)" "$(MSSQL_DBPASSWORD)"
 
 .PHONY: start-mysql
 ## start-mysql: start local MariaDB in Docker and wait for it to be ready

server/tests-hspec/README.md

@@ -28,7 +28,7 @@ For motivation, rationale, and more, see the [test suite rfc](../../rfcs/hspec-t
   - [Troubleshooting](#troubleshooting)
     - [`Database 'hasura' already exists. Choose a different database name.`](#database-hasura-already-exists-choose-a-different-database-name)
     - [General `DataConnector` failures](#general-dataconnector-failures)
-    - [`SQLServer` failures on Apple M1 chips](#sqlserver-failures-on-apple-m1-chips)
+    - [`SQLServer` failures on Apple M1 chips](#microsoft-sql-server-failures-on-apple-aarch64-chips)
 
 ## Required setup for BigQuery tests
 
@@ -424,42 +424,48 @@ This typically indicates persistent DB state between test runs. Try `docker comp
 
 The DataConnector agent might be out of date. If you are getting a lot of test failures, first try rebuilding the containers with `docker compose build` to make sure you are using the latest version of the agent.
 
-### `SQLServer` failures on Apple M1 chips
+### Microsoft SQL Server failures on Apple aarch64 chips
 
-We have a few problems with SQLServer on M1:
+This applies to all Apple hardware that uses aarch64 chips, e.g. the MacBook M1 or M2.
 
-1. Compiler bug in GHC 8.10.7 on M1.
+We have a few problems with Microsoft SQL Server on Apple aarch64:
 
-   Due to compiler bugs in GHC 8.10.7 we need to use patched Haskell ODBC bindings as a workaround for M1 systems.
-   Make the following change in the `cabal.project`:
+1.  Due to compiler bugs in GHC 8.10.7, we need to use patched Haskell ODBC bindings as a workaround for aarch64 systems.
 
-   ```diff
+    Add the following to `cabal.project.local`:
+
+    ```
     source-repository-package
       type: git
-   -   location: https://github.com/fpco/odbc.git
-   -   tag: 3d80ffdd4a2879f0debecabb56d834d2d898212b
-   +   location: https://github.com/soupi/odbc.git
-   +   tag: a6acf6b4eca31022babbf8045f31a0f7f26c5923
+      location: https://github.com/soupi/odbc.git
+      tag: a6acf6b4eca31022babbf8045f31a0f7f26c5923
     ```
 
-2. Microsoft did not release SQL Server for M1. We need to use Azure SQL Edge instead.
+2.  Microsoft has not yet released SQL Server for aarch64. We need to use Azure SQL Edge instead.
 
-   Switch the docker image in `docker-compose/sqlserver/Dockerfile` to `azure-sql-edge`:
+    You don't need to do anything if you're using the `make` commands; they will provide the correct image automatically.
 
-   ```diff
-   - FROM mcr.microsoft.com/mssql/server:2019-latest@sha256:a098c9ff6fbb8e1c9608ad7511fa42dba8d22e0d50b48302761717840ccc26af
-   + FROM mcr.microsoft.com/azure-sql-edge
+    If you run `docker compose` directly, make sure to set the environment variable yourself:
+
+    ```sh
+    export MSSQL_IMAGE='mcr.microsoft.com/azure-sql-edge'
     ```
 
-   Note: you might need to rebuild the Docker images with `docker compose build`
+    You can add this to your _.envrc.local_ file if you like.
 
-3. Azure SQL Edge does not ship with the `sqlcmd` utility with which we use to setup the SQL Server schema.
+3.  Azure SQL Edge for aarch64 does not ship with the `sqlcmd` utility with which we use to setup the SQL Server schema.
 
-   1. Install it locally instead, with brew: `brew install microsoft/mssql-release/mssql-tools`.
-   2. To start the test suite's backends, we need to setup the SQL Server schema using our local `sqlcmd`.
-      To start the backends, run this command instead:
+    If you need it, you can instead use the `mssql-tools` Docker image, for example:
 
-      ```diff
-      - docker compose up
-      + docker compose up & (cd docker-compose/sqlserver/ && ./run-init.sh 65003) && fg
     ```
+    docker run --rm -it --platform=linux/amd64 --net=host mcr.microsoft.com/mssql-tools \
+      /opt/mssql-tools/bin/sqlcmd -S localhost,65003 -U SA -P <password>
+    ```
+
+    To make this easier, you might want to define an alias:
+
+    ```
+    alias sqlcmd='docker run --rm -it --platform=linux/amd64 --net=host mcr.microsoft.com/mssql-tools /opt/mssql-tools/bin/sqlcmd'
+    ```
+
+    You can also install them directly with `brew install microsoft/mssql-release/mssql-tools`.