hasura/graphql-engine
1
0
Fork 0
mirror of https://github.com/hasura/graphql-engine.git synced 2024-07-07 07:56:36 +03:00
Code Issues Projects Releases Wiki Activity

docker: Always use the official SQL Server image for testing.

Browse Source 
Now that Docker for macOS supports using Rosetta for x86/amd64 emulation, we no longer need to use the `azure-sql-edge` image. We can always use the official `mcr.microsoft.com/mssql/server` one.

This also means that we no longer need the separate healthcheck container, because the official image ships with `sqlcmd`.

When this is merged, you will need to ensure you have enabled Rosetta emulation in the Docker settings to test against SQL Server on macOS. This requires macOS 13 (Ventura).

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10139
GitOrigin-RevId: 2225bf5f4c6d3632da1f29b2229c9b04ead5e34c
This commit is contained in:
Samir Talwar 2023-09-25 14:25:40 +01:00 committed by hasura-bot
parent ad0823094b
commit 2467e23ef6
13 changed files with 38 additions and 139 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
.envrc
View File

@ -3,12 +3,6 @@
# This file provides some tooling on an opt-in basis via `direnv`
# (https://direnv.net/)
# We cannot use the proper MSSQL Server image on arm64, as it's not available.
# Instead, we use `mcr.microsoft.com/azure-sql-edge`.
if [[ "$(uname -m)" == 'arm64' ]]; then
export MSSQL_IMAGE='mcr.microsoft.com/azure-sql-edge'
fi
# To use the functionality here, create an `.envrc.local` file in this folder
# that runs the functions you need.
# There is an example in `.envrc.local.example` you can start with

4
.envrc.local.example
View File

@ -24,10 +24,6 @@
# HASURA_BIGQUERY_SERVICE_KEY=$(cat ../bigquery-service-account.json)
# export HASURA_BIGQUERY_SERVICE_KEY
### Enable to use the correct image for SQLServer on an M1
### (the env var is used in our Docker Compose files)
# export MSSQL_IMAGE='mcr.microsoft.com/azure-sql-edge'
### Export EE license key for running pro tests locally
### This depends on a `ee-license-key.txt` living in the directory above
### the `graphql-engine` repository

15
docker-compose.yaml
View File

@ -21,11 +21,16 @@
#
# Facts:
#
# * The SERVICE PORTS numbering start at 65001, 65002, etc. to avoid bother
# * The SERVICE PORTS numbering start at 65001, 65002, etc. to avoid bothering
# existing instances of databases.
#
# * The login credentials are, where possible, all "hasura" to avoid unnecessary
# mental overhead.
#
# * The SQL Server image will only work on macOS if you enable the relevant
# settings. Open the Docker Desktop settings, enable "Use Virtualization
# framework" in the "General" tab, and "Use Rosetta for x86/amd64 emulation on
# Apple Silicon" in the "Features in development" tab.
version: "3.6"
@ -81,14 +86,6 @@ services:
volumes:
- mssql-data:/var/opt/mssql
sqlserver-healthcheck:
extends:
file: docker-compose/databases.yaml
service: sqlserver-healthcheck
depends_on:
sqlserver:
condition: service_started
dc-reference-agent:
extends:
file: dc-agents/docker-compose.yaml

30
docker-compose/databases.yaml
View File

@ -81,12 +81,8 @@ services:
- /var/lib/postgresql/data
sqlserver:
# We cannot use this image on arm64, as it's not available.
# Instead, we use `mcr.microsoft.com/azure-sql-edge`.
# This uses an environment variable so it can be overridden by scripts to
# provide that image instead.
image: ${MSSQL_IMAGE:-mcr.microsoft.com/mssql/server:2019-latest}
init: true # azure-sql-edge doesn't shut down properly without this
image: mcr.microsoft.com/mssql/server:2019-latest
platform: linux/amd64
ports:
- 1433
environment:
@ -95,35 +91,15 @@ services:
MSSQL_SA_PASSWORD: "Password!"
volumes:
- /var/opt/mssql
# We would prefer to attach the healthcheck directly to the `sqlserver` container.
# However, while we can do this in the `mcr.microsoft.com/mssql/server` image,
# it's not possible when running on macOS, as we don't use that image, but
# instead `mcr.microsoft.com/azure-sql-edge`.
# The `mcr.microsoft.com/azure-sql-edge` image does not provide `sqlcmd`.
# We therefore need to run it in a separate container.
# We can reconsider this once the MSSQL Server image can run on arm64.
sqlserver-healthcheck:
image: mcr.microsoft.com/mssql-tools
platform: linux/amd64
command:
- sleep
- inf
init: true # handle signals correctly
healthcheck:
test:
- CMD-SHELL
- |
/opt/mssql-tools/bin/sqlcmd -S 'sqlserver,1433' -U SA -P "Password!"
/opt/mssql-tools/bin/sqlcmd -U SA -P "$$SA_PASSWORD"
start_period: 5s
interval: 5s
timeout: 10s
retries: 20
# This is commented out because this file can be extended, and dependencies don't work for extensions.
# Instead, ensure you also run this one.
# depends_on:
# sqlserver:
# condition: service_started
mysql:
image: mysql

2
frontend/docker/README.md
View File

@ -21,7 +21,7 @@ You may also need to flip the [feature flags here](http://localhost:4200/setting
## MSSQL / SQL Server
In order to initialize and populate a MSSQL demo database automatically, we making use of the `mssql-tools` image. This image runs a few commands against the `mssql` image to create the demo data.
In order to initialize and populate a MSSQL demo database automatically, we making use of the `mssql-init` container. This image runs a few commands against the `mssql` image to create the demo data.
This scripting is done in `docker/DataSources/mssql/run-initialization.sh`.

32
frontend/docker/docker-compose.template.yml
View File

@ -35,6 +35,10 @@ services:
- '8100:8100'
volumes:
- ./DataSources/sqlite.db:/sqlite.db
# The SQL Server image will only work on macOS if you enable the relevant
# settings. Open the Docker Desktop settings, enable "Use Virtualization
# framework" in the "General" tab, and "Use Rosetta for x86/amd64 emulation on
# Apple Silicon" in the "Features in development" tab.
mssql:
container_name: 'mssql'
user: root
@ -43,14 +47,23 @@ services:
volumes:
- mssql-database-data:/var/opt/mssql
stdin_open: true
image: mcr.microsoft.com/azure-sql-edge:latest
# if not running Apple Silicon (M1/M2 etc), you can use this:
#image: mcr.microsoft.com/mssql/server:2022-latest
init: true # azure-sql-edge doesn't shut down properly without this
image: &mssql-image mcr.microsoft.com/mssql/server:2019-latest
platform: linux/amd64
environment:
ACCEPT_EULA: 'Y'
SA_PASSWORD: 'Password!'
MSSQL_SA_PASSWORD: 'Password!'
mssql-init:
depends_on:
- mssql
container_name: 'mssql-init'
image: *mssql-image
platform: linux/amd64
command:
- /opt/mssql_scripts/run-initialization.sh
stdin_open: true
volumes:
- ./DataSources/mssql:/opt/mssql_scripts
mongo:
image: mongo
restart: always
@ -62,17 +75,6 @@ services:
- '27017:27017'
command: mongod --setParameter enableLocalhostAuthBypass=0 --dbpath=/data/db --bind_ip_all --port 27017
# sqlcmd (tools) are not included in the arm64 image
mssql-tools:
depends_on:
- mssql
container_name: 'mssql-tools'
image: mcr.microsoft.com/mssql-tools:latest
command: bash /opt/mssql_scripts/run-initialization.sh
stdin_open: true
volumes:
- ./DataSources/mssql:/opt/mssql_scripts
graphql-engine:
image: hasura/graphql-engine:latest

2
scripts/make/repl.mk
View File

@ -3,7 +3,7 @@
.PHONY: repl-sqlserver
## repl-sqlserver: start a sqlserver docker image and connect to it using sqlcmd
repl-sqlserver:
@docker compose up -d --wait sqlserver-healthcheck
@docker compose up -d --wait sqlserver
@sqlcmd -S localhost,$(shell docker compose port sqlserver 1433 | sed -e 's#.*:\(\)#\1#') -U SA -P "Password!"
.PHONY: repl-postgres

10
scripts/make/test-infrastructure.mk
View File

@ -2,16 +2,6 @@ API_TESTS_DOCKER_COMPOSE = docker compose --project-directory=./server/lib/api-t
API_TESTS_PRO_DOCKER_COMPOSE = docker compose --project-directory=./pro/server/lib/api-tests
PYTHON_TESTS_DOCKER_COMPOSE = docker compose --project-directory=./server/tests-py
# Use the Azure SQL Edge image instead of the SQL Server image on arm64.
# The latter doesn't work yet.
ifeq ($(shell uname -m),arm64)
MSSQL_IMAGE=mcr.microsoft.com/azure-sql-edge
else
MSSQL_IMAGE= # allow the Docker Compose file to set the image
endif
export MSSQL_IMAGE
TEST_MSSQL_CONNECTION_STRING = Driver={ODBC Driver 18 for SQL Server};Server=localhost,65003;Uid=sa;Pwd=Password!;Encrypt=optional
TEST_POSTGRES_URL = postgres://hasura:hasura@localhost:65002/hasura

8
scripts/make/tests.mk
View File

@ -147,7 +147,7 @@ test-unit: remove-tix-file
.PHONY: test-integration-mssql
## test-integration-mssql: run MS SQL Server integration tests
test-integration-mssql: remove-tix-file
docker compose up --build --detach --wait sqlserver-healthcheck
docker compose up --build --detach --wait sqlserver
HASURA_MSSQL_CONN_STR='$(TEST_MSSQL_CONNECTION_STRING)' \
cabal run graphql-engine:test:graphql-engine-test-mssql
@ -162,7 +162,7 @@ test-integration-postgres: remove-tix-file
## test-native-queries: run all tests for the Native Query feature
test-native-queries:
cabal build exe:graphql-engine-pro
docker compose up --build --detach --wait postgres citus cockroach sqlserver-healthcheck
docker compose up --build --detach --wait postgres citus cockroach sqlserver
HSPEC_MATCH=NativeQueries make test-unit
HSPEC_MATCH=NativeQueries \
GRAPHQL_ENGINE=$(GRAPHQL_ENGINE_PRO_PATH) \
@ -183,7 +183,7 @@ test-native-queries-postgres:
## test-native-queries-sqlserver: run all sqlserver tests for the Native Query feature
test-native-queries-sqlserver: remove-tix-file
cabal build exe:graphql-engine-pro
docker compose up --build --detach --wait postgres sqlserver-healthcheck
docker compose up --build --detach --wait postgres sqlserver
HSPEC_MATCH=${HSPEC_MATCH:-"NativeQueries"}
HASURA_TEST_BACKEND_TYPE=SQLServer \
GRAPHQL_ENGINE=$(GRAPHQL_ENGINE_PRO_PATH) \
@ -203,7 +203,7 @@ test-native-queries-bigquery: remove-tix-file
## test-stored-procedures-sqlserver: run all sqlserver tests for the Stored Procedure feature
test-stored-procedures-sqlserver: remove-tix-file
cabal build exe:graphql-engine-pro
docker compose up --build --detach --wait postgres sqlserver-healthcheck
docker compose up --build --detach --wait postgres sqlserver
HASURA_TEST_BACKEND_TYPE=SQLServer \
HSPEC_MATCH=StoredProcedures \
GRAPHQL_ENGINE=$(GRAPHQL_ENGINE_PRO_PATH) \

44
server/lib/api-tests/README.md
View File

@ -311,43 +311,9 @@ brew install microsoft/mssql-release/mssql-tools@18
brew unlink mssql-tools18 && brew link mssql-tools18
```
### Microsoft SQL Server failures on Apple aarch64 chips
### The MS SQL Server container fails to start
This applies to all Apple hardware that uses aarch64 chips, e.g. the MacBook M1
or M2.
We have a few problems with Microsoft SQL Server on Apple aarch64:
1. Microsoft has not yet released SQL Server for aarch64. We need to use Azure
SQL Edge instead.
You don't need to do anything if you're using the `make` commands; they
will provide the correct image automatically.
If you run `docker compose` directly, make sure to set the environment
variable yourself:
```sh
export MSSQL_IMAGE='mcr.microsoft.com/azure-sql-edge'
```
You can add this to your _.envrc.local_ file if you like.
2. Azure SQL Edge for aarch64 does not ship with the `sqlcmd` utility with
which we use to setup the SQL Server schema.
If you need it, you can instead use the `mssql-tools` Docker image, for
example:
```
docker run --rm -it --platform=linux/amd64 --net=host mcr.microsoft.com/mssql-tools \
/opt/mssql-tools/bin/sqlcmd -S localhost,65003 -U SA -P <password>
```
To make this easier, you might want to define an alias:
```
alias sqlcmd='docker run --rm -it --platform=linux/amd64 --net=host mcr.microsoft.com/mssql-tools /opt/mssql-tools/bin/sqlcmd'
```
You can also install them directly with `brew install microsoft/mssql-release/mssql-tools`.
The SQL Server image will only work on macOS if you enable the relevant
settings. Open the Docker Desktop settings, enable "Use Virtualization
framework" in the "General" tab, and "Use Rosetta for x86/amd64 emulation on
Apple Silicon" in the "Features in development" tab.

8
server/lib/api-tests/docker-compose.yaml
View File

@ -52,14 +52,6 @@ services:
volumes:
- mssql-data:/var/opt/mssql
sqlserver-healthcheck:
extends:
file: ../../../docker-compose/databases.yaml
service: sqlserver-healthcheck
depends_on:
sqlserver:
condition: service_started
dc-reference-agent:
extends:
file: ../../../dc-agents/docker-compose.yaml

8
server/tests-py/docker-compose.yml
View File

@ -27,11 +27,3 @@ services:
extends:
file: ../../docker-compose/databases.yaml
service: sqlserver
sqlserver-healthcheck:
extends:
file: ../../docker-compose/databases.yaml
service: sqlserver-healthcheck
depends_on:
sqlserver:
condition: service_started

8
server/tests-py/run.sh
View File

@ -14,7 +14,7 @@ set -o pipefail
cd -- "$(dirname -- "${BASH_SOURCE[0]}")"
DATABASES=(postgres citus sqlserver sqlserver-healthcheck)
DATABASES=(postgres citus sqlserver)
(
cd ../..
@ -29,12 +29,6 @@ DATABASES=(postgres citus sqlserver sqlserver-healthcheck)
# shellcheck disable=SC1091
source .hasura-dev-python-venv/bin/activate
# Use the Azure SQL Edge image instead of the SQL Server image on arm64.
# The latter doesn't work yet.
if [[ "$(uname -m)" == 'arm64' ]]; then
export MSSQL_IMAGE='mcr.microsoft.com/azure-sql-edge'
fi
echo
echo '*** Starting databases ***'
docker compose up -d --wait "${DATABASES[@]}"