# Contributing This guide explains how to set up the graphql-engine server for development on your own machine and how to contribute. ## Pre-requisites - [GHC](https://www.haskell.org/ghc/) 9.2.5 and [cabal-install](https://cabal.readthedocs.io/en/latest/) - There are various ways these can be installed, but [ghcup](https://www.haskell.org/ghcup/) is a good choice if you’re not sure. - There are few system packages required like `libpq-dev`, `libssl-dev`, etc. The best place to get the entire list is from the [Dockerfile](../packaging/graphql-engine-base/ubuntu.dockerfile) - Additional Haskell tools (expected versions can be found in _VERSIONS.json_): - [HLint](https://github.com/ndmitchell/hlint), for linting Haskell code - [hpack](https://github.com/sol/hpack), for generating Cabal files - [Ormolu](https://github.com/tweag/ormolu), for formatting Haskell code - [Docker](https://www.docker.com/get-started/) - [Docker Compose](https://docs.docker.com/compose/install/) For running the test suite: - [node.js](https://nodejs.org/en/) (see [.nvmrc](../.nvmrc) for the version), and the bundled NPM version - Python >= 3.9 with pip3 and virtualenv For building the Console: - node.js, as above Additionally, you will need a way to run a PostgreSQL database server. The `dev.sh` script (described below) can set up a PostgreSQL instance for you via [Docker](https://www.docker.com), but if you want to run it yourself, you’ll need: - [PostgreSQL](https://www.postgresql.org) >= 10 - [postgis](https://postgis.net) ### Installing tooling with Nix Simply [install Nix](https://nixos.org/download.html) and type `nix develop`. If you don't want to start a new shell each time, you can also use [direnv](https://direnv.net/) and [nix-direnv](https://github.com/nix-community/nix-direnv), then create a _.envrc.local_ file with the contents: ```bash #!/usr/bin/env bash use flake ``` ### Installing tooling with direnv This project contains scripts for installing project dependencies automatically with [direnv](https://direnv.net/). For more information, see the `.envrc` file in the root. ## Development workflow You should fork the repo on github and then `git clone https://github.com/<your-username>/graphql-engine`. After making your changes ### Compile ...console assets: $ cd frontend $ nvm use $ npm ci $ npm run server-build:ce $ cd .. ...and the server: $ ln -s cabal/dev.project cabal.project.local $ cabal new-update $ cabal new-build graphql-engine To set up the project configuration to coincide with the testing scripts below, thus avoiding recompilation when testing locally, rather use `cabal/dev-sh.project.local` instead of `cabal/dev.project`: $ ln -s cabal/dev-sh.project.local cabal.project.local #### Compiling on MacOS If you are on MacOS, or experiencing any errors related to missing dependencies on MacOS, please try [this alternative setup guide](COMPILING-ON-MACOS.md), or try Nix (as above). ### IDE Support You may want to use [hls](https://github.com/haskell/haskell-language-server)/[ghcide](https://github.com/haskell/ghcide) if your editor has LSP support. A sample configuration has been provided which can be used as follows: ``` ln -s sample.hie.yaml hie.yaml ``` If you have to customise any of the options for ghcide/hls, you should instead copy the sample file and make necessary changes in `hie.yaml` file. Note that `hie.yaml` is gitignored so the changes will be specific to your machine. ``` cp sample.hie.yaml hie.yaml ``` ### Run and test via `run-new.sh` The `run-new.sh` scripts are an active work in progress, and will eventually replace the `dev.sh` option below. Run the Python integration tests with `./server/tests-py/run-new.sh`. Filter on specific test files with `./server/tests-py/run-new.sh -- create_async_action_with_nested_output_and_relation.py` If you have any issues with `run-new.sh`, please create a [GitHub issue](https://github.com/hasura/graphql-engine/issues/new/choose) and run and test via `dev.sh` instead. ### Run and test via `dev.sh` The `dev.sh` script in the top-level `scripts/` directory is a turnkey solution to build, run, and test `graphql-engine` using a Docker container to run a Postgres database. **Docker is necessary to use `dev.sh`.** To use `dev.sh`, first launch a new postgres container with: $ scripts/dev.sh postgres Then in a new terminal launch `graphql-engine` in dev mode with: $ scripts/dev.sh graphql-engine This command also starts the GraphQL Engine console, which you can access at http://localhost:8181/console. The `dev.sh` will print some helpful information and logs from both services will be printed to screen. You can run the test suite with: $ scripts/dev.sh test This should run in isolation. The output format is described in the [pytest documentation](https://docs.pytest.org/en/latest/usage.html#detailed-summary-report). Errors and failures are indicated by `F`s and `E`s. Optionally, launch a new container for alternative (MSSQL) backend with: $ scripts/dev.sh mssql Tests can be run against a specific backend (defaulting to Postgres) with the `backend` flag, for example: $ scripts/dev.sh test --integration -k TestGraphQLQueryBasicCommon --backend (bigquery|citus|mssql|postgres) ### Run and test manually If you want, you can also run the server and test suite manually against an instance of your choosing. #### Run The following command can be used to build and launch a local `graphql-engine` instance: ``` $ cabal new-run -- exe:graphql-engine \ --database-url='postgres://<user>:<password>@<host>:<port>/<dbname>' \ serve --enable-console --console-assets-dir=frontend/dist/apps/server-assets-console-ce ``` This will launch a server on port 8080, and it will serve the console assets if they were built with `npm run server-build:ce` as mentioned above. #### Test `graphql-engine` has several test suites, among them: 1. A small set of unit tests and integration tests written in Haskell, in `server/src-test`. 2. A new integration test suite written in Haskell, in `server/lib/api-tests`. 3. An extensive set of end-to-end tests written in Python, in `server/tests-py`. All sets of tests require running databases: - some unit tests hit the database, and running the unit test suite requires passing in a postgres connection string, - the Haskell integration test suite requires databases to run (they can be started via the docker command listed below), - the Python integration test suite also requires databases AND the engine to be running, which can be started via either the `dev.sh` script, or manually. ##### Running py tests The easiest way to run the Python integration test suite is by running: ```sh $ scripts/dev.sh test --integration ``` For more details please check out the [README](./tests-py/README.md). ##### Running the Haskell test suite There are three categories of unit tests: - unit tests - PostgreSQL integration tests (requires a PostgreSQL instance) - MS SQL Server integration tests (requires a MS SQL Server instance) The easiest way to run these tests is through `make`, which will automatically spin up and shut down Docker containers for the databases: ``` $ make test-unit $ make test-integration-postgres $ make test-integration-mssql ``` If you want to limit to a specific set of tests, use `HSPEC_MATCH`: ``` $ make test-unit HSPEC_MATCH='Memoize' ``` Alternatively, you can use Cabal directly (though you'll have to start the databases yourself): ``` $ cabal run -- graphql-engine:test:graphql-engine-tests $ HASURA_GRAPHQL_DATABASE_URL='postgres://<user>:<password>@<host>:<port>/<dbname>' \ cabal run -- graphql-engine:test:graphql-engine-test-postgres ``` ##### Running the Haskell integration test suite Run `make test-backends`. This effectively runs the following two commands: ``` $ docker compose up --detach --wait $ cabal run api-tests:exe:api-tests ``` For more details please check out the [README](./lib/api-tests/README.md). ##### Running unit tests and recompiling While working on features, you might want to add unit tests and work through getting them to pass. This is generally a slow process, but there is a workaround to allow loading both the `graphql-engine` library and the unit testing library in `ghcid` at the same time: ```sh $ ghcid -a -c "cabal repl graphql-engine-tests -f -O0 -fghci-load-test-with-lib" --test Main.main ``` This assumes you already have `HASURA_GRAPHQL_DATABASE_URL` and `HASURA_MSSQL_CONN_STR` exported as environment variables. If you just want to run all unit tests, you can add ` --setup ":set args unit"` to the command line above. If you want to run specific test(s), you can instead do `--setup ":set args unit --match name_of_test(s)"`. #### Building with profiling To build with profiling support, you need to both enable profiling via `cabal` and set the `profiling` flag. E.g. ``` $ cabal build exe:graphql-engine -f profiling --enable-profiling ``` ### Create Pull Request - Make sure your commit messages meet the [guidelines](../CONTRIBUTING.md). - If you changed the versions of any dependencies, run `scripts/cabal-freeze-update.sh --all` to update the freeze file. - Create a pull request from your forked repo to the main repo. - Every pull request will automatically build and run the tests. ## Code conventions The following conventions help us maintain a uniform style for all committers: make sure your contributions are in line with them. We enforce these by means of CI hooks which will fail the build if any of these are not met. - No compiler warnings: Make sure your code builds with no warnings (adding `-Werror` to `ghc-options` in your `cabal.project` is a good way of checking this.) - No lint failures: Use [hlint](https://github.com/ndmitchell/hlint) with our custom config to validate your code, using `hlint --hint=../.hlint.yaml`. - Consistent formatting: Use [ormolu](https://github.com/tweag/ormolu) to format your code. `ormolu -ei '*.hs'` will format all files with a `.hs` extension in the current directory. - Consistent style: Consider the [style guide](./STYLE.md) when writing new code. ## Testing Please see [testing-guidelines](./testing-guidelines.md) for details on how to add tests. ## Local hoogle instance [Hoogle](https://github.com/ndmitchell/hoogle) is a Haskell API search engine. The server at [hoogle.haskell.org](https://hoogle.haskell.org/) provides a version of Hoogle that enables searching through all packanges available in [Stackage](https://www.stackage.org/). Following instructions help in setting up a local hoogle server that enables searching through `graphql-engine` server code. ### Step 1: Installing hoogle Installing `hoogle` is fairly simple with `cabal`. ```bash $ cabal install hoogle ``` ### Step 2: Generating hoogle database A Hoogle database is a prebuilt index of a set of packages. The `hoogle.sh` script in the top-level `scripts/` directory helps in generating the hoogle database for GraphQL Engine server code and store it in `dist-newstyle/` directory. $ scripts/hoogle.sh generate ### Step 3: Running hoogle instance Running the following `hoogle.sh` script command starts a local hoogle server with the database generated in `Step 2`. $ scripts/hoogle.sh serve Use `--port` option to specify custom port to start hoogle server. $ scripts/hoogle.sh serve --port 8181