graphql-engine/server/CONTRIBUTING.md
Samir Talwar 0e4d709285 Factor tool versions out into server/VERSIONS.json.
This helps us use the same versions locally as in CI. If you're using the Nix setup, it guarantees it.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5111
GitOrigin-RevId: 6e00cd7a78593df1e60fac37cc1195aba60e488f
2022-07-22 12:23:46 +00:00

293 lines
10 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Contributing
This guide explains how to set up the graphql-engine server for development on your
own machine and how to contribute.
## Pre-requisites
- [GHC](https://www.haskell.org/ghc/) 8.10.7 and [cabal-install](https://cabal.readthedocs.io/en/latest/)
- There are various ways these can be installed, but [ghcup](https://www.haskell.org/ghcup/) is a good choice if youre not sure.
- There are few system packages required like `libpq-dev`, `libssl-dev`, etc. The best place to get the entire list is from the packager [Dockerfile](../.buildkite/dockerfiles/ci-builders/server-builder.dockerfile)
- Additional Haskell tools (expected versions can be found in _VERSIONS.json_):
- [HLint](https://github.com/ndmitchell/hlint), for linting Haskell code
- [Ormolu](https://github.com/tweag/ormolu), for formatting Haskell code
For building console and running test suite:
- [Node.js](https://nodejs.org/en/) (v12+, it is recommended that you use `node` with version `v12.x.x` A.K.A `erbium` or version `14.x.x` A.K.A `Fermium`)
- npm >= 5.7
- python >= 3.5 with pip3 and virtualenv
Additionally, you will need a way to run a Postgres database server. The `dev.sh` script (described below) can set up a Postgres instance for you via [Docker](https://www.docker.com), but if you want to run it yourself, youll need:
- [PostgreSQL](https://www.postgresql.org) >= 9.5
- [postgis](https://postgis.net)
### Installing tooling with direnv
This project contains scripts for installing project dependencies automatically with [direnv](https://direnv.net/). For more information, see the `.envrc` file in the root.
### Upgrading npm
If your npm is too old (>= 5.7 required):
$ npm install -g npm@latest # sudo may be required
or update your nodejs.
## Development workflow
You should fork the repo on github and then `git clone https://github.com/<your-username>/graphql-engine`.
After making your changes
### Compile
...console assets:
$ cd console
$ npm ci
$ npm run server-build
$ cd ..
...and the server:
$ ln -s cabal/dev.project cabal.project.local
$ cabal new-update
$ cabal new-build graphql-engine
To set up the project configuration to coincide with the testing scripts below, thus avoiding recompilation when testing locally, rather use `cabal/dev-sh.project.local` instead of `cabal/dev.project`:
$ ln -s cabal/dev-sh.project.local cabal.project.local
#### Compiling on MacOS
If you are on MacOS, or experiencing any errors related to missing dependencies on MacOS, please try [this alternative setup guide](COMPILING-ON-MACOS.md).
### IDE Support
You may want to use [hls](https://github.com/haskell/haskell-language-server)/[ghcide](https://github.com/haskell/ghcide) if your editor has LSP support. A sample configuration has been provided which can be used as follows:
```
ln -s sample.hie.yaml hie.yaml
```
If you have to customise any of the options for ghcide/hls, you should instead copy the sample file and make necessary changes in `hie.yaml` file. Note that `hie.yaml` is gitignored so the changes will be specific to your machine.
```
cp sample.hie.yaml hie.yaml
```
### Run and test via `dev.sh`
The `dev.sh` script in the top-level `scripts/` directory is a turnkey solution to build, run, and
test `graphql-engine` using a Docker container to run a Postgres database. **Docker is necessary to
use `dev.sh`.**
To use `dev.sh`, first launch a new postgres container with:
$ scripts/dev.sh postgres
Then in a new terminal launch `graphql-engine` in dev mode with:
$ scripts/dev.sh graphql-engine
This command also starts the GraphQL Engine console, which you can access at http://localhost:8181/console.
The `dev.sh` will print some helpful information and logs from both services
will be printed to screen.
You can run the test suite with:
$ scripts/dev.sh test
This should run in isolation. The output format is described in the [pytest documentation](https://docs.pytest.org/en/latest/usage.html#detailed-summary-report). Errors and failures are indicated by `F`s and `E`s.
Optionally, launch a new container for alternative (MSSQL) backend with:
$ scripts/dev.sh mssql
Tests can be run against a specific backend (defaulting to Postgres) with the `backend` flag, for example:
$ scripts/dev.sh test --integration -k TestGraphQLQueryBasicCommon --backend (bigquery|citus|mssql|postgres)
### Run and test manually
If you want, you can also run the server and test suite manually against an instance of your choosing.
#### Run
The following command can be used to build and launch a local `graphql-engine` instance:
```
cabal new-run -- exe:graphql-engine \
--database-url='postgres://<user>:<password>@<host>:<port>/<dbname>' \
serve --enable-console --console-assets-dir=console/static/dist
```
This will launch a server on port 8080, and it will serve the console assets if they were built with `npm run server-build` as mentioned above.
#### Test
`graphql-engine` has several test suites, among them:
1. A small set of unit tests and integration tests written in Haskell, in `server/src-test`.
2. A new integration test suite written in Haskell, in `server/tests-hspec`.
3. An extensive set of end-to-end tests written in Python, in `server/tests-py`.
All sets of tests require running databases:
- some unit tests hit the database, and running the unit test suite requires passing in a postgres connection string,
- the Haskell integration test suite requires databases to run (they can be started via the docker command listed below),
- the Python integration test suite also requires databases AND the engine to be running, which can be started via either the `dev.sh` script, or manually.
##### Running py tests
The easiest way to run the Python integration test suite is by running:
```sh
scripts/dev.sh test --integration
```
For more details please check out the [README](./tests-py/README.md).
##### Running the Haskell test suite
There are three categories of unit tests:
- true unit tests
- Postgres unit tests (require a postgres instance)
- MSSQL unit tests (require a MSSQL instance)
The easiest way to run these tests is through `dev.sh`:
```
./scripts/dev.sh test --unit
```
If you want to limit to a specific set of tests:
```
./scripts/dev.sh test --unit --match "some pattern" mssql
```
Note that you have to use one of 'unit', 'postgres' or 'mssql' when
using '--match'. There is no way to match without specifying the subset
of tests to run.
Alternatively, you can run unit tests directly through cabal:
```
cabal new-run -- test:graphql-engine-tests unit
HASURA_GRAPHQL_DATABASE_URL='postgres://<user>:<password>@<host>:<port>/<dbname>' \
cabal new-run -- test:graphql-engine-tests postgres
```
##### Running the Haskell integration test suite
1. To run the Haskell integration test suite, you'll first need to bring up the database containers:
```sh
docker-compose up
```
2. Once the containers are up, you can run the test suite via
```sh
cabal test tests-hspec --test-show-details=direct
```
For more details please check out the [README](./tests-hspec/README.md).
##### Running unit tests and recompiling
While working on features, you might want to add unit tests and work through
getting them to pass. This is generally a slow process, but there is a
workaround to allow loading both the `graphql-engine` library and the unit
testing library in `ghcid` at the same time:
```sh
ghcid -a -c "cabal repl graphql-engine-tests -f -O0 -fghci-load-test-with-lib" --test Main.main
```
This assumes you already have `HASURA_GRAPHQL_DATABASE_URL` and `HASURA_MSSQL_CONN_STR`
exported as environment variables.
If you just want to run all unit tests, you can add ` --setup ":set args unit"`
to the command line above. If you want to run specific test(s), you can instead
do `--setup ":set args unit --match name_of_test(s)"`.
#### Building with profiling
To build with profiling support, you need to both enable profiling via `cabal`
and set the `profiling` flag. E.g.
```
cabal build exe:graphql-engine -f profiling --enable-profiling
```
### Create Pull Request
- Make sure your commit messages meet the [guidelines](../CONTRIBUTING.md).
- If you changed the versions of any dependencies, run
`scripts/cabal-freeze-update.sh --all` to update the freeze file.
- Create a pull request from your forked repo to the main repo.
- Every pull request will automatically build and run the tests.
## Code conventions
The following conventions help us maintain a uniform style for all committers:
make sure your contributions are in line with them.
We enforce these by means of CI hooks which will fail the build if any of these
are not met.
- No compiler warnings: Make sure your code builds with no warnings (adding
`-Werror` to `ghc-options` in your `cabal.project` is a good way of checking
this.)
- No lint failures: Use [hlint](https://github.com/ndmitchell/hlint) with our
custom config to validate your code, using `hlint --hint=../.hlint.yaml`.
- Consistent formatting: Use [ormolu](https://github.com/tweag/ormolu) to
format your code. `ormolu -ei '*.hs'` will format all files with a `.hs`
extension in the current directory.
- Consistent style: Consider the [style guide](./STYLE.md) when writing new code.
## Testing
Please see [testing-guidelines](./testing-guidelines.md) for details on how to add tests.
## Local hoogle instance
[Hoogle](https://github.com/ndmitchell/hoogle) is a Haskell API search engine. The server at
[hoogle.haskell.org](https://hoogle.haskell.org/) provides a version of Hoogle that enables
searching through all packanges available in [Stackage](https://www.stackage.org/). Following
instructions help in setting up a local hoogle server that enables searching through `graphql-engine` server code.
### Step 1: Installing hoogle
Installing `hoogle` is fairly simple with `cabal`.
```bash
cabal install hoogle
```
### Step 2: Generating hoogle database
A Hoogle database is a prebuilt index of a set of packages. The `hoogle.sh` script in the
top-level `scripts/` directory helps in generating the hoogle database for GraphQL Engine server
code and store it in `dist-newstyle/` directory.
$ scripts/hoogle.sh generate
### Step 3: Running hoogle instance
Running the following `hoogle.sh` script command starts a local hoogle server with the database
generated in `Step 2`.
$ scripts/hoogle.sh serve
Use `--port` option to specify custom port to start hoogle server.
$ scripts/hoogle.sh serve --port 8181