1
1
mirror of https://github.com/tweag/nickel.git synced 2024-10-05 23:57:09 +03:00

Restructure integration tests into a single crate (#954)

* Restructure integration tests into a single crate

This commit replaces our multi-crate integration test setup with a
single crate that contains all tests, inspired by [this
blog](https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html).

This has dual benefits:
1. it should reduce the amount of time tests take to compile, as we're
   only producing a single binary,
2. it should reduce the amount of time tests take to run, as cargo
   schedules individual test binaries to run sequentially whereas tests
   within a binary can run in parallel.

* Update test references in HACKING.md

* Add README explaining test structure
This commit is contained in:
matthew healy 2022-11-30 11:26:29 +01:00 committed by GitHub
parent c14b543a95
commit b0119f149e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
73 changed files with 44 additions and 10 deletions

View File

@ -117,22 +117,23 @@ LICENSE package.json nickel_lang_bg.js nickel_lang_bg.wasm [..]
Tests are run via `cargo test`. They are two types of tests:
- Unit tests, located directly in the corresponding module.
- Integration tests, located in the dedicated crate `tests`.
- Integration tests, located in the dedicated crate `tests/integration`.
You can take inspiration from the existing tests to add your own. By convention,
tests expected to pass are written in a standalone Nickel file in `tests/pass/`.
Each `.ncl` file defines a list of expressions that must individually evaluate
to the boolean `true`. The whole file is an expression that returns true if and
only if every tests pass, or fail with a contract failure to help locating the
failing test (instead of returning just `false`).
tests expected to pass are written in a standalone Nickel file in
`tests/integration/pass/`. Each `.ncl` file defines a list of expressions that
must individually evaluate to the boolean `true`. The whole file is an
expression that returns true if and only if every tests pass, or fail with a
contract failure to help locating the failing test (instead of returning just
`false`).
If a test expected to pass is failing, run it directly using nickel with `nickel
-f tests/pass/test_that_doesnt_pass.ncl` to get better error messages than
`cargo test`.
If a test expected to pass is failing, run it directly using nickel with
`nickel -f tests/integration/pass/test_that_doesnt_pass.ncl` to get better error
messages than `cargo test`.
Tests expected to fail are often embedded directly into rust source code,
because you usually want to additionally check that the error is the one you
expect. For example ([`tests/records_fail.rs`](./tests/records_fail.rs)):
expect. For example ([`tests/integration/records_fail.rs`](./tests/records_fail.rs)):
```rust
#[test]

16
tests/README.md Normal file
View File

@ -0,0 +1,16 @@
# A quick note on how tests are structured
By default, `cargo` encourages developers to put integration test files directly
in the `/tests` directory. Doing so means that a different test crate (and
associated binary) is built for each file.
There is both a compile-time and a runtime cost to doing this, as (1) multiple
different binaries must be built to run all tests and (2) while `cargo` can
run multiple tests in parallel, it runs multiple binaries sequentially.
By instead having a single crate which contains all tests (as we do in
`/tests/integration`), we save on compilation time, as well as being able to
fully parallelise all of our test runs.
See [this blog post](https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html)
for more information.

17
tests/integration/lib.rs Normal file
View File

@ -0,0 +1,17 @@
mod basics_fail;
mod contracts_fail;
mod destructuring;
mod examples;
mod free_vars;
mod imports;
mod infinite_rec;
mod merge_fail;
mod parse_fail;
mod pass;
mod pretty;
mod query;
mod records_fail;
mod stdlib_arrays_fails;
mod stdlib_typecheck;
mod typecheck_fail;
mod unbound_type_variables;