mirror of
https://github.com/tweag/nickel.git
synced 2024-10-05 23:57:09 +03:00
Restructure integration tests into a single crate (#954)
* Restructure integration tests into a single crate This commit replaces our multi-crate integration test setup with a single crate that contains all tests, inspired by [this blog](https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html). This has dual benefits: 1. it should reduce the amount of time tests take to compile, as we're only producing a single binary, 2. it should reduce the amount of time tests take to run, as cargo schedules individual test binaries to run sequentially whereas tests within a binary can run in parallel. * Update test references in HACKING.md * Add README explaining test structure
This commit is contained in:
parent
c14b543a95
commit
b0119f149e
21
HACKING.md
21
HACKING.md
@ -117,22 +117,23 @@ LICENSE package.json nickel_lang_bg.js nickel_lang_bg.wasm [..]
|
||||
Tests are run via `cargo test`. They are two types of tests:
|
||||
|
||||
- Unit tests, located directly in the corresponding module.
|
||||
- Integration tests, located in the dedicated crate `tests`.
|
||||
- Integration tests, located in the dedicated crate `tests/integration`.
|
||||
|
||||
You can take inspiration from the existing tests to add your own. By convention,
|
||||
tests expected to pass are written in a standalone Nickel file in `tests/pass/`.
|
||||
Each `.ncl` file defines a list of expressions that must individually evaluate
|
||||
to the boolean `true`. The whole file is an expression that returns true if and
|
||||
only if every tests pass, or fail with a contract failure to help locating the
|
||||
failing test (instead of returning just `false`).
|
||||
tests expected to pass are written in a standalone Nickel file in
|
||||
`tests/integration/pass/`. Each `.ncl` file defines a list of expressions that
|
||||
must individually evaluate to the boolean `true`. The whole file is an
|
||||
expression that returns true if and only if every tests pass, or fail with a
|
||||
contract failure to help locating the failing test (instead of returning just
|
||||
`false`).
|
||||
|
||||
If a test expected to pass is failing, run it directly using nickel with `nickel
|
||||
-f tests/pass/test_that_doesnt_pass.ncl` to get better error messages than
|
||||
`cargo test`.
|
||||
If a test expected to pass is failing, run it directly using nickel with
|
||||
`nickel -f tests/integration/pass/test_that_doesnt_pass.ncl` to get better error
|
||||
messages than `cargo test`.
|
||||
|
||||
Tests expected to fail are often embedded directly into rust source code,
|
||||
because you usually want to additionally check that the error is the one you
|
||||
expect. For example ([`tests/records_fail.rs`](./tests/records_fail.rs)):
|
||||
expect. For example ([`tests/integration/records_fail.rs`](./tests/records_fail.rs)):
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
|
16
tests/README.md
Normal file
16
tests/README.md
Normal file
@ -0,0 +1,16 @@
|
||||
# A quick note on how tests are structured
|
||||
|
||||
By default, `cargo` encourages developers to put integration test files directly
|
||||
in the `/tests` directory. Doing so means that a different test crate (and
|
||||
associated binary) is built for each file.
|
||||
|
||||
There is both a compile-time and a runtime cost to doing this, as (1) multiple
|
||||
different binaries must be built to run all tests and (2) while `cargo` can
|
||||
run multiple tests in parallel, it runs multiple binaries sequentially.
|
||||
|
||||
By instead having a single crate which contains all tests (as we do in
|
||||
`/tests/integration`), we save on compilation time, as well as being able to
|
||||
fully parallelise all of our test runs.
|
||||
|
||||
See [this blog post](https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html)
|
||||
for more information.
|
17
tests/integration/lib.rs
Normal file
17
tests/integration/lib.rs
Normal file
@ -0,0 +1,17 @@
|
||||
mod basics_fail;
|
||||
mod contracts_fail;
|
||||
mod destructuring;
|
||||
mod examples;
|
||||
mod free_vars;
|
||||
mod imports;
|
||||
mod infinite_rec;
|
||||
mod merge_fail;
|
||||
mod parse_fail;
|
||||
mod pass;
|
||||
mod pretty;
|
||||
mod query;
|
||||
mod records_fail;
|
||||
mod stdlib_arrays_fails;
|
||||
mod stdlib_typecheck;
|
||||
mod typecheck_fail;
|
||||
mod unbound_type_variables;
|
Loading…
Reference in New Issue
Block a user