36fe0abb09
* Remove ledger api test tool * Force full compat run-full-compat: true * Force run all tests run-all-tests: true --------- Co-authored-by: Paul Brauner <paul.brauner@digitalasset.com> |
||
|---|---|---|
| .. | ||
| bazel_tools | ||
| head_sdk | ||
| versions | ||
| .bazelignore | ||
| .bazelrc | ||
| bazel-haskell-deps.bzl | ||
| BUILD | ||
| build-release-artifacts-windows.ps1 | ||
| build-release-artifacts.sh | ||
| build-ts-libs.sh | ||
| deps.bzl | ||
| maven_install.json | ||
| new_cache | ||
| old_cache | ||
| README.md | ||
| releases-github-api.json | ||
| stack-snapshot.yaml | ||
| stackage_snapshot_windows.json | ||
| stackage_snapshot.json | ||
| test-all-installs.sh | ||
| test-daml-version-and-install.sh | ||
| test-windows.ps1 | ||
| test.sh | ||
| update-versions.sh | ||
| versions.bzl | ||
| WORKSPACE | ||
This directory contains the infrastructure and test setup for cross-version compatibility testing. We make this a separate Bazel workspace to make it easier to enforce that we only depend on release artifacts.
How to
Before using this, make sure to always run the scripts that build the
SDK, namely ./build-release-artifacts.sh and ./build-ts-libs.sh.
These should be run every time there is a relevant change you want
to be there for this workspace to use.
Any Bazel command can then be used afterwards. ./test.sh are the
tests run on CI.
What is currently being tested?
We include the following SDK versions in our tests:
- All stable SDK versions.
- All snapshots
x.y.z-snapshot-*for which there is no stable releasex.y.z - HEAD.
Since running all tests can be rather slow, we run them in a daily cron job. On each PR we only include HEAD and the latest stable release.
Data-continuity for Sandbox classic
We have migration tests that work as follows:
-
Start the oldest Sandbox version and upload a DAR.
-
Iterate over SDK versions in order. For each version:
-
Start sandbox of that SDK version.
-
Run a custom Scala binary that interacts with the ledger to create contracts, exercise choices and query the ACS and the transaction service.
-
Validates the results. This includes verifying that the transaction streams are the same after the migration and that the ACS is the same + additional test-specific checks.
-
We have two tests here: One that includes snapshot versions and one that only iterates through stable versions up to HEAD. This ensures that both individual migrations are correct as well as the migrations from one stable version to the next work as a whole.
Backwards-compatibility for Daml Script
We test that the Daml Script runner from a given SDK version can load DARs built against the Daml Script library from an older SDK. We only guarantee backwards compatibility here.
Backwards-compatibility for Daml Triggers
We test that the Daml Trigger runner from a given SDK version can load DARs built against the Daml Script library from an older SDK. We only guarantee backwards compatibility here.
Backwards-compatibility for data-dependencies
We test that we can import DARs built in older SDK versions via
data-dependencies. We only test backwards compatibility here.
Cross-version compatibility for create-daml-app
We test that we can run the create-daml-app tests with JS client libraries and codegen from one version against the JSON API and Sandbox from another version. We test all version combinations here. Currently we do not test different versions of the JSON API and Sandbox.