* Support version ranges in test tool exclusions
This cleans up the map of exclusions for the ledger-api-test-tool part
of the compat tests. For now, you can only use ranges for the inner
map. In principle we could also add it to the outer map but I’ll leave
that for a separate PR.
I’ve also run the script to update versions so this includes the
latest two snapshots.
fixes#5851
changelog_begin
changelog_end
* Address review comments
changelog_begin
changelog_end
* buildifier
changelog_begin
changelog_end
Previously we ran `daml codegen js` and `daml build` in the tests and
we also ran it twice. While this makes sense for the SDK integration
tests, it doesn’t really make sense for the compatibility tests. These
are SDK-only features so they don’t actually test compatibility. These
steps are super slow and running them in the tests also means that we
run them a quadratic number of times.
This PR moves the `daml build` and the `daml codegen js` step to
separate genrules. In the actual test, the only expensive steps are to
run `yarn install` and the actual jest tests. In principle, we could
probably try to factor out the `yarn install` step as well but I’m a
bit scared of coyping around node_modules so I’ll not attempt to do
that for now.
changelog_begin
changelog_end
* Include puppeteer tests in compat tests
This PR adds the puppeteer based tests to the compatibility
tests. This also means that they are now actually compatibility
tests. Before, we only tested the SDK side.
Apart from process management being a nightmare on Windows as usually,
there are two things that might stick out here:
1. I’ve replaced the `sh_binary` wrapper by a `cc_binary`. There is a
lengthy comment explaining why. I think at the moment, we could
actually get rid of the wraper completely and add JAVA to path in
the tests that need it but at least for now, I’d like to keep it
until we are sure that we don’t need to add more to it (and then
it’s also in the git history if we do need to resurrect it).
2. These tests are duplicated now similar to the `daml ledger *`
tests. The reasoning here is different. They depend on the SDK
tarball either way so performance wise there is no reason to keep
them. However, we reference the other file in the docs which means
we cannot change it freely. What we could do is to make this
sufficiently flexible to handle both the `daml start` case and
separate `daml sandbox`/`daml json-api` processes and then we can
reference it in the docs. There is still added complexity for
Windows but that’s necessary for users as well that want to run
this on Windows so that seems unavoidable. (I should probably also
remove my snarky comments 😇) I’d like to kee it duplicated
for this PR and then we can clean it up afterwards.
changelog_begin
changelog_end
* Bump timeouts
changelog_begin
changelog_end
The workspace for the vendored node wrapper script `@nodejs_linux_amd64`
did previously not record a dependency on the nixpkgs provided node
workspace. This patch enforces that dependency by introducing a dummy
read of the vendored node binary.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
For some reason this seems to be failing in particular on Linux but
I’ve now also seen it fail on MacOS. My suspicion is that running it
in parallel, somehow corrupts the cache but I don’t understand the details.
changelog_begin
changelog_end
* update rules_nixpkgs
* Use hermetic nixpkgs cc toolchain
CHANGELOG_BEGIN
CHANGELOG_END
* Work around Bazel's cc toolchain autodetection
* Use --crosstool_top for hermetic cc toolchain
When using --incompatible_enable_cc_toolchain_resolution instead
cc actions still depend on
`external/local_config_cc/builtin_include_directory_paths`
as well as
`external/nixpkgs_cc_toolchain_config/builtin_include_directory_paths`.
* override local_config_cc
* remove unused attribute
* Fix posix toolchain on Windows
* nixpkgs cc toolchain not on Windows
* Fix nixpkgs cc toolchain on MacOS
* nixpkgs cc toolchain uses bin/cc
* Use darwin.binutils on MacOS
* Remove clang(++) and gcc (g++) symlinks
The toolchain only considers `bin/cc` and having the other symlinks
around could lead to confusion
* Use hermetic toolchain in compatibility workspace
* Avoid empty linker flags
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Include the first feature in compat tests for create-daml-app
This adds the step from the create-daml-app tests that applies the
patch for the messaging feature and tests that things will build. This
is the last step before we can actually run the puppeteer tests which
will turn this into an actual compatibility test.
changelog_begin
changelog_end
* temporarily run all tests
changelog_begin
changelog_end
* Fix exclusions
changelog_begin
changelog_end
* Revert "temporarily run all tests"
This reverts commit 7425dd09cf48f2bfd6064b55d857c76d51afc821.
* Remove accidentally committed messaging.patch
changelog_begin
changelog_end
This is the first part of #5700
It adds tests that build create-daml-app using `daml build` and then
run the codegen and build the UI. Contrary to our main tests these
also run on Windows. This is actually reasonably simple by first
building the typescript libraries on Linux and then downloading them
on Windows.
There are two parts that are still missing from the tests in the main
workspace:
1. Building the extra feature. This should be fairly easy to add.
2. Running the pupeeter tests. At least MacOS and Linux should be
reasonably easy. I don’t know what horrors Windows will throw at
us. This step is what actually makes this a compatibility
test. Currently it doesn’t actually launch Sandbox and the JSON API.
Since this PR is already pretty large, I’d like to tackle those things
separately.
changelog_begin
changelog_end
While the tests are exclusive and I’ve kept them exclusive for
resource usage, the hardcoded port numbers can still cause issues if
for whatever reason you have a postgres instance running outside of
Bazel. This could happen because you simply have a local postgres
instance for testing other things or, what seems to have happened on
CI, due to a leftover postgres instance from a previous test.
While we should do something about the latter, this at least gets us
into a situation where the node isn’t completely broken at this point.
changelog_begin
changelog_end
This should fix the following issue that we see constantly on CI:
```
ld: warning: object file _ was built for newer OSX version (10.15) than being linked (10.14)
```
The issue was that the CC toolchain was not fully used in
haskell_cabal_package. --with-gcc (which is really --with-cc) only
applies when Cabal is calling the C compiler. However, in most cases
it is actually GHC itself which calls the C compiler. To make sure
that the right compiler is used in those cases, we have to pass
`-pgmc` and friends to GHC. This matches what rules_haskell does for
non cabal targets.
changelog_begin
changelog_end
* Implement version name shortening for Windows
CHANGELOG_BEGIN
CHANGELOG_END
* Temporarily disable `--quick`.
* Revert "Temporarily disable `--quick`."
This reverts commit 025e6c2ddd.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
This changes the code for starting postgres to print the log if
`pg_ctl start` fails.
In addition to that it marks the migration tests exclusive since they
rely on this code which has a hardcoded port number.
I tested this on CI by temporarily disabling --quick.
changelog_begin
changelog_end
* Make Bearer prefix optional in Haskell ledger bindings
This matches the change to the JVM bindings in 1.0.0
changelog_begin
- [DAML Assistant] The ``daml ledger`` commands no longer require the
``Bearer `` prefix in the access token file. This matches the behavior
of DAML Script and other SDK tools.
changelog_end
* Fix tasty crash
changelog_begin
changelog_end
* Update compatibility/bazel_tools/daml_ledger/Main.hs
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* Apply plotform_suffix on all Windows pipelines
To distinguish action keys between the compatibility and the main
workspace and avoid the "undeclared input(s)" error. We also modify the
main workspace's action cache keys to avoid poisoned cache items.
CHANGELOG_BEGIN
CHANGELOG_END
* Avoid exceeding MAX_PATH on Windows
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Update SDK versions in compatibility tests
This adds a Haskell script to generate a versions.bzl file that
contains the list of versions as well as their hashes. This should
make it a bit easier to keep things up2date going forward.
The script is a bit slow since downloading all the SDKs takes quite a
while but for now it should be good enough and is much more pleasant
than having to figure this out manually.
changelog_begin
changelog_end
* Address review comments
changelog_begin
changelog_end
* Fix excluded tests
changelog_begin
changelog_end
* Add basic Sandbox data continuity tests
This adds some basic tests that check that data migrations work
properly. For now, I use DAML Script to create and query contracts at
each step. This isn’t perfect since queries can only use the active
contract service but not things like the transaction stream but it’s
clearly better than nothing.
The runner for executing the tests is a simple Haskell executable. It
didn’t really seem useful to throw tasty at this.
I’ve added two sets of tests, one that runs only through stable
versions and one that includes snapshots since migrating through
snapshots is not necessarily equivalent.
Sadly these tests use sandbox-classic since I discovered while writing
these tests that sandbox-next does not actually support migrating data
between SDK versions.
changelog_begin
changelog_end
* Use the sandbox module instead of a custom withSandbox
changelog_begin
changelog_end
* Update compatibility/sandbox-migration/SandboxMigrationRunner.hs
Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
To avoid "undeclared inclusion(s)" errors by modifying the action keys.
Another option is `--action_env`. However, this only affects actions
that set `use_default_shell_env=True`, which is few, since that setting
is bad for hermeticity.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* daml new script-example
* Build DAML script DAR
* daml_script_dar macro over sdk_version
* Run an individual daml-script test
* DAML script test matrix
* format
CHANGELOG_BEGIN
CHANGELOG_END
* Use named arguments on daml_script_test
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Set JVM memory limits for sandbox
Using the same settings as defined in `@daml//bazel_tools:scala.bzl`.
* Mark tests as large
Their memory consumption is somewhere around 300MiB which is considered
"large" according to
https://docs.bazel.build/versions/master/be/common-definitions.html?cl=head#common-attributes-tests.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Make compat tests work on windows
This required some changes to the daml_sdk rule since the read-only
installation by the assistant breaks Bazel completely. We could only
apply those changes on Windows but I think I prefer the consistency
across platforms here over trying to stay close to how the SDK is
installed on user machines given that the SDK installation is not
something we’ve had issues with.
I’ve excluded the postgresql tests for now. I don’t expect them to be
particularly hard to fix but I’ve already spent almost 2 days on this
and having some tests run on Windows seems like a clear improvement
over running no tests on Windows :)
changelog_begin
changelog_end
* Remove todo
changelog_begin
changelog_end
* Copy daml ledger test to compatibility
Specifically:
- `daml-assistant/daml-helper/test/DA/Daml/Helper/Test/Deployment.hs`
- `libs-haskell/test-utils/DA/Test/Sandbox.hs`
* Adapt daml ledger test for compatibility
Avoid dependencies on the daml repository and make the SDK and platform
components configurable.
* Add test certificates to compatibility
Taken from `@daml//ledger/test-common/test-certificates`.
* Define daml_ledger_test
CHANGELOG_BEGIN
CHANGELOG_END
* Add note on keeping daml ledger tests in sync
* ./fmt.sh
* Strictly require --sdk-version flag
Tasty doesn't allow options without a default value, so we default to
`error` to enforce a value being set.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
This PR extends the existing Linux compatibility tests to run on macOS
too. Fixes#5692.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Fix ./fmt.sh check
We accidentally introduced hlint lints when we added the compatibility
workspace. The hlint code in ./fmt.sh correctly detects that there is
a hint but then exits with 0 since it uses the exit code from `echo`
rather than the one from `hlint`. The info that we printed with
`echo` didn’t actually make any sense so I added a new comment as to
why we run hlint here. This requires some setup changes so that the
compatibility workspace gets the same Haskell flags since otherwise
using the same hlint rules obviously does not make sense.
Fixes#5701
changelog_begin
changelog_end
* Update fmt.sh
Co-Authored-By: Samir Talwar <samir.talwar@digitalasset.com>
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Extend compatibility tests to DAML on SQL
This feels a bit hacky since the runfiles don’t work quite like I
would expect them to but it’s at least not more hacky than what we do
for the head-based tests we currently have.
Progress towards #5695
changelog_begin
changelog_end
* Fix runfiles with more bash
changelog_begin
changelog_end
* remove redundant port options
changelog_begin
changelog_end
* Create fewer sandbox targets
changelog_begin
changelog_end
* Apply suggestions from code review
Co-Authored-By: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
* Fix runfiles snippet
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
* Abstract over SDK/platform client_server_test
* Define test-matrix using simple list comprehension
CHANGELOG_BEGIN
CHANGELOG_END
* copy right header
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
This is a first step towards testing cross-version
compatibility. It doesn’t actuall do much yet but hopefully it should
be easier to parallelize once we have the initial boilerplate in place
so ideally I’d like to address most missing things and issues in
separate PRs.
changelog_begin
changelog_end
