* Removed engine constrcutor parameter.
* Basic abstractions for pre-executing submission validator.
* Fix build.
* Shared byteToLogEntryId with BatchedSubmissionValidator.
* Added some more metrics.
* Removed involved participants from PreExecutionResult.
CHANGELOG_BEGIN
CHANGELOG_END
* Reformatted/added headers.
* Fix build.
* Code tidying.
* Moved PreExecutionOutput into a separate file.
* Moved type definition into companion object.
* Added some tests for PreExecutingSubmissionValidator.
* Update ledger/participant-state/kvutils/src/main/scala/com/daml/ledger/validator/preexecution/LedgerStateReaderWithFingerprints.scala
Co-authored-by: Simon Meier <meiersi-da@users.noreply.github.com>
* Removed left-in code.
* Added some assertions for .zip operations.
* Test result for unexpected message.
* Test setting min/max record time.
* Reuse code for generating a key-value pair for log entries.
* Tests for CachingDamlLedgerStateReaderWithFingerprints.
* Store empty values as well with their fingerprints.
* Moved weight function into companion object.
* Test that the results come back in right order.
* Code tidying.
* Basic test case for LogAppenderPreExecutingCommitStrategy. Added missing headers.
* Code tidying.
* Fix build.
* Fix build.
* Revert "Store empty values as well with their fingerprints."
This reverts commit ca68ea967f.
# Conflicts:
# ledger/participant-state/kvutils/src/test/suite/scala/com/daml/ledger/validator/caching/CachingDamlLedgerStateReaderWithFingerprintsSpec.scala
* Added some docs.
* Added test case for ensuring that we don't cache None values.
* Fix build.
Co-authored-by: Simon Meier <meiersi-da@users.noreply.github.com>
This reverts commit 61e9df3eaf.
This interacts very badly with the fact that we check out old commits
for releases. While we could fix it for this particular issue, I don’t
think this buys us enough to make this worth doing and it makes it
easy to introduce issues in the future if we modify lib.sh
changelog_begin
changelog_end
I accidentally broke this when I added the types (very sorry about
that). Unfortunately while the tests should actually have caught this,
they didn’t because they are also broken. The `&&` results in bash
assuming the exit code is handled and `set -e` doesn’t kick in so
despite an error things proceeded just fine.
The change turns
```
Add2: (<a_a1Ag>(a_a1Ag: damlTypes.Serializable<a_a1Ag>) => damlTypes.Serializable<Expr2.Add2>);
```
into
```
Add2: (<a_a1Ag>(a_a1Ag: damlTypes.Serializable<a_a1Ag>) => damlTypes.Serializable<Expr2.Add2<a_a1Ag>>);
```
which is obviuosly what we want and also fixes the test. I did verify
that the tests fail afterwards without the change to daml2js.
changelog_begin
changelog_end
* adding a test case for the offset order returning expected transactions,
moving the exiting test case under flat transactions behavior group
* Fixing event_offset to event_sequential_id conversion
handling the conversion on an empty ledger,
adding a bit more logging,
skipping DB query on an empty range
CHANGELOG_BEGIN
CHANGELOG_END
* addressing code review comments
simplifying the query
returning SQL that forces the event_offset index usage
* addressing code review comments
simplifying the query
returning SQL that forces the event_offset index usage
* simplifying it a bit more
CHANGELOG_BEGIN
Default new Engine constructor to Engine.StableConfig so it does not
need to be overridden unless you specifically want to run in
Engine.DevConfig mode
CHANGELOG_END
Signed-off-by: Brian Healey <brian.healey@digitalasset.com>
* DAML REPL - Explicit package imports
changelog_begin
- [DAML REPL] The REPL no longer imports all modules from the main DALFs
of all specified DARs automatically at start-up. Instead, the REPL
will only import modules from packages specified on the command-line
using the `--import` flag.
changelog_end
* Accept package-name or package-id
* REPL test case for --import flag
* DAML REPL use `UnitId` for import packages
Addressing review comment
https://github.com/digital-asset/daml/pull/6707#discussion_r453731353
* DAML REPL Parse package-name/id at CLI
* DAML REPL Simplify unversioned pkgs
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Turns out you can easily intersect function types with object literals
to get what we want here.
While I am very annoyed by it, I don’t know how to write a test for
this. Afaik I can only test this by writing something that hits the
decoder directly but that doesn’t work since it’s impossible (afaik)
to write DAML code that generates DAML-LF that uses the nested type
directly in some API-relevant position (choice arg/return type, key
type, …). Of course, I could handwrite some DAML-LF but only Rémy is
allowed to do that.
changelog_begin
changelog_end
This asks Azure to run a full compat test against the branch that
updates the compat test matrix, so we know it's good when we merge it
rather than the next morning.
CHANGELOG_BEGIN
CHANGELOG_END
* Respect project name in create-daml-app
Now `daml new foobar create-daml-app` creates a project called
`foobar` replacing the name everywhere. The tests now run twice once
with the original project name since that’s what we use in the GSG and
once with a modified one. I guess you could argue that only running
the second should be enough so I’m open to removing the first one.
fixes#6681
changelog_begin
- [DAML Assistant] `daml new foobar create-daml-app` now properly
respects the project name and creates a project called `foobar`
instead of hardcoding the name fo `create-daml-app`.
changelog_end
* .
changelog_begin
changelog_end
* Remove debugging output
changelog_begin
changelog_end
This PR fixes a few things with the script that automatically updates
the compat test matrix on release, based on its use over the past two
weeks. Specifically:
- Send an error message to Slack in case this job fails. Previously,
failures here were silent.
- Add an exponential backoff strategy to wait for the artifact to be
available on Maven. Previously, the update script just failed.
- Allow for rerunning on the same machine after failure by removing the
branch if it exists.
- Fix the commit message to include proper newlines instead of literal
`\n`'s.
CHANGELOG_BEGIN
CHANGELOG_END
The intention was to pull in the source for the GSG but it turns out
that the templates tarball doesn’t just bundle up files, it also
includes generated scala code for the scala quickstart which depends
on damlc. This PR splits the GSG sources including the patching into a
separate rule which does not depend on damlc and only copies that in
live-preview.
changelog_begin
changelog_end
[Sandbox][Ledger Integration Kit]: Metrics can now be exported to graphite with a prefix using the format ``--metrics-reporter=graphite://host:port/prefix``.
CHANGELOG_BEGIN
CHANGELOG_END
* include DefTemplate's key types under the Ty type variable
* note that contract key types are included by folding over DefTemplates' Tys
- the topo sort from --root now correctly includes contract keys
* test that contract key dependencies get included in the Scala codegen plan
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* Removed unused code.
* Basic interface draft for producing pre-execution results.
* Share code for running all steps for a committer.
* Code tidying.
* Removed type parameter Submission from Committer.
* Simplify creation and using of committers.
* ExecuteSubmission => SubmissionExecutor
* Code tidying.
* Code tidying.
CHANGELOG_BEGIN
CHANGELOG_END
* Update ledger/participant-state/kvutils/src/main/scala/com/daml/ledger/participant/state/kvutils/committer/Committer.scala
Co-authored-by: Andreas Lochbihler <36634420+andreaslochbihler-da@users.noreply.github.com>
* Fixed build.
* Basic test cases for dryRun.
* Set to-be-notified participants to be everyone.
* Some tests for runSteps.
* Fixing merge.
* Organized imports.
* Added separate timer for pre-execution.
* Set min/max record time from context.
* preexecution => preExecution
* Code tidying.
* Code tidying.
* Set out-of-time-bounds log entry from context.
* Code tidying.
* Code tidying.
* Keep track of deduplication time window for transactions.
* Set deduplicateUntil in the generated log entry.
* Made min/max record time optional in pre-execution result.
Co-authored-by: Andreas Lochbihler <36634420+andreaslochbihler-da@users.noreply.github.com>
This bumps the timeout of the compat tests on PRs to 360 minutes
matching other jobs on a PR (we mainly hit this if ghc-lib is rebuilt)
and the timeout on the daily jobs to 720 minutes (we hit this if
_everything_ is rebuilt).
I am slightly worried about the timeout on the daily job. After having
taken a look at it, there are a few reasons how we ended up here:
1. We started including more tests, e.g., sandbox-classic. Not much we
can do here, those tests are useful.
2. We have a very large number of snapshots for 1.3.0. There are a few
reasons for this:
1. Timing: We branched off early for the 1.2.0 release so the first
snapshot for 1.3 was on June 3th. For 1.4 it looks like the first
snapshot will be on July 15th so that’s roughly 2 extra
snapshots just due to timing.
2. Additional snapshots: We had one broken snapshot due to a broken
VSCode extension that we didn’t delete (probably not worth doing
at this point). We also had to backport to an old snapshot which
resulted in another extra snapshot. We also had one extra
snapshot which was supposed to be the RC but wasn’t since the
ANF revert needed to go in.
The only thing that is clearly useless is the one broken snapshot
but that doesn’t change things that much. I see 2 orthogonal
options for improving this assuming we agree that the current
runtime is worryingly high.
1. Prune snapshots more aggressively, e.g., only include the last 3
snapshots. That’s a pretty arbitrary decision but it would
enforce a hard limit.
2. Reduce test combinations. E.g., only test snapshots vs stable
releases but not snapshots vs snapshots.
3. We end up forcing a full build quite frequently. Here are just 2
examples of how we’ve done that so far.
1. Upgrade rules_haskell. Basically all tests are run by a Haskell
binary so this forces a full rebuild.
2. Change runfiles of `daml`.
I don’t think there is much we can do about 1 or 3 which leaves us
with 2. One not entirely unreasonable option is to just do nothing. We
did have periods where things went pretty smoothly for the most part
and each month we reset to a much smaller number of releases (we also
have to start throwing out old stable releases at some
point). Otherwise reducing the number of test combinations seems the
most promising option to me.
changelog_begin
changelog_end
* Fix fine-grained exclusions in ledger-api-test-tool
31995ee000 accidentally changed
`allTestCaseNames` to be a list of test suite names instead of a list
of test case names. That breaks fine-grained exclusions that we use in
our compatibility tests. I’ve also added a very basic test that both
the coarse-grained and fine-grained syntax is supported.
changelog_begin
changelog_end
* Update ledger/ledger-api-test-tool/BUILD.bazel
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Generate JS + typings to speed up dam2js
This speeds up daml2js to the point where it’s basically instant on
the GSG (based on 3 runs before and after it’s somewhere between a
40-50x speedup).
Most of this is fairly straightforward. I looked at the generated code
from typescript and the docs for declaration files and adapted things.
There is one interesting point here:
We set the TypeScript configuration to commonjs + ES5. While ES6
modules work fine via `yarn start`, they unfortunately work less fine
in nodejs and in particular in jest. I spent some time trying to fix
it and in the end decided that this is not worth doing since all our
users have to do the same which sucks.
Therefore, the codegen also emits ES5 + commonjs so this is not a
breaking change.
The tests pass and I went through the GSG manually to check things
still work and also verified that types show up correctly in VSCode.
What I was sadly unable to do is to keep the ESLint test for the .d.ts
files. typescript-eslint really wants everything to belong to a
typescript project with a tsconfig.json which doesn’t make sense
here. I don’t think that’s a huge loss. We still have ESLint for the
JS files.
changelog_begin
- [daml2js] daml2js now generates JS + typescript declaration files
directly instead of generating TypeScript code and invoking `yarn` to
generate the JS + the declaration files. This does not require any
changes to your code but it brings a significant speedup and removes the
dependency on `yarn` for running `daml codegen js`.
changelog_end
* Update language-support/ts/codegen/src/TsCodeGenMain.hs
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* Update language-support/ts/codegen/src/TsCodeGenMain.hs
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* Add details about validity windows
* Reword description of auth service concepts
* Elaborate on auth flow dealing with token/credential expiry
CHANGELOG_BEGIN
CHANGELOG_END
* declare key type in Scala codegen output
* add `key` type and function to TemplateCompanion
* template key exercise* case
* test idiomatic ExerciseByKey creation
* add changelog
CHANGELOG_BEGIN
- [Scala Codegen] Support for creating exercise-by-key commands, using same
exercise functions as exist for contract-IDs and the ``.createAnd`` pattern.
See `issue #6466 <https://github.com/digital-asset/daml/pull/6466>`_.
CHANGELOG_END
* Ledger API Test Tool: deprecate unneeded tests and options
Furthermore, now all active tests are now run be default without needing
to specify --all-tests. The scheduler makes sure to run non-isolated
tests last (and sequentially).
Fixes#3747Fixes#6518
A bit of historical context: tests in the Ledger API Test Tool used to
be the sandbox integration tests. As part of the Ledger API Test Tool
project, those tests have been ported one-to-one, with little review of
those same tests.
Examining the tests that cause #3747, it became evident that those tests
were originally put in place to verify that an isolated sandbox spun up
for a single test could withstand a number of commands slightly lower
than what at the time was the hard-coded back-pressure threshold. Hence,
implementing back-pressure on the Ledger API Test Tool would have
basically negated the usefulness of those tests. Furthermore, those
tests can be easily passed by simply raising the back-pressure
threshold. As such, they do not convey any meaningful information for
the DAML ledger implementor. This means we are retiring those tests. The
tests will be nominally kept in for a deprecation period to not break
existing build scripts, but running them explicitly will simply show
them as skipped. The documentation has been updated with the information
necessary to users to deal with retired tests.
Since all tests are now run by default --all-tests is being deprecated.
--load-scale-factor is also deprecated as it was used only by
TransactionScaleIT.
changelog_begin
[Ledger API Test Tool] The LotsOfPartiesIT and TransactionScaleIT test
suite have been deemed not providing relevant signal to DAML ledger
implementers and have been retired. The tests will be nominally kept in
but will be skipped while they are in a deprecation period. You are
advised to remove explicit references to those tests before they are
fully removed.
[Ledger API Test Tool] All tests are now run by default. The --all-tests
option is now ineffective and deprecated. You are advised to remove its
usages from your build scripts. Non-isolated tests that could affect the
global state of the ledger and interfere with other tests are now
automatically scheduled by the test tool to run sequentially at the end
of the run.
[Ledger API Test Tool] The --load-scale-factor option is now unused and
deprecated. You are advised to remove its usages from your build
scripts.
changelog_end
* Add test for deprecated CLI options
* Address https://github.com/digital-asset/daml/pull/6651#discussion_r452100158
* Address https://github.com/digital-asset/daml/pull/6651#discussion_r452126405
* Address https://github.com/digital-asset/daml/pull/6651#discussion_r452143811
* Address test failures
The default cache directories in yarn don’t work in the Bazel
sandbox. This results in Yarn creating tons of temp directories for
each test that are never cleared up.
This PR changes this to create actual temp directories that are
properly cleaned up.
changelog_begin
changelog_end
* REPL: Factor out Dar loading
* REPL accept multiple DARs
* REPL CLI take multiple packages
CHANGELOG_BEGIN
- [DAML REPL] The REPL now supports loading multiple DARs specified at
the command line. The DARs are loaded as data-dependencies, and the
modules of each DAR's main DALF will be imported into the REPL session
at startup.
CHANGELOG_END
* REPL test loading multiple DARs
* REPL Load packages from current session
* REPL func-test test symbols from two DARs
* Update compiler/damlc/daml-compiler/src/DA/Daml/Compiler/Repl.hs
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Fix leakage of SDK installations
This fixes an issue where the integration tests leaked SDK
installations. Easily verified by `ls /tmp/extra-dir*` before and
after running the tests.
changelog_begin
changelog_end
* I hate windows so much
changelog_begin
changelog_end
* sandbox: Move helpers in EmptyLedgerIT to a companion object.
* sandbox: Accept a new time model when none is set.
CHANGELOG_BEGIN
- [Ledger API Server + (Sandbox / Ledger Integration Kit)]
Accept a new time model if none is set. Previously, it would
erroneously be rejected because the generation number submitted to
was incorrectly set to `2` rather than `1`.
This would not affect most users of Sandbox or other kvutils-based
ledgers, as if a configuration is set automatically on startup when
creating a new ledger. It only affects users who explicitly override
the initial ledger configuration submit delay to something longer than
a few milliseconds.
CHANGELOG_END
* sandbox: Ensure that completions can be streamed from the beginning.
* sandbox: In EmptyLedgerIT, use the submission service.
* sandbox: Split EmptyLedgerIT into two.
* sandbox: In CompletionServiceWithEmptyLedgerIT, assume CI will be slow.
In other words, don't create a completion stream and expect a submission
to show up in a reasonable amount of time.
This changed by the revert of the ANF changes which is harmless by the
same reasoning that made bumping it harmless when we introduced it.
changelog_begin
changelog_end
We saw a flake recently where PostgreSQL stopped accepting connections
during a CI run, leading the build to fail. This increases the number of
connections to 200 from the default of 100, hopefully mitigating issues
such as this one.
CHANGELOG_BEGIN
CHANGELOG_END
