* Canton conformance test debug logging and reduced test parallelism
To reign in flakiness of a handful of TransactionServiceIT tests
that had run into timeouts during synchronize
Closes#7316
changelog_begin
changelog_end
* Review feedback from Stefano
This is clearly a breaking change but triggers are still alpha so we
can get away this (confirmed with Bernhard).
changelog_begin
- [DAML Trigger] Daml.Trigger.Assert now uses DAML Script instead of scenarios.
changelog_end
This PR perform the following changes:
- reduce function size by spliting them out,
- rename "translate" function prefix to "compile",
- replace 'private' by 'private[this]',
- add 'inline' annotation to function used only once.
CHANGELOG_BEGIN
CHANGELOG_END
* Filter implicit parameter instances from data-dependencies
Occasionally GHC likes to float implicit parameters to the top-level
making them look like any other typeclass instances. Implicit
parameters are by design not unique so if we try to reconstruct them
we get an error about a duplicate instance.
I haven’t figured out when exactly GHC floats things to the top-level
but I did verify that this testcase breaks without the fix.
changelog_begin
changelog_end
* Assert that there are 2 implicit instances
changelog_begin
changelog_end
* get a LoggingContext into the TriggerRunnerImpl
* make some implicits more implicitly scoped and explicitly ascribed
* make some private/final markings
* most of JsonFormat[Identifier] is in companion
* experimental LoggingContext with phantom type
* ActorContext#log isn't really doing that much
* more details of LoggingContextOf
* make LoggingContextOf compile
* add trigger message logging, yet without context
* fix parent compile errors
* use Config as the phantom for its own logging extensions
* LocalDateTimeFormat cleanup
* switch TriggerRunner to contextual logging
* add trigger definition ID to logs
* log trigger-submitted commands, fix trigger test compile
* log trigger stopping and DAR uploads
* add context to PostStop/PreRestart logs
* add changelog
CHANGELOG_BEGIN
- [Triggers] More detailed logging of trigger actions and trigger service actions.
See `issue #7205 <https://github.com/digital-asset/daml/pull/7205>`_.
CHANGELOG_END
* missed copyright header
* switch to Unit, scala/bug#9240 fixed
* mark canton-conformance-test as flaky
CHANGELOG_BEGIN
CHANGELOG_END
* Address Moritz's review
* fix
* fix
* Disable a handful of TransactionServiceIT changes that flake
rather than making the entire canton suite as flaky
Related to #7316
changelog_begin
changelog_end
* formatting
Co-authored-by: Oliver Seeliger <oliver.seeliger@digitalasset.com>
CHANGELOG_BEGIN
CHANGELOG_END
Update exclusions to include new snapshot
changelog_begin
changelog_end
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
I’ve removed dockerode which we don’t seem to use and which depends on
a bunch of outdated versions in particular bl 1.x transitively.
changelog_begin
changelog_end
@rohanjr is taking care of 1.5.0-snapshot.20200901.5116.0.4460cb5e (#7296), so they get pushed back to the end of the line.
Please do not merge this before #7296.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* Preserve empty variant constructor in data-deps.
This fixes#7207 and adds a regression test. In a
separate PR I'll add a warning for variants with
single argument of unit type and add a changelog
entry.
changelog_begin
changelog_end
* Add special case for single constructor variants
* Add test for special case
* uniquely name tests to easily facilitate running them one by one for troubleshooting
CHANGELOG_BEGIN
CHANGELOG_END
Signed-off-by: Brian Healey <brian.healey@digitalasset.com>
* ensure a few further tests do not include a portion of another tests shortIdentifier
* no overlap for CSsubmitAndWaitForTransactionTree
* Migrate skeleton to DAML Script
Also kills the separate setup module since it’s the same thing
changelog_begin
changelog_end
* Fix test
changelog_begin
changelog_end
* Adding `uniqueModuleEntity`
and making sure that generated domain Template IDs do not
have module entity duplicates, so resolution should work
with no problem.
changelog_begin
changelog_end
* cleaning up
* release 1.5.0-snapshot.20200901.5116.0.4460cb5e
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@rohanjr is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
* release 1.5.0-snapshot.20200902.5118.0.2b3cf1b3
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Partial update to codegen docs
* ADD: JS codegen to codegen docs
* CHANGELOG_BEGIN
ADD: JS codegen to codegen docs
CHANGELOG_END
* Make it clear we're talking about codegen
* Add extra reference to JS, JS -> TS, and mention missing Java/Scala classes
* Add placeholder for SDK version
* Run gatling scenario from the perf runner main
reports are disabled for now, getting a class not found
when generating them
changelog_begin
changelog_end
* kvutils/tools: Split the integrity checkers by package.
The two versions of the integrity checker are intertwined when they
really don't rely on each other at all. This splits them into two
separate packages to make the distinction clear.
CHANGELOG_BEGIN
CHANGELOG_END
* kvutils/tools: Split the integrity checkers by directory.
This splits the integrity checkers further into separate directories, to
make it clear they do not interact at all.
* kvutils/tools: Rename "integrity-check" to "integrity-check-v1".
* ledger-on-memory: Recommend ledger exports (v3), not ledger dumps (v1).
* kvutils/tools: Split the benchmarks from integrity-check-v2.
There are two sources of flakiness and I’ve seen both on CI:
1. We can get more than one offset at the beginning if things are too
slow. This is addressed by just filtering those out.
2. The stream completes as soon as the input is closed. This is
addressed by keeping the stream open and closing it with `take`.
Point 2 is a problem for all tests, see #7293, but I’ll leave the
other tests for separate PRs (I’ve also never seen them flake).
You can observe the failures locally if you add a `Thread.sleep` between
creating the stream future and sending the commands.
changelog_begin
changelog_end
* ledger-api-client: `maxInboundMessageSize` -> `maxInboundMetadataSize`.
CHANGELOG_BEGIN
- [Scala Bindings] Rename a field in the ``LedgerClientConfiguration``
to ``maxInboundMetadataSize``, to match the builder Netty channel
builder. It was incorrectly named ``maxInboundMessageSize``, which is
a different channel property that configures the maximum message size,
not the header size.
CHANGELOG_END
* ledger-api-client: Introduce a `maxInboundMessageSize` config property.
We use this a lot; easier if it's in the configuration.
CHANGELOG_BEGIN
- [Scala Bindings] Replace the
``LedgerClientConfiguration.maxInboundMessageSize`` property with a
new one that represents the maximum size of the response body.
CHANGELOG_END
* kvutils: Implement a basic protocol buffer-based export.
* kvutils: Add a header to the protobuf-based export.
So if we decide to go for a v4, it's easier to figure this stuff out.
* kvutils: Use our own header format for export.
The idea is to be somewhat forward-compatible, so relying on protobuf
to always be the format of the future seems a little odd. We may switch.
* kvutils: Construct the importer up-front during integrity checks.
* kvutils: Move the various importers and exporters to versioned packages.
* kvutils: Move the export version to the package object.
* kvutils: Create helpers to construct importers and exporters from paths.
* kvutils: Process either v2 or v3 exports with the integrity checker.
* kvutils: Sort the write set during export.
* kvutils: Switch to v3 of the export.
The integrity checker will work with either version.
CHANGELOG_BEGIN
- [Integration Kit] kvutils-based ledgers will now export the ledger
using a new serialization format based on Protocol Buffers, which we
are referring to as "v3". Existing "v2" exports can still be read and
verified using the integrity checker, which can read both old and new
versions.
CHANGELOG_END
* kvutils: Aggregate export data in a sorted map.
We need to sort it anyway; might as well do it on input.
* kvutils: Improve a test string.
* kvutils: Use the v3 importer in the replay test.
* kvutils: Remove v2 of the export format.
We're pretty sure no one is using it.
* Fix trace statement handling in DAML Script service
This PR makes sure that we get both trace statements coming from the
ledger client as well as from the ledger server. In the script service
those are two different Speedy machines so we need to interleave
them. This is a bit messy so let me explain the steps (apologies for
not splitting it up but I think it’s easier to understand the
motivation when it is a single PR):
1. Abstract over the tracelog interface. The ringbuffer doesn’t makes
sense for our purposes and is annoying to clear.
2. Pass in the respective fields of `Machine` to `Conversions` instead
of the whole machine. This is both a simpler design (passing around
mutable objects scares me) and it allows us to assemble the fields
from different machines.
3. Initialize the ledger machine with a simple ArrayBuffer tracelog.
4. After `submit` and `submitMustFail` copy the tracelog from the
ledger to the client.
fixes#7280
changelog_begin
changelog_end
* Address review feedback
changelog_begin
changelog_end
fix JSON API multikey stream
In the current state, the JSON API only handles multiple keys _from
different templates_. This makes it work for multiple keys from the same
template too.
Extracted from #7066 with the following changes:
- Use of a mutable `HashSet` to test for keys, because perf.
- Addition of a test at the JSON API level.
CHANGELOG_BEGIN
- [JSON API] Fix a bug where streaming multiple keys could silently
ignore some of the given keys.
CHANGELOG_END
* apply @cocreature's patch
https://gist.github.com/cocreature/d35367153a7331dc15cca4e5ea9098f0
* fix fmt
* Only include stakeholder contracts in result of query
This fixes a bug in the script service. We need to filter out divulged
contracts since this should behave exactly like the ACS endpoint on
the ledger API.
changelog_begin
changelog_end
* Update daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/scenario/ScenarioLedger.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Update daml-script/runner/src/main/scala/com/digitalasset/daml/lf/engine/script/LedgerInteraction.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
* fmt
changelog_begin
changelog_end
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Add queryContractId to DAML Script
I’ve switched between a few different naming options and in the end
settled on this one:
- `lookupContract`, not too bad but misses the connection to `query`
- `queryCid`, we don’t abbreviate this in other places in our API so I
don’t think we should here.
- `queryContractId`, makes the connection clear and no
abbreviation. We could also add `queryContractKey` later
changelog_begin
- [DAML Script] Add a `queryContractId` function for querying for a
contract with the given identifier.
changelog_end
* Fix test
changelog_begin
changelog_end
The executor may no longer be available. Fortunately we can avoid using
resources entirely in this case, and just acquire the channel the
old-fashioned way.
CHANGELOG_BEGIN
- [Scala Bindings] Fixed a bug in the ledger client which meant that the
client was not closed properly on application shutdown. You may have
seen some ``RejectedExecutionException`` errors in your logs if this
has affected you. We encourage all users of the `LedgerClient` to call
the ``close`` method explicitly to ensure it is closed at an
appropriate time in your application lifecycle.
CHANGELOG_END
An extraction script that makes the LF reading libraries (DAML LF archive decoding to an AST) usable in other Haskell projects.
This script once existed but is not available any more, so I have resurrected the script, adapted to current usage, and upgraded to the newest stackage LTS that can support it. These libraries _could_ be versioned manually to released SDKs and published if desirable (without making any guarantees about the stability, therefore prefixing the version with `0.`.
CHANGELOG_BEGIN
Tool to extract Haskell libraries for DAML LF archives for use in other Haskell projects
CHANGELOG_END
* reintroducing the main
* Introducing `ledger-service/http-json-testing`
* cleaning up
* Starting sandbox and json-api from perf-test main
changelog_begin
changelog_end
It turns out we were too aggressive in resetting the state. We only
want to reset in `submit` on successful commit and in `submitMustFail`
on failed commits. I’ve also split the reset in two parts, one that is
done at the beginning every time and one at the end. I don’t think it
really makes a difference but it matches what the scenario runner does
which at least makes me stop wondering if the difference matters.
fixes#7275fixes#7276
changelog_begin
changelog_end
* ledger-on-(memory + sql): Include logback as a test dependency.
Otherwise logs don't show up when we get failures, which makes debugging
a lot harder.
* kvutils: On unit test timeout, show a useful error.
* kvutils: Increase the streaming updates timeout in integration tests.
This is timing out every now and again on CI, and I can reproduce it
easily by running the tests in parallel.
CHANGELOG_BEGIN
CHANGELOG_END
I’ve also removed the restriction to only support going forward since
scenarios don’t have that restriction and it seems potentially useful
for tests.
factored out from #7264
changelog_begin
- [DAML Script] Add `passTime` helper to advance the time by the
given interval.
- [DAML Script] In DAML Studio, you can now set the time to the
past. This is not supported when running against a ledger.
changelog_end
Factored out from #7264
We have enough cases of scenarios that don’t actually do ledger
operations and need some hint for type inference. While adding type
annotations works, this is a bit easier at least for the migration and
some people might prefer it.
No test, since this seems a bit excessive here :)
changelog_begin
- [DAML Script] Add a `script` function to aid type inference. This
is the equivalent of `scenario`.
changelog_end
Factored out from #7264 and added a test. Just makes the migration a
tiny bit easier.
changelog_begin
- [DAML Script] Add `archiveCmd cid` as a convenience wrapper around
`exerciseCmd cid Archive`.
changelog_end
The new behavior here was only added in SDK 1.4.0 so the tests
fail (as expected) for older SDK releases. We only test against the
latest stable release on each PR so we didn’t catch this directly on CI.
changelog_begin
changelog_end
* java-quickstart replace scenarios by DAML Script
This replaces the scenarios in the java quickstart project by DAML
script test cases.
The following changes were required
* Replace `getParty` by `allocateParty`.
* Replace `create` and `exercise` by `createCmd` and `exerciseCmd`.
* Replace the pattern of `submit ... do create ...; exercise ...`
by `submit ... do createAndExerciseCmd ... ...`.
* Replace instances of `submit ... do fetch ...` by `query ... ...`.
To that end I've added the following helper function.
```
queryIou : Party -> ContractId Iou -> Optional Iou
```
changelog_begin
changelog_end
* Replace queryIou by FetchIou choice
changelog_begin
changelog_end
* Add comment explaining fetch choice
Addressing review comment
https://github.com/digital-asset/daml/pull/7183#discussion_r478918758
* Support dependencies in daml_test rule
Cherry-picked from https://github.com/digital-asset/daml/pull/7264/files
Renamed `dependencies` as `deps` to stick to Bazel conventions.
* Fix quickstart `daml_test` target
* fmt
* Quote deps in daml.yaml
To avoid issues with `:` in Windows paths.
* //docs:quickstart-java - use temp dir
This target repeatedly failed to build on Windows with
```
ERROR: D:/a/1/s/docs/BUILD.bazel:538:8: declared output
'docs/quickstart-java.tar.gz' was not created by genrule. This is
probably because the genrule actually didn't create this output, or
because the output was a directory and the genrule was run remotely
(note that only the contents of declared file outputs are copied from
genrules run remotely)
```
The genrule was operating in the current working directory and builds on
Windows are not sandboxed. Conflicting files in the current working
directory could cause this to fail.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Add a dummy OAuth2 authorization server
This is intended for use in the trigger service integration tests not
as an artifact that we ship to users.
changelog_begin
changelog_end
* Address review feedback
changelog_begin
changelog_end
* Refactor EnrichedTransaction
This PR prepares for work to interleave _authorization_ with _execution_, which has been an open issue for a long time: #132.
- Split computation of `BlindingInfo` and `FailedAuthorizations` into separate modules.
- Fixup callers: `Blinding.scala` and `ScenarioLedger.scala`, to do:
```
val failedAuthorizations =
AuthorizingTransaction.checkAuthFailures(authorization, tx)
val blindingInfo =
BlindingTransaction.calculateBlindingInfo(tx)
```
- Simplify code to remove `DontAuthorize` and `Authorization` types, which were only needed to restrict the combined _enrichment_ code from doing the authorization check when `blind`ing only is required. Now `Blinding.blind` calls just `calculateBlindingInfo`.
* address comments from Stefano
* Track source locations on Daml.Script.submit
Adds a `HasCallStack` constraint on `HasSubmit.submit` and
`HasSubmit.submitMustfail`.
We need to move the definition of `HasCallStack` to `GHC.Stack.Types` to
avoid a cyclic dependency between `DA.Stack` and `DA.Internal.LF`.
Alternatively, we could inline `HasCallStack` in `DA.Internal.LF` but it
seems preferable to use the alias on the signature of
`submit(MustFail)`.
We need to adapt the type of the builtins for scenario `submit` and
`submitMustfail`.
changelog_begin
changelog_end
* Test-case for scenario/script source locations
* Test second commit with intermediate definition
Addressing review comment
https://github.com/digital-asset/daml/pull/7252/files#r478518834
* Store the full callstack on submit
For forwards compatibility.
Addressing review comment
https://github.com/digital-asset/daml/pull/7252/files#r478520268
* Document unused location definition
Addressing review commit
https://github.com/digital-asset/daml/pull/7252/files#r478522373
* Make user facing `SrcLoc` zero based
Following LSP precedent rather than GHC
Addressing review comment
https://github.com/digital-asset/daml/pull/7252/files#r478529138
* Add comment for backwards compatibility code
Addressing review comment
https://github.com/digital-asset/daml/pull/7252/files#r478529138
* fmt
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
