* Add race condition tests for exceptions
This PR addresses
https://github.com/digital-asset/daml/pull/9400#pullrequestreview-634770251
and adds tests that match RaceConditionTests but to the read side in a
rollback (we cannot do writes in rollbacks, they are rolled back :)).
The tests are as close as possible to the other race condition tests
to ease maintenance and reduce confusion.
changelog_begin
changelog_end
* remove commented lines
changelog_begin
changelog_end
* Disable accidentally enabled ClosedWorldIT
changelog_begin
changelog_end
* Lift constraint tuples up to a type synonym in data-dependencies.
Fixes#9663 and adds a regression test.
GHC was crushing the constraint tuples into regular constraints, so the function caller no longer matched the original LF API. By lifting the constraint tuple into a type synonym, we eliminate this effect.
Part of the issue here is that we're losing the original constraint tuple type synonym during LFConversion. I'll open a separate issue for that, since I think it's also worth fixing that going forward.
(Preserving the type synonym would solve this issue for newly compiled packages. By contrast, the fix in this PR works for both new and old packages.)
changelog_begin
- [Daml Compiler] Fixed a data-dependencies bug where functions in a data-dependency that used a constraint tuple constraint (e.g. `Template t`) could not be directly invoked.
changelog_end
* use foldl'
* Fix exception test that spawned this mess
This is a preparatory refactoring PR so we can use the same utilities
for the existing race condition tests and for a test suite that tests
race conditions in combinations with exceptions.
changelog_begin
changelog_end
* Add a data-dependencies test for exceptions.
Part of #8020
This only tests that (catching & throwing) user-defined exceptions compiles as expected across data-depndencies.
Once they're supported, we should also add a test involving throwing a built-in exception (arithmetic or contract errors) in the data-dependency and catching them in main.
changelog_begin
changelog_end
* Fix weird formatting
* Rename test case
* syntax error
* fix easy problems
* Work around constraint tuple bug.
Currently we have the following dependency chain:
1. The compiler depends on the scenario service for daml test
2. The scenario service depends on Daml Script
3. Daml Script depends on the Sandbox code only for daml test-script
The last one can easily be split. The scenario service does not care
about this code.
This means that now if we change ledger code, at least not all
compiler tests are going to rerun.
Verified that this actually breaks the dependency fully via
```
bazel query 'somepath(//compiler/damlc/tests:packaging, //ledger/participant-integration-api/...)'
```
changelog_begin
changelog_end
The first one ended up with a corrupted Windows installer. I couldn’t
find anything in the logs indicating why it would be broken and this
is the first time I’ve seen this so hopefully just rereleasing fixes it.
changelog_begin
changelog_end
* Throw on internal errors instead of setting ptx to aborted
The main purpose of aborted is to be able to get out a ptx in the
scenario service. However, for internal errors that makes no
sense. Users should never encounter them. And the use of `aborted`
here has swallowed these errors silently in a few places before which
is clearly bad.
changelog_begin
changelog_end
* Fix tryHandleSubmitMustFail
changelog_begin
changelog_end
* restore natural join
CHANGELOG_BEGIN
CHANGELOG_END
* JOIN with ON clause rather than NATURAL JOIN so table aliasing will work cross-platform
* scalafmt
@victormueller-da is taking care of 1.13.0-snapshot.20210511.6892.0.ca9e89b3 (#9647), so they get pushed back to the end of the line.
Please do not merge this before #9647.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* Remove version field from rollback node.
changelog_begin
changelog_end
fix transaction decode & testcases
reinstate/adapt testcases
fmt
rebase change to fix conflicts
* treat missing version on transaction node as being version minExceptions when computing tx version
* add 8020 marker for grepability
* reinstate Node.version to avoid logic duplication
* Revert "reinstate Node.version to avoid logic duplication"
This reverts commit 00dacb453d.
* remove Node.updateVersion
* remove comments with dubious utility
Caching doesn't seem to work very well here. On a release, we build an
old commit, which has already been tested twice (once as a commit on
`main`, once as part of the release PR).
CHANGELOG_BEGIN
CHANGELOG_END
In the automated process, the Azure build is triggered on the branch
directly, which will be named `auto-release-pr-$(date -I)`. But if a
manual change needs to be made, and people subsequently use the `/azp
run` feature of Azure, the build then runs for the PR, which means it
actually runs on the merge commit of the branch and `main`, not on the
branch itself. In that case, the branch that we run the build on is
called `merge` and is thus not starting with `auto-release-pr-`.
This change should get us the notification back on manual PR builds too.
CHANGELOG_BEGIN
CHANGELOG_END
* Drop version check from TransactionPreprocessor
This is only supposed to be run on nodes that have been normalized via
de(serialization) so this is just useless shotgun validation.
changelog_begin
changelog_end
* Fix test
changelog_begin
changelog_end
* release 1.13.0-snapshot.20210511.6892.0.ca9e89b3
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@victormueller-da is in charge of this release.
Commit log:
```
ca9e89b3da check whether collection.compat is unused when compiling for Scala 2.12 (#9604)
4a460973a2 Block concurrent SDK installations. (#9640)
c757b61516 Check byKey in transaction validation (#9641)
4f0ceff7a2Fix#7170 (#9618)
d3106682dd Add a REPL for each Scala target, for debugging. (#9643)
22b36b0b01 Include byKey in LF transaction spec (#9642)
ca027e3dde Add Transaction.contractKeyInputs to inputs of a transaction (#9631)
c3d79d4656 Allow OverloadedLists in data-dependencies (#9636)
f8b35fef9c Cleans stateUpdates of ReadWriteServiceBridge (#9630)
3750b16a11 Serializalize byKey flag in Daml-LF transactions (#9632)
3b59f8ce87 ledger-api-bench-tool - prototype [DPP-370] (#9606)
f7b33d93fb Authorization for daml script exports (#9629)
0d1f3db8a4 LF: refactor builtin exceptions in Speedy (#9605)
d6d01b04ef Swap order SEScopeExercise and SBUBeginExercise (#9621)
6c919b3877 sandbox-on-x: Support `beginAfter` in state updates. (#9624)
26a8011ba5 TLS for Daml Script exports (#9626)
9242540022 Make error throw a GeneralError. (#9613)
bdcf0ecb66 update NOTICES file (#9623)
4c1fbeb194 Add duplicate contract key checks to Speedy (#9607)
871279f3a6 LF: extends LF command with Fetch and Lookup (#9587)
0acc4f115c Patch old Bazel derivation (#9622)
b082274885 Address vulnerabilities in Navigator (#9617)
560653a5d7 Unwind partial transaction on mustFail (#9608)
f51145cb88 Support mapping from old to new party ids in daml exports (#9591)
89e46bf90b Address security vulnerabilities in //compiler/daml-extension (#9615)
fd62671e0b Introduce SINCE-LF-FEATURE in integration tests. (#9616)
48939b513b Address vulnerabilities in //language-support/ts/packages (#9614)
cf59246d44 Add support for daml exceptions for append-only indexer (#9609)
ab29f7c07c Move activeness check of globalKeyInputs into archive (#9610)
5c28de36cb Upgrade grunt to address cve (#9611)
22ba5fddd2 Remove builtin exception types from protobuf and ASTs. (#9595)
b09a95fb6f Adapt indexer to empty divulgences (#9598)
2fc7489e44 Filter divulgence to an empty set of parties (#9600)
ad45213b66 participant-integration-api: Avoid the serial EC in integration testing. (#9603)
f742a4334e Dpp 336 sandbox classic on append only (#9561)
e68bc0dff0 Mark reset service tests as flaky (#9602)
2176173ba2 Remove 1.dev-only things from LF 1.13 protofile. (#9599)
03034ec3bf DPP-359 Add indexer flow level benchmark (#9509)
5aa54017eb Drop todo for conversion of exception primitives (#9597)
968b5d8d39 KVL-921 Expose opentelemetry Context from Telemetry and TelemetryContext (#9573)
112c387e5a Refactor out setGlobalLogLevel into ContextualLogging (#9592)
e584fecd3c Drop com.daml.lf.engine.Event.collectEvents (#9596)
80b07da309 Only archive a key if it was brought into scope before (#9546)
7ea9340d64 Support old bash in daml-sdk-head (#9593)
45fbdefb06 Handle rollback nodes in protoNodeInfo (#9589)
42d189dbbe Support exceptions in Daml-LF encoder (#9590)
9c3913a3d6 LF: SBUKeyBuiltin clean up. (#9572)
5128206ebf Ledger API Server: Named threadpools (#9588)
26a53d8469 Add logging command line option to ledger http service (#9581)
409833f850 Address rollback todo in ancient migration (#9583)
a27b2c56bc Address more exception todos (#9582)
2040556dbb Support rollback nodes in replay adapter (#9584)
192a77a6d8 update compat versions for 1.13.0-snapshot.20210504.6833.0.9ae787d0 (#9575)
81d508b775 LF: make SBuiltins pure (#9580)
de80a6dc60 Drop ValueBuiltinException (#9576)
1c456be79d Release yet another 1.13 snapshot (#9574)
800989145b Disable migration tests on macos on PRs (#9571)
2e93de79a4 Fix recording exceptions in Spans, add unit tests [KVL-874] (#9544)
121ded3565 Accept a comma-separated list of parties for daml ledger export (#9568)
```
Changelog:
```
- [Daml Assistant] The assistant will now avoid
installing SDK versions concurrently, waiting
for the previous installation to finish before
starting the next installation (if still
necessary). This fixes a bug where the SDK
would become corrupted because two
installations were started concurrently.
http-json:
- add contextual id as logging context to distinguish different application runs in logs
- add request id as logging context to distinguish different http requests within an application run
- add for non-static endpoints trace logs which show how long processing it took in ns
- [Integration Kit] - Created the ledger-api-bench-tool prototype for benchmarking ledger transaction streaming capabilities
* [Daml export] Users can now define a mapping from parties in the
original ledger state to parties to be used when recontructing the
ledger state. The ``parties`` component of the argument to the
generated export script now takes a mapping from old party name to new
party name.
- [Ledger HTTP Json Service] Logging now also tells service name if log level was changed.
- [Ledger HTTP Json service] Logging can now be configured via the `--log-level` cli argument. Valid values are `error`, `warn`, `info` (default), `debug`, `trace`
```
CHANGELOG_BEGIN
CHANGELOG_END
* Release snapshot 1.14.0
changelog_begin
changelog_end
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: victor.mueller@digitalasset.com <mueller.vpr@gmail.com>
Note that these tests are switched off for now until Robert has
implemented the migration.
To avoid having to stuff too much in one PR the plan is to merge this
one first and then switch the flag once the migrations work.
changelog_begin
changelog_end
* check whether collection.compat is unused when compiling for Scala 2.12
- Instead of always suppressing warnings for collection.compat._,
we should only do it for Scala 2.13
- We can also reduce boilerplate by automatically adding this
option when both silencer_plugin and collection-compat are
present
CHANGELOG_BEGIN
CHANGELOG_END
* remove unused import
* remove another unused import
* remove even more unused imports
* missed compat dependency
* more missed compat dependencies
* missed compat dependency
* use scala_deps in scaladoc_jar
- #8423 inlined the major version expansion, but this seems to
have been prior to proper support by scaladoc_jar
* restore custom handling of participant-integration-api
- fixing scaladoc_jar isn't worth it for a single case, as with
deps vs scala_deps
* Block concurrent SDK installations.
Fixes#9612
changelog_begin
- [Daml Assistant] The assistant will now avoid
installing SDK versions concurrently, waiting
for the previous installation to finish before
starting the next installation (if still
necessary). This fixes a bug where the SDK
would become corrupted because two
installations were started concurrently.
changelog_end
* Remove filelock from stack-snapshot.yaml
* Update windows snapshot pin
* Add message if file lock is not immediately available.
* Check byKey in transaction validation
No tests because validation has no tests in general :(
part of #7622
changelog_begin
changelog_end
* Fix preprocessing of transaction root nodes
changelog_begin
changelog_end
CHANGELOG_BEGIN
http-json:
- add contextual id as logging context to distinguish different application runs in logs
- add request id as logging context to distinguish different http requests within an application run
- add for non-static endpoints trace logs which show how long processing it took in ns
CHANGELOG_END
* Add a REPL for each Scala target, for debugging.
For each Scala library target, this adds a `*_repl` target that can be
built and run to provide a Scala REPL with access to the library and all
its dependencies.
To use:
```
$ bazel build //ledger/ledger-on-memory:ledger-on-memory_repl
INFO: Invocation ID: f1c4ec07-68e7-4bc2-a182-40f6ea035110
INFO: Analyzed target //ledger/ledger-on-memory:ledger-on-memory_repl (32 packages loaded, 714 targets configured).
INFO: Found 1 target...
Target //ledger/ledger-on-memory:ledger-on-memory_repl up-to-date:
bazel-bin/ledger/ledger-on-memory/ledger-on-memory_repl.jar
bazel-bin/ledger/ledger-on-memory/ledger-on-memory_repl
INFO: Build completed successfully, 7 total actions
$ ./bazel-bin/ledger/ledger-on-memory/ledger-on-memory_repl
Welcome to Scala 2.12.13 (OpenJDK 64-Bit Server VM, Java 1.8.0_272).
Type in expressions for evaluation. Or try :help.
scala> import scala.concurrent.ExecutionContext.Implicits.global
scala> import com.google.protobuf.ByteString
scala> import com.daml.ledger.on.memory._
scala> import com.daml.ledger.participant.state.kvutils.OffsetBuilder
scala> import com.daml.ledger.participant.state.kvutils.Raw
scala> import com.daml.ledger.participant.state.kvutils.api.LedgerRecord
scala> val state = InMemoryState.empty
state: com.daml.ledger.on.memory.InMemoryState = com.daml.ledger.on.memory.InMemoryState@ff21443
scala> state.write { (log, state) => Future { log += LedgerRecord(OffsetBuilder.fromLong(1), Raw.LogEntryId(ByteString.copyFromUtf8("A")), Raw.Envelope(ByteString.EMPTY)) } }
```
The REPL target has the tag "manual", so will only be built on demand.
CHANGELOG_BEGIN
CHANGELOG_END
* Turns out you can run a REPL with `bazel run`.
In order not to make the appearance that current state of sandbox-on-x can support anything than clean initialization form scratch, and clean initialization from already ingested database.
changelog_begin
changelog_end
* Simple scaffolding for the ledger-api-bench-tool
* Added README
* Added resource management and the LedgerIdentityService
* Changed the default log level to DEBUG
* Added the TransactionService
* Added stream configuration options
* Options for ledger configuration
* Minor improvements
* Refactored packages
* Minor improvement
* CHANGELOG_BEGIN
- [Integration Kit] - Created the ledger-api-bench-tool prototype for benchmarking ledger transaction streaming capabilities
CHANGELOG_END
* Unified endpoint argument with the ledger-api-test-tool + other minor fixes
* Logger as an argument to LogOnlyObserver
* Add token parameter to daml export
changelog_begin
changelog_end
* Add test case for --access-token-flag
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Swap order SEScopeExercise and SBUBeginExercise
SEScopeExercise pushes the KCloseExercise continuation. However, the
corresponding transaction node is only inserted when SBUBeginExercise
finishes. This causes an issue if SBUBeginExercise crashes/throws an
exception.
changelog_begin
changelog_end
remove weird comment
changelog_begin
changelog_end
Reenable another test
changelog_begin
changelog_end
* Update failing test
changelog_begin
changelog_end
* Make error throw a GeneralError.
As well as abort, fail, etc.
changelog_begin
changelog_end
* keep the error message when you have an unhandled error in scenario
* Disable crashing opsem tests for now.
* Update CommandServiceIT regex pattern.
* Put | in wrong place :-|
* forgot to escape "
* Illegal repetition!
