* Removes TODO related to BatchingParallelIngestionPipe type: the result of the ingestion is not needed at the moment
* Removes TODO related to groupedWithin: at this point the introduced latency seems not significant
* Refactor AsyncSupport
* Removing some TODOs related to postponed work-items
* moving mutable initialisation code does not make sense without refactoring/segregating the initialisation part of the LedgerDao. That effort is postponed as per group decision.
* Injecting the dispatcher makes little sense at this point, further features might require it.
* Fixes type derivation in ParallelIndexerFactory
* Moves instrumentedBufferedSource to metrics (+ unit tests)
* Removes TODO related to logging: primarily to pass down the initializing LoggingContext is fine. (further logging discussion is in progress related to ingestion at the time of writing)
* Refactor type cast towards synthetic zero-element generation
* Expose configuration for the instrumented input, lowers default.
* Remove TODO related wiring of indexer HealthChecks: there is already an issue for this: https://github.com/digital-asset/daml/issues/2733
* Remove TODO related to lowering connectionTimeout for Hikari database-connection pool: it is already the minimum, no smaller connection timeout is available for selection.
changelog_begin
changelog_end
* Split BigNumeric.shift, add BigNumeric.roundToNumeric.
This is based on some feedback we received. The shift direction was
counterintuitive so splitting `shift` into `shiftLeft` and `shiftRight`
makes this more explicit. It's also convenient to have a function that
combines the rounding and casting of BigNumeric to a specific Numeric
scale.
changelog_begin
- [Daml Standard Library] `DA.BigNumeric.shift` has been split into
`DA.BigNumeric.shiftLeft` and `DA.BigNumeric.shiftRight`.
`DA.BigNumeric.roundToNumeric` is introduced, for rounding and
converting a BigNumeric to a Numeric in a single move.
changelog_end
* hide `round` in DA.BigNumeric
* Simplify error docs, add doctest for roundToNumeric
* Update BigNumeric test
* Fix simplifier safety for AnyExceptionMessage
This highlights the danger of shifting definitions. At some point this safety was correct (AnyException contained the message string directly, "throw" took a message argument), and then we decided to have AnyException call a function associated with the type (to speed up exception throwing & catching), and this safety became incorrect :-(
changelog_begin
changelog_end
* s/may crash/may throw
* speedy: Compile new AnyException primitives
changelog_begin
changelog_end
* s/Contract/ContractError
* remove random eszett
* fix booleans
* Combine some cases!
* define and use getSException
* use SException directly in pattern match
* client_server runner - use temp dir for port file
The sandbox will not overwrite an already existing port file but instead
fail, or worse, silently ignore the error and leave the port-file empty.
changelog_begin
changelog_end
* sandbox: Fail if writing the port file fails
So far this was being silently ignored, leaving a pre-existing port-file
untouched.
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
The only thing `sanitize` does is mangle the names of identifiers and
types that end in `#`. That can't be a good thing. This PR removes
sanitize and enables MagicHash in data-dependencies, meaning names can
end in `#` without causing issues. (MagicHash also enables unboxed
literals, but that doesn't matter here.)
In practice, sanitize was only affecting `GHC.Prim.Void#`, which we
define because GHC sometimes references it. AFAIK there's no other
use of `#` in the standard library that we don't rewrite away.
changelog_begin
changelog_end
For better or for worse, kvutils validation insists on treating a
negative input from a create different to a negative key lookup (and
to make things more annoying the first one will never blow up during
submission only during validation).
While, nobody seems to argue all that strongly that the current errors
are very sensible, we agreed to do it in two steps:
1. Expose enough information in `contractKeyInputs` so that we can use
it in kvutils while preserving the current error semantics.
2. Revisit contract key error handling across ledgers which is
currently an inconsistent mess.
changelog_begin
changelog_end
.
changelog_begin
changelog_end
* Add race condition tests for exceptions
This PR addresses
https://github.com/digital-asset/daml/pull/9400#pullrequestreview-634770251
and adds tests that match RaceConditionTests but to the read side in a
rollback (we cannot do writes in rollbacks, they are rolled back :)).
The tests are as close as possible to the other race condition tests
to ease maintenance and reduce confusion.
changelog_begin
changelog_end
* remove commented lines
changelog_begin
changelog_end
* Disable accidentally enabled ClosedWorldIT
changelog_begin
changelog_end
* Lift constraint tuples up to a type synonym in data-dependencies.
Fixes#9663 and adds a regression test.
GHC was crushing the constraint tuples into regular constraints, so the function caller no longer matched the original LF API. By lifting the constraint tuple into a type synonym, we eliminate this effect.
Part of the issue here is that we're losing the original constraint tuple type synonym during LFConversion. I'll open a separate issue for that, since I think it's also worth fixing that going forward.
(Preserving the type synonym would solve this issue for newly compiled packages. By contrast, the fix in this PR works for both new and old packages.)
changelog_begin
- [Daml Compiler] Fixed a data-dependencies bug where functions in a data-dependency that used a constraint tuple constraint (e.g. `Template t`) could not be directly invoked.
changelog_end
* use foldl'
* Fix exception test that spawned this mess
This is a preparatory refactoring PR so we can use the same utilities
for the existing race condition tests and for a test suite that tests
race conditions in combinations with exceptions.
changelog_begin
changelog_end
* Add a data-dependencies test for exceptions.
Part of #8020
This only tests that (catching & throwing) user-defined exceptions compiles as expected across data-depndencies.
Once they're supported, we should also add a test involving throwing a built-in exception (arithmetic or contract errors) in the data-dependency and catching them in main.
changelog_begin
changelog_end
* Fix weird formatting
* Rename test case
* syntax error
* fix easy problems
* Work around constraint tuple bug.
Currently we have the following dependency chain:
1. The compiler depends on the scenario service for daml test
2. The scenario service depends on Daml Script
3. Daml Script depends on the Sandbox code only for daml test-script
The last one can easily be split. The scenario service does not care
about this code.
This means that now if we change ledger code, at least not all
compiler tests are going to rerun.
Verified that this actually breaks the dependency fully via
```
bazel query 'somepath(//compiler/damlc/tests:packaging, //ledger/participant-integration-api/...)'
```
changelog_begin
changelog_end
The first one ended up with a corrupted Windows installer. I couldn’t
find anything in the logs indicating why it would be broken and this
is the first time I’ve seen this so hopefully just rereleasing fixes it.
changelog_begin
changelog_end
* Throw on internal errors instead of setting ptx to aborted
The main purpose of aborted is to be able to get out a ptx in the
scenario service. However, for internal errors that makes no
sense. Users should never encounter them. And the use of `aborted`
here has swallowed these errors silently in a few places before which
is clearly bad.
changelog_begin
changelog_end
* Fix tryHandleSubmitMustFail
changelog_begin
changelog_end
* restore natural join
CHANGELOG_BEGIN
CHANGELOG_END
* JOIN with ON clause rather than NATURAL JOIN so table aliasing will work cross-platform
* scalafmt
@victormueller-da is taking care of 1.13.0-snapshot.20210511.6892.0.ca9e89b3 (#9647), so they get pushed back to the end of the line.
Please do not merge this before #9647.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* Remove version field from rollback node.
changelog_begin
changelog_end
fix transaction decode & testcases
reinstate/adapt testcases
fmt
rebase change to fix conflicts
* treat missing version on transaction node as being version minExceptions when computing tx version
* add 8020 marker for grepability
* reinstate Node.version to avoid logic duplication
* Revert "reinstate Node.version to avoid logic duplication"
This reverts commit 00dacb453d.
* remove Node.updateVersion
* remove comments with dubious utility
Caching doesn't seem to work very well here. On a release, we build an
old commit, which has already been tested twice (once as a commit on
`main`, once as part of the release PR).
CHANGELOG_BEGIN
CHANGELOG_END
In the automated process, the Azure build is triggered on the branch
directly, which will be named `auto-release-pr-$(date -I)`. But if a
manual change needs to be made, and people subsequently use the `/azp
run` feature of Azure, the build then runs for the PR, which means it
actually runs on the merge commit of the branch and `main`, not on the
branch itself. In that case, the branch that we run the build on is
called `merge` and is thus not starting with `auto-release-pr-`.
This change should get us the notification back on manual PR builds too.
CHANGELOG_BEGIN
CHANGELOG_END
* Drop version check from TransactionPreprocessor
This is only supposed to be run on nodes that have been normalized via
de(serialization) so this is just useless shotgun validation.
changelog_begin
changelog_end
* Fix test
changelog_begin
changelog_end
* release 1.13.0-snapshot.20210511.6892.0.ca9e89b3
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@victormueller-da is in charge of this release.
Commit log:
```
ca9e89b3da check whether collection.compat is unused when compiling for Scala 2.12 (#9604)
4a460973a2 Block concurrent SDK installations. (#9640)
c757b61516 Check byKey in transaction validation (#9641)
4f0ceff7a2Fix#7170 (#9618)
d3106682dd Add a REPL for each Scala target, for debugging. (#9643)
22b36b0b01 Include byKey in LF transaction spec (#9642)
ca027e3dde Add Transaction.contractKeyInputs to inputs of a transaction (#9631)
c3d79d4656 Allow OverloadedLists in data-dependencies (#9636)
f8b35fef9c Cleans stateUpdates of ReadWriteServiceBridge (#9630)
3750b16a11 Serializalize byKey flag in Daml-LF transactions (#9632)
3b59f8ce87 ledger-api-bench-tool - prototype [DPP-370] (#9606)
f7b33d93fb Authorization for daml script exports (#9629)
0d1f3db8a4 LF: refactor builtin exceptions in Speedy (#9605)
d6d01b04ef Swap order SEScopeExercise and SBUBeginExercise (#9621)
6c919b3877 sandbox-on-x: Support `beginAfter` in state updates. (#9624)
26a8011ba5 TLS for Daml Script exports (#9626)
9242540022 Make error throw a GeneralError. (#9613)
bdcf0ecb66 update NOTICES file (#9623)
4c1fbeb194 Add duplicate contract key checks to Speedy (#9607)
871279f3a6 LF: extends LF command with Fetch and Lookup (#9587)
0acc4f115c Patch old Bazel derivation (#9622)
b082274885 Address vulnerabilities in Navigator (#9617)
560653a5d7 Unwind partial transaction on mustFail (#9608)
f51145cb88 Support mapping from old to new party ids in daml exports (#9591)
89e46bf90b Address security vulnerabilities in //compiler/daml-extension (#9615)
fd62671e0b Introduce SINCE-LF-FEATURE in integration tests. (#9616)
48939b513b Address vulnerabilities in //language-support/ts/packages (#9614)
cf59246d44 Add support for daml exceptions for append-only indexer (#9609)
ab29f7c07c Move activeness check of globalKeyInputs into archive (#9610)
5c28de36cb Upgrade grunt to address cve (#9611)
22ba5fddd2 Remove builtin exception types from protobuf and ASTs. (#9595)
b09a95fb6f Adapt indexer to empty divulgences (#9598)
2fc7489e44 Filter divulgence to an empty set of parties (#9600)
ad45213b66 participant-integration-api: Avoid the serial EC in integration testing. (#9603)
f742a4334e Dpp 336 sandbox classic on append only (#9561)
e68bc0dff0 Mark reset service tests as flaky (#9602)
2176173ba2 Remove 1.dev-only things from LF 1.13 protofile. (#9599)
03034ec3bf DPP-359 Add indexer flow level benchmark (#9509)
5aa54017eb Drop todo for conversion of exception primitives (#9597)
968b5d8d39 KVL-921 Expose opentelemetry Context from Telemetry and TelemetryContext (#9573)
112c387e5a Refactor out setGlobalLogLevel into ContextualLogging (#9592)
e584fecd3c Drop com.daml.lf.engine.Event.collectEvents (#9596)
80b07da309 Only archive a key if it was brought into scope before (#9546)
7ea9340d64 Support old bash in daml-sdk-head (#9593)
45fbdefb06 Handle rollback nodes in protoNodeInfo (#9589)
42d189dbbe Support exceptions in Daml-LF encoder (#9590)
9c3913a3d6 LF: SBUKeyBuiltin clean up. (#9572)
5128206ebf Ledger API Server: Named threadpools (#9588)
26a53d8469 Add logging command line option to ledger http service (#9581)
409833f850 Address rollback todo in ancient migration (#9583)
a27b2c56bc Address more exception todos (#9582)
2040556dbb Support rollback nodes in replay adapter (#9584)
192a77a6d8 update compat versions for 1.13.0-snapshot.20210504.6833.0.9ae787d0 (#9575)
81d508b775 LF: make SBuiltins pure (#9580)
de80a6dc60 Drop ValueBuiltinException (#9576)
1c456be79d Release yet another 1.13 snapshot (#9574)
800989145b Disable migration tests on macos on PRs (#9571)
2e93de79a4 Fix recording exceptions in Spans, add unit tests [KVL-874] (#9544)
121ded3565 Accept a comma-separated list of parties for daml ledger export (#9568)
```
Changelog:
```
- [Daml Assistant] The assistant will now avoid
installing SDK versions concurrently, waiting
for the previous installation to finish before
starting the next installation (if still
necessary). This fixes a bug where the SDK
would become corrupted because two
installations were started concurrently.
http-json:
- add contextual id as logging context to distinguish different application runs in logs
- add request id as logging context to distinguish different http requests within an application run
- add for non-static endpoints trace logs which show how long processing it took in ns
- [Integration Kit] - Created the ledger-api-bench-tool prototype for benchmarking ledger transaction streaming capabilities
* [Daml export] Users can now define a mapping from parties in the
original ledger state to parties to be used when recontructing the
ledger state. The ``parties`` component of the argument to the
generated export script now takes a mapping from old party name to new
party name.
- [Ledger HTTP Json Service] Logging now also tells service name if log level was changed.
- [Ledger HTTP Json service] Logging can now be configured via the `--log-level` cli argument. Valid values are `error`, `warn`, `info` (default), `debug`, `trace`
```
CHANGELOG_BEGIN
CHANGELOG_END
* Release snapshot 1.14.0
changelog_begin
changelog_end
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: victor.mueller@digitalasset.com <mueller.vpr@gmail.com>
Note that these tests are switched off for now until Robert has
implemented the migration.
To avoid having to stuff too much in one PR the plan is to merge this
one first and then switch the flag once the migrations work.
changelog_begin
changelog_end
* check whether collection.compat is unused when compiling for Scala 2.12
- Instead of always suppressing warnings for collection.compat._,
we should only do it for Scala 2.13
- We can also reduce boilerplate by automatically adding this
option when both silencer_plugin and collection-compat are
present
CHANGELOG_BEGIN
CHANGELOG_END
* remove unused import
* remove another unused import
* remove even more unused imports
* missed compat dependency
* more missed compat dependencies
* missed compat dependency
* use scala_deps in scaladoc_jar
- #8423 inlined the major version expansion, but this seems to
have been prior to proper support by scaladoc_jar
* restore custom handling of participant-integration-api
- fixing scaladoc_jar isn't worth it for a single case, as with
deps vs scala_deps
