wildcard pattern matches bite once again. We clearly do not want to
use the committers here.
I tried to move most of the logic to PartialTransaction but
unfortunately moving the submitters to PartialTransaction doesn’t
quite work since as usual scenarios make things difficult because
submitters change at weird places and we reset the partial transaction
in weird places.
changelog_begin
changelog_end
Before this change, when converting a string which exceeds the max
length, the application return the error "too long" without saying what
the bounds are, requiring a user to lookup the bounds in code. Now, the
user gets the actual length that is enforced as part of the error
message.
CHANGELOG_BEGIN
CHANGELOG_END
This PR simplifies the internal of Speedy by factorizing SAny and
SAnyException. By side effect we also get the comparison of LF
AnyException values that was not implemented before.
CHANGELOG_BEGIN
CHANGELOG_END
Note that it only has rules for normalizing `create` and `exercise`, but that's because the LF spec only has `create` and `exercise` actions for now.
changelog_begin
changelog_end
* Normalize rollbacks: first draft
CHANGELOG_BEGIN
CHANGELOG_END
NormalizeRollbackSpec, WIP
WIP2
adapt to asVersionedTransaction
first stab at traversal for normalize-rollbacks... implement normalization rule #1
adapt existing testcase for rollback normalization rule #1
pluralize spec filename
temp disable rollback normalization
methodically test normalization rule #1
lots more tests
spec: check all 3 norm conditions
cleanup test/Shape code a bit
implement normalization rules #2 and #3 (using canonical types) and enable tests
add some comments and some 8020 todo markers
* add 2 more testcases from Sofia
* ensure tx produced when normalizing rollbacks has increasing node-ids when listed in pre-order
* enable rollback normalization in interpreter
* manage state functionaly for the created tx (counter & node-map)
* un-nest sub defs from normalizeTx (we can because we removed the mutable state)
* rename: force* --> push*
* introduce CPS for push functions
* introduce trampolines for push functions to be stack safe
* one more bounce
* ensure generated node-ids start from 0
* test that transaction node-ids start from 0
* add commets about pass1/2; move makeRoll (part of pass 1) earlier in file
* intro CPS for pass-1 over original tx
* intro trampolines for pass-1. everything is stack safe now
* clarify comment
* remove make stack-safe todo
* be more private
* factorize/share Trampoline implementation with previous implementation in speedy.Anf
* prefer Vector over List, for better algorithmic complexity
* make Trampoline private to lf
* Update LF spec for exceptions.
The changes here are:
* Removing the built-in exception types
* Changing the result in the operational semantics to include both an "exception thrown" case and a "fatal error" case.
* Cleaning up the semantics of create/exercise/etc and try/catch.
Not included in this PR is anything to do with built-in arithmetic exceptions. There's room to add it in the future (as a value of AnyException type), but I would do it in a follow-up PR.
changelog_begin
changelog_end
* finish renaming Throw cases
* Update type ordering for AnyException.
The name of some builtins will be exposed as part of the Exception
message. This PR, try to make conversion builtins more consistent and
more obvious, before we could not rename those.
This is part of #8020
CHANGELOG_BEGIN
CHANGELOG_END
* Split DA.Internal.Exception.Types into three packages.
I think this is a good idea because it means the user can disambiguate between these modules in daml. All three types define a `message` field, so if the programmer wants to refer to the field for one of these types specifically, it's nice to have a canonical way to disambiguate them (by module name).
Also the module names show up in the error message for uncaught exceptions. These names are a bit nicer than "DA.Internal.Exception.Types".
Decoupling these types just sounds like a good idea going forward.
changelog_begin
changelog_end
* buildifier-fix
* update isInternal
* Update tests
* Use more general pattern in CommandServiceIT
* Update DarReaderTest
* speedy: Compile new AnyException primitives
changelog_begin
changelog_end
* s/Contract/ContractError
* remove random eszett
* fix booleans
* Combine some cases!
* define and use getSException
* use SException directly in pattern match
For better or for worse, kvutils validation insists on treating a
negative input from a create different to a negative key lookup (and
to make things more annoying the first one will never blow up during
submission only during validation).
While, nobody seems to argue all that strongly that the current errors
are very sensible, we agreed to do it in two steps:
1. Expose enough information in `contractKeyInputs` so that we can use
it in kvutils while preserving the current error semantics.
2. Revisit contract key error handling across ledgers which is
currently an inconsistent mess.
changelog_begin
changelog_end
.
changelog_begin
changelog_end
* Throw on internal errors instead of setting ptx to aborted
The main purpose of aborted is to be able to get out a ptx in the
scenario service. However, for internal errors that makes no
sense. Users should never encounter them. And the use of `aborted`
here has swallowed these errors silently in a few places before which
is clearly bad.
changelog_begin
changelog_end
* Fix tryHandleSubmitMustFail
changelog_begin
changelog_end
* Remove version field from rollback node.
changelog_begin
changelog_end
fix transaction decode & testcases
reinstate/adapt testcases
fmt
rebase change to fix conflicts
* treat missing version on transaction node as being version minExceptions when computing tx version
* add 8020 marker for grepability
* reinstate Node.version to avoid logic duplication
* Revert "reinstate Node.version to avoid logic duplication"
This reverts commit 00dacb453d.
* remove Node.updateVersion
* remove comments with dubious utility
* Drop version check from TransactionPreprocessor
This is only supposed to be run on nodes that have been normalized via
de(serialization) so this is just useless shotgun validation.
changelog_begin
changelog_end
* Fix test
changelog_begin
changelog_end
* check whether collection.compat is unused when compiling for Scala 2.12
- Instead of always suppressing warnings for collection.compat._,
we should only do it for Scala 2.13
- We can also reduce boilerplate by automatically adding this
option when both silencer_plugin and collection-compat are
present
CHANGELOG_BEGIN
CHANGELOG_END
* remove unused import
* remove another unused import
* remove even more unused imports
* missed compat dependency
* more missed compat dependencies
* missed compat dependency
* use scala_deps in scaladoc_jar
- #8423 inlined the major version expansion, but this seems to
have been prior to proper support by scaladoc_jar
* restore custom handling of participant-integration-api
- fixing scaladoc_jar isn't worth it for a single case, as with
deps vs scala_deps
* Check byKey in transaction validation
No tests because validation has no tests in general :(
part of #7622
changelog_begin
changelog_end
* Fix preprocessing of transaction root nodes
changelog_begin
changelog_end
* Swap order SEScopeExercise and SBUBeginExercise
SEScopeExercise pushes the KCloseExercise continuation. However, the
corresponding transaction node is only inserted when SBUBeginExercise
finishes. This causes an issue if SBUBeginExercise crashes/throws an
exception.
changelog_begin
changelog_end
remove weird comment
changelog_begin
changelog_end
Reenable another test
changelog_begin
changelog_end
* Update failing test
changelog_begin
changelog_end
This PR includes :
- Adding ApiCommand to distinguish between generic command (that are
accepted by the engine) and command that are accepted by the ledger
API.
- Reimplement Canton's reinterpret method using commands instead of
node.
CHANGELOG_BEGIN
CHANGELOG_END
This PR fixes a very confusing and subtle bug in the handling of
submitMustFail in scenarios:
submitMustFail did not properly unwind the transaction. Afaict, this
only matters for contract id derivation in this context.
It also only matters for errors during interpretation not errors that
happen when committing to the ledger (global duplicate key errors).
Specifically what happens is that we end up leaking the exercise
context that we were in when we hit the failure into the next
submission. I’m not sure if this actually breaks anything (if it does,
doesn’t really matter, it’s scenarios) but it is definitely not what
should be happening.
I hit this when adding uck checks to the engine since that turns some
commit errors into interpretation errors and changed contract ids in
tests that really shouldn’t have changed.
changelog_begin
changelog_end
* Move activeness check of globalKeyInputs into archive
This is semantically equivalent but it seems slightly simpler and more
importantly it leads to slightly nicer semantics by reducing the
number of cases where we have to worry about accidentally skipping an
activeness check.
changelog_begin
changelog_end
* s/optCid/keyMapping/
changelog_begin
changelog_end
* review feedback
changelog_begin
changelog_end
* Filter divulgence to an empty set of parties
As @nmarton-da noticed painfully, we currently include divulgence to
an empty set of parties. While this is arguably not wrong it is at
least confusing and useless. The whole point of divulgence is to track
visibility. Divulging to an empty set of parties does not affect
visibility so it is not meaningfully different from no
divulgence. Therefore this PR filters it out and adds a doc comment
that the list of divulgees is always non-empty.
changelog_begin
changelog_end
* Fix tests
changelog_begin
changelog_end
This looks like an attempt at moving the conversion from transactions
to Ledger API events to the Ledger API. However, that attempt is not
used anywhere (checked pkvutils, corda & canton).
I don’t think we benefit from having an unused method sitting around
here so this PR kills that method.
changelog_begin
changelog_end
* Only archive a key if it was brought into scope before
This PR implements the semantics we agreed on in
https://github.com/digital-asset/daml/pull/9472#discussion_r620097844.
Before, archive always marked the key as inactive in `keys`. This is
weirdly inconsistent with other operations. For example, a regular
fetch does not bring a key in scope in `keys`. This PR changes this to
a more consistent model where `keys` is modified exactly under the
following circumstances:
1. A create with a key always overwrites.
2. An archive will only mark a key as inactive if there was an entry
in `keys` with the same contract id.
3. Lookup/fetch by key first check if we currently have an entry in `keys`.
If so, we use that.
If not, we need to ask the ledger. We first check if we have a
ceched entry in `globalKeyInputs`. If we do use that.
If we don’t, we ask the ledger.
If the result is not active or not visible we proceed updating `keys`
and `globalKeyInputs`.
If the result was a cid, we check if it’s still active.
changelog_begin
changelog_end
* Review comments
changelog_begin
changelog_end
* Apply suggestions from code review
Co-authored-by: Remy <remy.haemmerle@daml.com>
* fmt
changelog_begin
changelog_end
Co-authored-by: Remy <remy.haemmerle@daml.com>
This PR clarifies that NodeInfo is only intended for actions by
renaming it to ActionNodeInfo and correspondingly also changes
protoNodeInfo to protoActionNodeInfo and makes it return a `Left` for
rollback nodes.
changelog_begin
changelog_end
This PR some minor clean up for #9537 and #9538.
In particular it:
- Remove the unecessary class KeyOperation (SBUKeyBuiltin is already a
somehow a key Operation)
- Rename and refactor methods handleKeyFound, handleKeyArchived,
cidToSValue and cidToSExpr to gave them a more obvious meaning.
- Replace immutable.Map#+ by immutable.Map#updated (cosmetic)
- Replace trait by abstract class (cosmetic)
CHANGELOG_BEGIN
CHANGELOG_END
* Address more exception todos
A bit of a kitchen sink PR to address a bunch of the trivial todos
that didn’t seem worth splitting out into separate PRs.
changelog_begin
changelog_end
* Revert changes to TransactionSpec
changelog_begin
changelog_end
As discussed, we don’t want to expose this via serializable values at
least for now (and it’s not exposed on the ledger API anyway) so this
PR drops the type.
changelog_begin
changelog_end
* Cache global key inputs
This PR changes speedy such that it will query at most once for a global
key. This is a prerequisite for a (sensible) duplicate key detection
and for sensible multi-key semantics.
Note that there is one outstanding issue here:
As a general rule, we only want by-key operations to bring a key in
scope not just a random operation like an exercise or a fetch that
happens to have a key.
However, for consuming exercises this is currently not the case. They
always bring the key in scope. The plan here is for an archive to not
block a further lookup and instead add an activeness check to the
lookup afterwards.
changelog_begin
changelog_end
* Update daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/speedy/SBuiltin.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
* s/CtrlImportValue/SEImportValue/
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
* Use a custom Scala type instead of Option[Value.ContractId]
changelog_begin
changelog_end
Co-authored-by: Remy <remy.haemmerle@daml.com>
No need to treat rollback nodes any differently here. A lookup for a
cid -> node mapping that does not point to a create node is borked and
we turn it into notfound.
changelog_begin
changelog_end
This is only used in one test where we can just reuse the scenario
ledger. I really don’t want more broken ledger implementations.
This also removes the need to address the TODO around rollback nodes.
changelog_begin
changelog_end
* Add Ledger API test tool tests for exceptions
changelog_begin
changelog_end
* Update daml-lf/language/daml-lf.bzl
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* Address review comments
changelog_begin
changelog_end
* Shuffle around test
changelog_begin
changelog_end
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* Document and test multi-key semantics
Canton relies on the Engine working correctly even in a setting where
we do not have multiple keys. So far this worked by accident but the
semantics of this are rather unclear. To make things worse, Canton
upgrades rely on those semantics being stable so we really do care
about the choices we make here.
This PR adds a bunch of tests as an executable documentation of the
current behavior. However, we do not provide stability guarantees for
the current behavior and therefore these tests can be changed as
needed. But at least we are aware of those changes rather than doing
them by accident.
changelog_begin
changelog_end
* Update daml-lf/engine/src/test/scala/com/digitalasset/daml/lf/engine/EngineTest.scala
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* Update daml-lf/tests/MultiKeys.daml
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* Update daml-lf/tests/MultiKeys.daml
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* Share code between SBUFetchKey and SBULookupKey
Those do almost the same and the logic is growing increasingly complex
so I really want to share it. This also makes it easier to see the
things that are different.
Along the way I’ve also discovered a bug:
SBUFetchKey called crash where it should produce a proper error that
can be caught by submitMustFail. Not going to resolve that in this PR
though so I just added a TODO.
changelog_begin
changelog_end
* review comments
changelog_begin
changelog_end
* Move builtin exceptions to a stable package.
This PR moves the built-in exceptions (GeneralError, ArithmeticError, ContractError) to a stable package in daml-prim, in preparation for removing them from LF proper.
Part of #8020.
changelog_begin
changelog_end
* Update tests
* rename the module
* couple that got away
* update hash
* Update dar reader test.
* typo
* questionable lint
* Allow missing exercise results when version >= minExceptions
imp:
- adapt encode & decode implementations
- add version dependant checks for version < `minExceptions`
testing:
- update scalacheck `ValueGenerators` to produce exercise nodes with the missing result value guarded by the versions
- update `normalizeExe` in `TransactionCoderSpec`
- update `transactionVersionGen` to be inclusive on upper bound
- unhappy-path test for encode/missing-exercise-result when not allowed
- unhappy-path test for decode/missing-exercise-result when not allowed
changelog_begin
changelog_end
Just some small changes to take into account the new ledger model transaction structure (see #9396). This PR doesn't fix all the issues with the LF spec not matching the ledger model, but it tries to avoid widening that gap.
* Removed itr, iexercise which we don't need.
* Removed the exception type and value from the rollback node.
* We're not calling rollback nodes ledger actions anymore, so separate it out.
Part of #8020
changelog_begin
changelog_end
* Switch from `@silent` to `@nowarn`.
This annotation is native to Scala 2.12.13+ and 2.13.2+. It replaces
most usages of `@silent`.
I had to get creative about a couple of use cases that didn't work.
Specifically:
1. Suppressing deprecation warnings works, but Scala 2.12 erroneously
complains that the `@nowarn` is unnecessary. I had to suppress
this warning too with `-Ywarn-unused:-nowarn`.
2. I can't seem to suppress the warning, "The outer reference in this
type test cannot be checked at run time." Instead, I have
refactored the code to remove the warning.
We still need to use the silencer plugin to suppress some warnings about
unused imports (because of compatibility between Scala 2.12 and 2.13),
but this means we no longer need the library, and therefore it is not a
transitive dependency that downstream consumers need to worry about.
CHANGELOG_BEGIN
CHANGELOG_END
* Add some comments around `@nowarn` support.
* language-support/scala: Fix a warning suppression.
* Revert to the default warnings.
Compatibility was complaining.
* Add a helper for getting all inactive contracts to Transaction
This comes up in a few places and rather than inlining the
implementation everywhere, I’d rather rely on this.
changelog_begin
changelog_end
* 2.13
changelog_begin
changelog_end
CHANGELOG_BEGIN
- [Scala bindings] ``InsertOrdMap`` is deprecated; it was never
needed for the release version of ``GenMap``. Use an ordinary
``Map``, or ``Primitive.GenMap`` for Scala codegen, instead.
CHANGELOG_END
* Fix validation of rollback nodes
To my own embarassment there is no test in this PR. We don’t have any
direct tests for validation atm. We absolutely need some and I already
discussed with Remy that this is something we should focus on once
exceptions are done but for now I’ll stick to being sad.
changelog_begin
changelog_end
* Fix bug
changelog_begin
changelog_end
* Check visibility for by-key operation of local contracts
fixes#9454
I tried out two approaches for this:
1. The one here where we add a new callback. This has the advantage
that the engine remains oblivious to visibility checks. They are all
done outside and the engine doesn’t even know about the reading
parties.
2. Make the engine aware of the reading parties. A start of that is in
#9458.
Both work in principle but I ended up going for 1 in the end. Doing
half of the visibility checks outside the engine and half inside just
seems worse than the current state.
changelog_begin
- [Daml Engine] Fix a bug where it was possible to
fetch/lookup/exercise a local contract by key even if the reading parties
are not stakeholders. See #9454 for details.
changelog_end
* Disable new test on Canton
changelog_begin
changelog_end
* Exclude from compat tests
changelog_begin
changelog_end
* s/LocalLookup/LocalFetch/
changelog_begin
changelog_end
* Address review
changelog_begin
changelog_end
* Extend Transaction proto format for rollback nodes.
- proto format
- encode/decode
- testcase
- always `allowRollback` from scalagen testing; and so remove control flag
- two more `8020` TODOs, for other code which matches `node.getNodeTypeCase` and sees `ROLLBACK`
changelog_begin
changelog_end
* add comments about version of rollback nodes in transaction.proto
* check rollback node version during encode & encode
* factor out code to decode children for exercise & rollback node
* test decode check for unexpected rollback node in old version
This focuses on the semantics rather than the display in Daml Studio
which needs more work (and seems not all that important at this
stage).
This already uncovered a bug which also applies outside of scenarios:
The consumedBy field was not affected by rollbacks which breaks the
mustBeActive check in partial transactions. This PR fixes this by
caching on try and restoring on rollback.
changelog_begin
changelog_end
* KV: do not use "Effects" part of InputsAndEffects
The PR is based #9429.
changelog_begin
changelog_end
* Factor out helpers
changelog_begin
changelog_end
* Apply suggestions from code review
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* review comments
changelog_begin
changelog_end
* fmt
changelog_begin
changelog_end
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
The current generators are a bit too dumb and relatively
frequently (~1/10) fail to generate 5 successful results before
failing forever. This PR changes things slightly to avoid this. We
could be even more clever and draw the second version after having
selected the first one so we cannot even draw the same but that
doesn’t seem worth the complexity for now.
changelog_begin
changelog_end
Local contract IDs are collected in exercise children during preprocessing transaction for replay.
CHANGELOG_BEGIN
- [Engine] Fix contract ID freshness check when validating transaction
CHANGELOG_END
* Pattern matching for RoundingMode
- Fix the order of RoundingMode constructors in GHC.Types to match the LF built-in order. Try to match this order across all code and documentation, and added a test for this order.
- Implements pattern matching for RoundingMode. The added machinery could also be useful for solving #5753 in the future.
- Implements Show instance for RoundingMode. (Mainly so we can use them in tests.) Moved BigNumeric Show instance to GHC.Show.
changelog_begin
changelog_end
* mkScrutineeEquality typo
* fix roundingModeLiteralMap order
* Use custom type for building case body
* Factor GeneralisedCaseAlternative into GeneralisedCasePattern
* Fix finalize
* Remove unused bindings
There have been a few changes and the names in the current version are
no longer valid.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Expose rounding modes as constructors.
This PR exposes the rounding modes as constructors for RoundingMode. Pattern matching for RoundingMode is not implemented and not critical (will open a separate issue).
This PR also adds documentation for BigNumeric.
changelog_begin
changelog_end
* Fix rounding mode literals
* Update data-types.rst
* expose constructors
* expose constructors (part 2)
* Update compiler/damlc/daml-prim-src/GHC/Types.daml
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Try to improve `Numeric n` description.
* Add message for RoundingMode match and a test.
* Restrict RoundingMode test to 1.dev
* Update version numbers to 1.13
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
This doesn’t really make that much sense in there and it makes some
things like Daml Script harder because I need to duplicate the logic
to get the exception message.
changelog_begin
changelog_end
* Improve TX normalization.
Extend `foreachInExecutionOrder` / `foldInExecutionOrder` to handle rollback nodes. Two new args control behaviour when entering/leaving rollback nodes. Caller can control (with a bool) if the traversal continues under exercise/rollback nodes. Update existing callers.
Use `foldInExecutionOrder` to define `reachableNodeIds`. And test.
Use `foldInExecutionOrder` to improve the behaviour of `NormalizeTransaction` so that all nodes within a rollback node are dropped. Also completely reworking the spec/test for the new behaviour.
changelog_begin
changelog_end
* fix build
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
- temporarily controlled by `allowRollback`
- set `allowRollback = false` for tests which dont yet work. Marked by `TODO`
- `allowRollback = true` used in `NormalizerSpec` tests
- implement `version` support in `NodeRollback` (so `NormalizerSpec` test will pass)
changelog_begin
changelog_end
* Add BigNumeric literals in Daml.
Adds support for BigNumeric literals (when available). Only literals that will fit in a single Numeric are supported for now.
I introduced the `IsNumeric t` typeclass because this is an easy way to restrict `fromRational` without moving it into its own typeclass. (Moving it into a typeclass causes some gnarly problems with the specializer -- it starts creating references to GHC.Real in an attempt to optimize the invocation, and that's a problem because GHC.Real doesn't exist in LF.)
I then added the "fromNumeric" and "fromBigNumeric" conversion functions in that class, since they seem really convenient, and it also means we could in the future make it so that any type that implements `IsNumeric` gains literals (via Numeric or BigNumeric literals). This would improve a lot of Numeric code, since it eliminates the need to annotate types so often! But for now only Numeric and BigNumeric literals are supported.
changelog_begin
changelog_end
* lint
* Add test for too large bigNumeric
