* Test authorization within Exercise choice.
CHANGELOG_BEGIN
CHANGELOG_END
* fix build following rebase
* 4x shouldBe a Right
* test for exercise within exercise
* implicit Name.assertFromString
* def --> val. massive speedup!
* be more private
* implicit Identifier creation
* use implicit for Party and ChoiceName creation
* comment test purpose
* delete stray text; fix build
* Typecheck nested LF type applications more efficiently
This improves performance typechecking in the Scala LF typechecker by
matching on nested foralls & type applications which allows us to
check them with only a single substitution as opposed to the previous
state of doing one for each type application.
This is about a 25% improvement on typechecking performance of
GHC.Classes from 200ms to 150ms.
changelog_begin
changelog_end
* interfaces: Do some TODOs
- Add uniqueness check between fixed choices and virtual choices in
haskell decoder.
- Encode interface methods and fixed choices in scala encoder.
- ExprIterable for interfaces.
changelog_begin
changelog_end
* scalafmt
* [Self-service error codes] Refactored for Canton integration
* Introduces ErrorCodeLoggingContext which is used to abstract away the logging tech stack
CHANGELOG_BEGIN
CHANGELOG_END
* Moved all error definitions in //ledger/error
* Formatting
* Docs and remove of implicit conversion to DamlErrorCodeLoggingContext
* Fix compilation issues
* Fix tests
* Addressed Pawel's review comments
* Addressed Robert's review comments
* interfaces: introduce TemplateOrInterface class in stdlib
The template typeclass is to strong for many applications. The new
constraint `TemplateOrInterface` only contains the methods to convert
contract IDs and choices.
CHANGELOG_BEGIN
CHANGELOG_END
* ghc-lib update, interface script tests
* pinned stackage on unix
* added missing implementation in preprocessor
* added test
* fixing tests
* remove Iface type
* pinned stackage windows
* make sure createAndExercise is not called on interfaces
* LF: clarify compilation of choice Body.
This is a purely cosmetic choice.
We move up the intro of the choice arg binder to make it clear it
affect the compilation of every expression in the body of a choices (
namely controllers, observers, and update expression).
CHANGELOG_BEGIN
CHANGELOG_END
* formatting
- enable NonUnitStatements, and find a useless statement.
- add missing `final`s,
- replace `{ ... }` by `( ... )` for one statement anonymous function
- drop useless `new` for case class.
- replace `sealed class X { ... object Y extend X` by
`class X private[Z] { ... val Y = new X` to avoid unecessary class
alocation.
- move invariant check inside the body of some case class instead of
the companion object.
- other cosmetic changes.
CHANGELOG_BEGIN
CHANGELOG_END
* interface methods: Scala Typechecker
Part of #11006.
I tried to match the behavior of the Haskell typechecker.
changelog_begin
changelog_end
* scalafmt
* Fix type
* interface methods: Scala AST
Part of #11006
This PR takes care of the Scala AST and decoder.
It leaves the encoder, type checker, and speedy for later.
changelog_begin
changelog_end
* scalafmt
* AstRewriter
* interface methods: Speedy
Part of #11006
changelog_begin
changelog_end
* scalafmt
* Remove InterfaceMethods test
* Update daml-lf/archive/src/main/scala/com/digitalasset/daml/lf/archive/DecodeV1.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
Co-authored-by: Remy <remy.haemmerle@daml.com>
* interface methods: Scala AST
Part of #11006
This PR takes care of the Scala AST and decoder.
It leaves the encoder, type checker, and speedy for later.
changelog_begin
changelog_end
* scalafmt
* AstRewriter
* Add NonUnitStatements warning in daml-lf/transaction & fix using discard or similar.
CHANGELOG_BEGIN
CHANGELOG_END
* address comment
* prefer parens to braces for single statement map/foreach bodies
* interface methods: Add protobuf
Adds protobuf definitions for interface methods and calling them.
Encoders/decoders just ignore the extra stuff or error out.
Part of #10810 (maybe)
changelog_begin
changelog_end
* Update issue numbers where appropriate
* update stable protos
(Also: generalize type of `enrichTransaction` to work for _submitted_ and _committed_ transactions & adapt existing callers)
This change paves the way to remove the `transactionNormalization` flag, and always normalize transactions coming out of the engine.
CHANGELOG_BEGIN
CHANGELOG_END
1st attempt. Causes package recompilation (bad!).
CHANGELOG_BEGIN
CHANGELOG_END
fix build
Change ValueEnricher interface to work without passing an Engine
ValueEnricher has optional preprocessor
simplify new interface to ValueEnricher: caller passes translateValue function
Part of #10810
- Implemented ToInterface, FromInterface in speedy
- Added a test that exercises and fetches an interface
- Fixed a bug in SBUChoiceInterface
Hey, interfaces work now!
changelog_begin
changelog_end
fixes#10977
Turns out assertions are good unless they’re wrong …
This only affects scenarios, the engine never looks at the callback.
changelog_begin
changelog_end
* ifaces: name collision, typecheck fetch/exercise
This adds name collision detection and adds typechecking for
fetch/exercising of interface instances.
CHANGELOG_BEGIN
CHANGELOG_END
* turn on exercises in InterfaceDesugared test case
* Update compiler/daml-lf-tools/src/DA/Daml/LF/TypeChecker/NameCollision.hs
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* interfaces: scala typechecker implementation
This is the scala side of the lf typechecker for interfaces.
CHANGELOG_BEGIN
CHANGELOG_END
* added collision check
* added exercise/fetch typechecking
* review suggestions
* added todos for collision/typing scala tests
* Extracted common error implementations
* ErrorGroups
* TransactionError hierarchy
* LedgerApiErrors
* PackageServiceError
* ProtoDeserializationError
* PruningServiceError
* SubmissionErrors
CHANGELOG_BEGIN
CHANGELOG_END
* Adapt extracted error groups and definitions to adhere to local tech stack:
* Use //ledger/error:error core API
* Use DAML SDK logging stack
* Manual rebase to latest changes from Canton
* Extracted RejectionGenerator from Canton
* Adapted RejectionGenerator
* Added //ledger/error:error to artifacts
* Pass correlationId to errors
* Pass parameters as implicits to shave some lines and improve readability
* Workaround for encountered Scala/JDK8 bug
* Addressed Ratko's review comments
* Separate exercise & fetch for interfaces from templates
part of #10810
changelog_begin
changelog_end
* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Base.hs
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Base.hs
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
99% of our usecases use Value[ContractId] so this PR just fixes it.
The few other usescases are:
1. Value[Nothing] which we use for keys. This is technically more
precise but we benefit very little from it.
2. Value[String] mostly because in a few places we are lazy.
We don’t have any code which benefits from being polymorphic in the
contract id type.
changelog_begin
changelog_end
Following #10763, we drop the ad-hoc builders for `FrontStack`.
* Building a `Fronstack` from individuals elements should be done with
standard scala buidler.
* Building a `Fronstack` from a `TraversableOne` should be done with
the scala 2.13 `.to(FrontStack)` methd
* Building a `Fronstack` from a `ImmArray` should be done with the
`toImmArray` method.
CHANGELOG_BEGIN
CHANGELOG_END
* interfaces: protobuf encoder haskell side
This is the implementation of the protobuf encoder on the haskell side.
CHANGELOG_BEGIN
CHANGELOG_END
* missing decoder cases for added interface data constructor
* Enhance InMemoryLedger to use the ValueEnricher.
Allowing SandboxServer to run Engine with the default transactionNormalization=false
CHANGELOG_BEGIN
CHANGELOG_END
* enrich only when responding to verbose API queries
Since we switch to scala 2.13, ImmArray companion object extends
`Factory`. Hence:
- the `apply` methods of `ImmArray` override the one from `Factory`
- we can use the notation `.to(ImmArray)` to convert an `Iterable` to
`ImmArray`
This PR drops those `apply` ImmArray. Conversion from Iterable to
`ImmArray` should use the `.to(ImmArray)`.
CHANGELOG_BEGIN
CHANGELOG_END
This gets us into a state where PartialTransaction always has the
authorization context which is much more sensible than having the
committers in speedy while the rest is in ptx.
The reason for having the split in the first place was the scenario
service but now that we create a new speedy machine per submission
that’s no longer a valid reason.
changelog_begin
changelog_end
Create normalized TXs when a partial TX is finalised.
Except in limited cases! (i.e for scenario-runner, sandbox)
CHANGELOG_BEGIN
CHANGELOG_END
normalize values in the engine as they are converted from speedy-values
fix 2.12 build
backout redundant change
ensure byKey field is correctly normalized when constructed by engine
rename flag: valueNormalization -> transactionNormalization
improve comment
delete commented-out code
rename: toValueNorm --> toNormalizedValue
rename: (SValue.) toValue --> toUnNormalizedValue
revert changes to ptx so that the interface to insertCreate() etc is Value-based (not SValue-based)
improve comments
respell: toUnNormalizedValue --> toUnnormalizedValue
fix build
This PR makes possible to check for contract IDs suffix during
preprocessing.
This is the first part of the task 3 described in #10504.
CHANGELOG_BEGIN
CHANGELOG_END
This PR enforces that LF Comparisons (both Ordering and Equality)
fails at runtime if comparing local vs global CID.
This corresponds to the second task of #10504.
CHANGELOG_BEGIN
CHANGELOG_END
* Upgrade to a newer canton version (post 0.27.0 snapshot version)
with canton-community configuration that supports higher throughput.
changelog_begin
changelog_end
* Disable flaky reject DeeplyNestedValueIT:Reject tests that time out half the time
As stated in #10504 the contract ID freshness check cannot be
implemented correctly in general.
This PR drops the support for this (buggy) check.
This corresponds to the fist task of #10504.
CHANGELOG_BEGIN
CHANGELOG_END
* build correctly versioned empty tx
* dont call normalizeTx from Engine.validate
* dont recalc the tx-version in normalizeTx; just assume it is correct
CHANGELOG_BEGIN
CHANGELOG_END
* Normalize transactions & values as a separate pass. Use for simpler defintiion of isReplayedBy.
CHANGELOG_BEGIN
CHANGELOG_END
normalize transaction version
* remove stray import from bad merge which breaks scala 2_12 build
* change isReplayedBy to only norm its RIGHT (replay) argument
* add forgotton normalization for ValueEmum
* switch to use existing value normalization code (remove my newly coded duplicate code)
* normalize submittedTransaction before calling engine.validate
* dont call normalizeTx from Engine.validate
* *do* call normalizeTx from Engine.validate
* Upgrade Scalatest to v3.2.9.
Because of some coupling we also have to upgrade Scalaz to the latest
v7.2 point release, v7.2.33.
The Scalatest changes are quite involved because the JAR has been broken
up into several smaller JARs. Because Bazel expects us to specify all
dependencies and doesn't allow transitive dependencies to be used
directly, this means that we need to specify the explicit Scalatest
components that we use.
As you can imagine, this results in quite a big set of changes. They
are, however, constrained to dependency management; all the code remains
the same.
CHANGELOG_BEGIN
CHANGELOG_END
* http-json-oracle: Fix a Scalatest dependency.
* ledger-api-client: Fix a Scalatest dependency.
* LF 1.6 ledger export integration test
Test that Daml ledger export script handles DALF packages in LF version
1.6. These packages don't contain metadata, meanint `--package` flags
have to use the hash rather than the package name, and they contain no
type class instances, meaning the export script needs to contain newly
defined instances for type classes required to issue ledger commands in
Daml script.
changelog_begin
changelog_end
* Expose unconstrained Daml script commands
This will be used in Daml ledger exports of contracts defined in Daml LF
versions before 1.8. These versions did not include typeclass instances,
meaning that instances such as HasTemplateTypeRep need to be recreated
in the export Script. Not all instances required in the `Template t`
constraint can be recreated at the use-site. E.g. `HasSignatory`. By
avoiding these kinds of constraints it is possible to handle these cases
in Daml ledger exports anyway.
changelog_begin
changelog_end
* Identify all templates with missing instances
changelog_begin
changelog_end
* Use internal*Cmd on templates missing instances
changelog_begin
- [Daml export] Daml ledger export now handles templates in packages
using LF versions 1.7 or older. These package versions don't include
type class instances and Daml ledger export needs to generate
replacement instances in the generated script. The generated script
uses less type-safe versions of Daml script ledger commands.
changelog_end
* Add encodeType to encode Ast.Type in ledger export
This will be required to encode HasContractKey instances.
changelog_begin
changelog_end
* Simplify newline handling in encodeExport
* Encode missing template type class instances
* ZIP entries must use forward slash as path separator
* Throw error on encode of Any
Addressing
https://github.com/digital-asset/daml/pull/10526#discussion_r685807454
* Qualify choice using argument type
Addresses review comment
https://github.com/digital-asset/daml/pull/10526#discussion_r685817745
* Use lf.language.Util
Addresses
https://github.com/digital-asset/daml/pull/10526#discussion_r685827150
* Factor out header comment in tests
Addressing review comment
https://github.com/digital-asset/daml/pull/10526#discussion_r685830832
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Expose set of stable-packages package-ids
Needed in Daml ledger export to decide whether to depend on a package
without metadata or not.
changelog_begin
changelog_end
* Handle package dependencies without metadata in Daml export
This addresses the immediate issue reported in
https://github.com/digital-asset/daml/issues/10435.
DALF before LF version 1.8 don't contain metadata such as the package
name and version. However, Daml ledger export assumed that metadata was
available to generate `--package` flags for data-dependencies.
This change generates `--package=<hash>` flags for such dependencies on
packages that don't have metadata available.
This change requires additional care when checking if a package is a
stable-package. As the package name is not available without metadata we
cannot just check against names such as `daml-stdlib`. Instead this
change uses the list of stable-package package-ids exposed by
`com.daml.lf.language.StablePackages.Ids`, which was introduced in
0da814d250 (diff-208c6948c7c7f2b582faa8e4792bfe78b231188881947ad0cdd5046e0b9a40e7)
Note that this does not fully resolve#10435. Building a generated Daml
ledger export script will fail on missing instances of standard classes
such as `HasTemplate` because packages prior to LF version 1.8 don't
contain type-class instances. This will be addressed in a separate PR.
changelog_begin
changelog_end
* Fix Scala 2.12
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
part of #9974
- rename `msg` field to `message`
- drop `engine.Error.Validation.Generic`
- rename argument `where` to `location`
- Builtin set the `dependsOnTime` before throwing the callback `SResultNeedTime`
CHANGELOG_BEGIN
CHANGELOG_END
* logging-entries: Make `LoggingEntries` a non-case class.
There's no reason for it to need `equals`, etc.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-api-domain: Convert commands into a logging value.
Instead of having a function, let's use `ToLoggingContext`.
This also adds a couple of missing items, and always logs `workflowId`.
* participant-state: Convert updates into a logging value.
Instead of having a function, let's use `ToLoggingContext`.
This changes some of the logging context structure, but otherwise
everything remains the same.
* Make sure Scaladoc is lined up for modified code.
* Stable packages bypass version restrictions.
Part of #10029
changelog_begin
changelog_end
* Add missing bypass
* scalafmt
* Fix tests
* fix script runner
* Fix scenario service
* Dont make stable packages configurable
* unused import
* revert unnecessary formatting changes
* Added a StableOnly engine mode and a conformance test that uses it
* buildifier-fix
* Inline LanguageVersions.StableOnly
* scalafmt
* Remove EngineMode in favor of representing the version range directly
* Add -unsafe to flag name
* Apply suggestions from code review
We drop this check for the following reasons:
* Its only usage was inside the transaction builder, this is subsumed
by the check we added in the SValue to Value translation (#10370)
* It seems to origianl check more that nesting (and so to be
overcomplicate)
* It was used in non-consitent way. It was used to check create
arguments and choice arguments, but niether choice result nor
contract key.
CHANGELOG_BEGIN
CHANGELOG_END
The conversion of SValue to Value already ensures the resulting value
has a serializable type. Here we add a check to ensure it does not
overpass the maximum allow nesting.
* Limit length of package ids to 64 characters
In practice, you cannot use a package id that is not a valid sha256
outside of a test anyway since we validate them. However, we didn’t
enforce it at the daml-lf level which is a bit annoying.
This PR fixes this and adds a length restriction of 64
characters (i.e. 256 bytes for sha256).
We could go a bit further and also restrict the characters to valid
sha256 hashes (i.e. 64 character hex strings). I don’t feel all that
strongly about that either way. We use other characters in tests but
fixing that shouldn’t be particularly hard either. The extra
characters don’t seem to cause problem so far, so I think it’s fine to
keep that.
No changelog entry since I don’t see how a user can be affected by this.
changelog_begin
changelog_end
* Update daml-lf/encoder/src/main/scala/com/digitalasset/daml/lf/archive/testing/DamlLfEncoder.scala
* participant-state: Remove the aliases to `Ref.LedgerString`.
Use the aliases directly in `Ref` instead.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-lf/data: Improve comments on ID aliases.
* daml-lf/data: Fix a typo in a comment.
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* Pass commitLocation along in engine warning log.
changelog_begin
changelog_end
* Use commitLocation in diagnostics.
* scalafmt
* update divulgence test
* Fix commitLocation and expected diagnostic locations
* daml-lf/data: Move ID aliases to `Ref` from _ledger-api-common_.
This allows us to remove a lot of dependencies on _ledger-api-common_,
and use these aliases in other places where that module is not used.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-integration-api: Remove an unused import.
* http-json-oracle: Remove `ledger-api-common` as a dependency.
* bindings-rxjava: Remove a now-unused dependency.
* Remove Optional[Location] field from transaction nodes.
xoptLocation
working but very messy!
cleanup PartialTransaction code
pass locationInfo in CompleteTransaction
dont pass locationInfo in SumittedTransaction
pass optLocation to CheckAuthorization
temp reinstate xoptLocation in ExerciseContextInfo
cleanup Conversions.scala
remove Optional[Location] field from transaction nodes.
CHANGELOG_BEGIN
CHANGELOG_END
* fix bad merge
* fix build after merge
* rename
* doc comment, and question
* LF: change type from Try to Either in archive module
This is the first part of restructuring errors in archive module.
This is part of #9974.
CHANGELOG_BEGIN
CHANGELOG_END
* Apply suggestions from code review
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* remove type alias
* apply stephen suggestion
* fix after rebase
* fix test
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Separate traces from warnings in engine.
I decided to separate the engine warnings from the tracelog after all,
because I think it will make testing and maintenance easier in the
long run.
Part of #9947, follow up from #10179
changelog_begin
changelog_end
* scalafmt
* Apply suggestions from code review
Co-authored-by: Remy <remy.haemmerle@daml.com>
* dont use case class for WarningLog
* revert TraceLog changes from last PR
* Scala 2.12 doesnt have ArrayBuffer.addOne :(
* remove isWarnEnabled check
Co-authored-by: Remy <remy.haemmerle@daml.com>
* LF: Make DarReader ZipEntries immulatble
CHANGELOG_BEGIN
CHANGELOG_END
* optim
* Revert "optim"
This reverts commit 3b25a8c68b.
* a function should not close a stream it didn’t open
* typo
* we do not support dar without manifest !
* LF: Simplify archive reader.
- decouple Reader and Decoder
- introduce case class to handle hash, proto payload, and version
CHANGELOG_BEGIN
CHANGELOG_END
* Address Moritz' review
* cosmetic
We probably want to start displaying the full ptx on the client side
but for now this at least moves things out of speedy and into the
rendering layer where they belong.
It also fixes IncompleteTransaction to unwind properly so the roots
are really the roots.
changelog_begin
changelog_end
* Use PartialFunction for more concise code.
CHANGELOG_BEGIN
CHANGELOG_END
* tweak to use guards
* blind fix for 2.12
* try fix 2.12 build
* loose if-guard parens
* fixes for 2.12 build (should actually work this time!)
* Add divulgence warning and test in script service.
Part of #9947, building on the key visibility checks from #10136
This PR adds a divulgence warning inside the traceLog.
I wasn't sure whether:
1. these warnings should be kept in a separate structure from the
traceLog, and therefore transmitted separately, or
2. these warnings should be kept together with traces, but the severity
should be tracked and also transmitted over grpc, and warnings should
be logged as warnings instead of as debug messages, or
3. these warnings should be kept together with traces, but
logged as warnings instead of debug messages,
4. these warnings should be treated exactly like traces
I'm leaning toward #2, but this PR implements #3.
This PR tests the warning via the script service tests.
changelog_begin
changelog_end
* scalafmt
* Address Moritz's review
* Rename traceLog.add to traceLog.addDebug
* fix test
* Add test using exercise
* add single transaction test
* Testing for Validation.isReplayedBy
changelog_begin
changelog_end
try fix scala 2_12 build
fetch node: samples and tweaks
list sig/insig field tweaks side-by-side so coverage is easier to eyeball
exerciseResult tweak sig & insig
add sig/insig tweak for byKey of Fetch node
add sig/insig tweak for key of Fetch node
must use ImmArray.empty instead of ImmArray() for scala 2.12
another blind attempt to fix scala 2.12
use for/yield instead of flatMap for list-comprehensions
test sig/insig tweaks to LookupByKey nodes
wont attempt to tweak rollbacks
(and shorten trailing dash in comment banners)
comment about how Tweak testing is organized
dont dep/import scalacheck (not used!)
no wild imports
* val --> private def
* def --> val
* prefer fail to println/assert(false)
* daml-lf/data: Truncate party names in log output, on request.
The party name can grow quite long, so we offer ledger implementors the
opportunity to truncate it in structured log output.
Unfortunately, because we use Logback through the global
`LoggerFactory`, there is no place to inject logging configuration. This
means we also need to use global, mutable state to configure logging
output. I have added a `LoggingConfiguration` class+object in Daml-LF
Data, which may not be the best place, but I can't think of a better
one right now. I suggest we leave it there until it has reason to grow,
at which point we may want to move it.
CHANGELOG_BEGIN
CHANGELOG_END
* logging-entries: Make `ToLoggingValue` mixin-able.
* participant-integration-api: Truncate parties in filters when logging.
* participant-integration-api: Cast to `Party` for logging.
Invalid input should not break the request at this point. No assertions.
* daml-lf/data: Move `Party to LoggingValue` to a new package.
This avoids the transitive dependency issue most of the time.
* daml-lf-data: Move the `Identifier` logging to another package.
Again, reduces the need for transitively depending on _logging-entries_.
* WIP: reinterpret must fail
changelog_begin
changelog_end
EXPLORE: add x/y prefix to reinterpret
MiniEngineTest.scala
MiniTests.daml
setup reinterpret testing one step down, to be able to see rollback nodes
MiniEngineTest.scala - refactor yreinterpret
first minimal reinterpret test
setup testcase for reinterpret on a command which throws
catch exception during reinterpret; and test
introduce type for Validating; avoid boolean blindness
remove exploratory MiniTests
remove exploratory x/y prefix on reinterpret
question
fix build
* small cleanup
* add `unsafeCompileForReinterpretation`; preserve interface of existing `unsafeCompile` (command list)
* remove the Validating type (not needed for this work)
* undo format change
* more private
* update comment for Engine.reinterpret
* test ensure failure works correctly; no rollback expected
* refactor to share duplicated code in reinterpret tests
* remove unhelpful create/ensure tests
* test that reinterpretation of an uncatchable error remains a failure
* test that rollback-version-check-error remains a failure for reinterpretation
* remove dev comment left in by accident
* Run scenarios in off-ledger machine
This PR builds on the previous PR that split scenario execution in two
different speedy machines and now actually makes the machine that runs
scenarios run in off-ledger mode just like we handle Daml Script.
This required a bunch of refactoring to make it nice so apologies for
the slightly large PR. Hopefully it’s still relatively easy to follow
and luckily it deletes more code than it adds.
changelog_begin
changelog_end
* Apply suggestions from code review
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* review comments
changelog_begin
changelog_end
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
It makes no sense to pass NotVisible to Speedy especially since that
is not how visibility is handled for the Engine. Also it lets us
delete code and I like deleted code.
changelog_begin
changelog_end
* Use ScenarioRunner.submit in Daml Script
changelog_begin
changelog_end
* privatize ledger variable
changelog_begin
changelog_end
* drop space
changelog_begin
changelog_end
* Split scenario & ledger execution
This PR by no means aims to solve everything we can do here. It is
rather the minimal change that I could get to work that provides us
with one Speedy machine for scenario execution (which is still an
on-ledger machine to avoid having to change the callsites to much in
this PR) and one speedy machine per submission.
There is tons of cleanup we can do afterwards but this should
hopefully set the right foundations.
changelog_begin
changelog_end
* upgrade scalacheck to 1.14.3
* regenerate maven_install files
* some different names and implicits
* remove some fromTryCatchNonFatal
* more porting
* port fromTryCatchNonFatal to attempt
* factor the assertions in SignatureSpec to avoid \/
* deal with invariant \/
* make partial unification do what we want
* \/, parse*, and toNel
* many uses of the .right method
* a legitimate use of fromTryCatchThrowable
* rebuild maven pins
* further invariant \/
* OneAnd and Nel interface changes
* further Either games
* \/ and reformatting
* \/ in http-json
* \/ in http-json
* deprecations
* more invariance
* cleanup unused
* more invariance; http-json compiles
* final either follies
* small 2.12 extra incompatibility
* rebuild deps
* revisit a couple earlier fixes using nicer expressions I learned later
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* repin 2.12
* Clarify semantics of DIV_INT64 in LF spec.
Especially around negative operands. Adds a couple of unit tests to make sure the behavior stays this way, and that MOD_INT64 is consistent with DIV_INT64.
changelog_begin
changelog_end
* spelling
* scalafmt
* hardcode numbers in MOD_INT64 test
* Drop pkgIds argument from Engine.validatePackages
Doesn’t do anything useful, we just use it to validate that it matches
the keys of pkgs.
changelog_begin
changelog_end
* add test showing broken behavior
* Fix buggy tests
changelog_begin
changelog_end
Co-authored-by: Remy Haemmerle <Remy.Haemmerle@daml.com>
* Add proper error handling for missing contract keys
These are clearly not internal errors so we should not be calling
`crash` here. Canton in fact already started string matching on the
crash message which is definitely not what we want.
changelog_begin
changelog_end
* backwards compat
changelog_begin
changelog_end
* fix tests
changelog_begin
changelog_end
This PR is part of the effort to improve engine Errors. Here we
classify all the errors the engine can report in 4 categories:
- Errors happening during package loading
- Error happening during command/transaction preprocessing
- Error happening during interpretation
- Error happening during transaction validation
part of #9974
CHANGELOG_BEGIN
CHANGELOG_END
This modifies a test about version encoding that should not be run
with Rollback nodes as those do no have explicit version.
CHANGELOG_BEGIN
CHANGELOG_END
* Include PackageMetadata in LF interface
I want to use this in codegens but more generally, I think it also
fits well into the scope of the iface library in that it’s only
metadata and relatively stable.
changelog_begin
changelog_end
* Iterating more is less
changelog_begin
changelog_end
* fix tests
changelog_begin
changelog_end
* fmt
changelog_begin
changelog_end
* fix scala 2.12
changelog_begin
changelog_end
Splitting the test between ExceptionTest and SBuiltinTest is just
confusing so this PR drops the test from ExceptionTest and moves it to
SBuiltinTest. We should probably do the same for From/ToAnyException
but I’ll leave that for #9861 or a follow-up PR.
changelog_begin
changelog_end
This PR fixes the todo in danglingRefGenNode to include rollback
nodes.
Some tests simply don’t work on rollback nodes so I added
danglingRefGenActionNode. For other tests, we just need to switch how
we generate nodes and versions to be a bit more sensible.
changelog_begin
changelog_end
* Track minChildVersion when building transactions
Track in the `Context` if we have seen any _old_ children (predating `minException`).
Avoiding potential quadratic blowup of `smallestContainedVersion` in `PartialTransaction`.
Also make a small cleanup of the `Context` code, to avoid code duplication when creating a new _empty_ context.
changelog_begin
changelog_end
prefer to track minChildVersion:TxVersion instead of oldChild:Boolean in PartialTransaction.Context
propagate minChildVersion up through exercise nodes
* test that old-contracts created within new-exercises cannot be contained in a rollback node
* add some comments; propogate minChildVersion through rollback
* fmt
wildcard pattern matches bite once again. We clearly do not want to
use the committers here.
I tried to move most of the logic to PartialTransaction but
unfortunately moving the submitters to PartialTransaction doesn’t
quite work since as usual scenarios make things difficult because
submitters change at weird places and we reset the partial transaction
in weird places.
changelog_begin
changelog_end
Before this change, when converting a string which exceeds the max
length, the application return the error "too long" without saying what
the bounds are, requiring a user to lookup the bounds in code. Now, the
user gets the actual length that is enforced as part of the error
message.
CHANGELOG_BEGIN
CHANGELOG_END
This PR simplifies the internal of Speedy by factorizing SAny and
SAnyException. By side effect we also get the comparison of LF
AnyException values that was not implemented before.
CHANGELOG_BEGIN
CHANGELOG_END
Note that it only has rules for normalizing `create` and `exercise`, but that's because the LF spec only has `create` and `exercise` actions for now.
changelog_begin
changelog_end
* Normalize rollbacks: first draft
CHANGELOG_BEGIN
CHANGELOG_END
NormalizeRollbackSpec, WIP
WIP2
adapt to asVersionedTransaction
first stab at traversal for normalize-rollbacks... implement normalization rule #1
adapt existing testcase for rollback normalization rule #1
pluralize spec filename
temp disable rollback normalization
methodically test normalization rule #1
lots more tests
spec: check all 3 norm conditions
cleanup test/Shape code a bit
implement normalization rules #2 and #3 (using canonical types) and enable tests
add some comments and some 8020 todo markers
* add 2 more testcases from Sofia
* ensure tx produced when normalizing rollbacks has increasing node-ids when listed in pre-order
* enable rollback normalization in interpreter
* manage state functionaly for the created tx (counter & node-map)
* un-nest sub defs from normalizeTx (we can because we removed the mutable state)
* rename: force* --> push*
* introduce CPS for push functions
* introduce trampolines for push functions to be stack safe
* one more bounce
* ensure generated node-ids start from 0
* test that transaction node-ids start from 0
* add commets about pass1/2; move makeRoll (part of pass 1) earlier in file
* intro CPS for pass-1 over original tx
* intro trampolines for pass-1. everything is stack safe now
* clarify comment
* remove make stack-safe todo
* be more private
* factorize/share Trampoline implementation with previous implementation in speedy.Anf
* prefer Vector over List, for better algorithmic complexity
* make Trampoline private to lf
* Update LF spec for exceptions.
The changes here are:
* Removing the built-in exception types
* Changing the result in the operational semantics to include both an "exception thrown" case and a "fatal error" case.
* Cleaning up the semantics of create/exercise/etc and try/catch.
Not included in this PR is anything to do with built-in arithmetic exceptions. There's room to add it in the future (as a value of AnyException type), but I would do it in a follow-up PR.
changelog_begin
changelog_end
* finish renaming Throw cases
* Update type ordering for AnyException.
The name of some builtins will be exposed as part of the Exception
message. This PR, try to make conversion builtins more consistent and
more obvious, before we could not rename those.
This is part of #8020
CHANGELOG_BEGIN
CHANGELOG_END
* Split DA.Internal.Exception.Types into three packages.
I think this is a good idea because it means the user can disambiguate between these modules in daml. All three types define a `message` field, so if the programmer wants to refer to the field for one of these types specifically, it's nice to have a canonical way to disambiguate them (by module name).
Also the module names show up in the error message for uncaught exceptions. These names are a bit nicer than "DA.Internal.Exception.Types".
Decoupling these types just sounds like a good idea going forward.
changelog_begin
changelog_end
* buildifier-fix
* update isInternal
* Update tests
* Use more general pattern in CommandServiceIT
* Update DarReaderTest
* speedy: Compile new AnyException primitives
changelog_begin
changelog_end
* s/Contract/ContractError
* remove random eszett
* fix booleans
* Combine some cases!
* define and use getSException
* use SException directly in pattern match
For better or for worse, kvutils validation insists on treating a
negative input from a create different to a negative key lookup (and
to make things more annoying the first one will never blow up during
submission only during validation).
While, nobody seems to argue all that strongly that the current errors
are very sensible, we agreed to do it in two steps:
1. Expose enough information in `contractKeyInputs` so that we can use
it in kvutils while preserving the current error semantics.
2. Revisit contract key error handling across ledgers which is
currently an inconsistent mess.
changelog_begin
changelog_end
.
changelog_begin
changelog_end
* Throw on internal errors instead of setting ptx to aborted
The main purpose of aborted is to be able to get out a ptx in the
scenario service. However, for internal errors that makes no
sense. Users should never encounter them. And the use of `aborted`
here has swallowed these errors silently in a few places before which
is clearly bad.
changelog_begin
changelog_end
* Fix tryHandleSubmitMustFail
changelog_begin
changelog_end
* Remove version field from rollback node.
changelog_begin
changelog_end
fix transaction decode & testcases
reinstate/adapt testcases
fmt
rebase change to fix conflicts
* treat missing version on transaction node as being version minExceptions when computing tx version
* add 8020 marker for grepability
* reinstate Node.version to avoid logic duplication
* Revert "reinstate Node.version to avoid logic duplication"
This reverts commit 00dacb453d.
* remove Node.updateVersion
* remove comments with dubious utility
* Drop version check from TransactionPreprocessor
This is only supposed to be run on nodes that have been normalized via
de(serialization) so this is just useless shotgun validation.
changelog_begin
changelog_end
* Fix test
changelog_begin
changelog_end
* check whether collection.compat is unused when compiling for Scala 2.12
- Instead of always suppressing warnings for collection.compat._,
we should only do it for Scala 2.13
- We can also reduce boilerplate by automatically adding this
option when both silencer_plugin and collection-compat are
present
CHANGELOG_BEGIN
CHANGELOG_END
* remove unused import
* remove another unused import
* remove even more unused imports
* missed compat dependency
* more missed compat dependencies
* missed compat dependency
* use scala_deps in scaladoc_jar
- #8423 inlined the major version expansion, but this seems to
have been prior to proper support by scaladoc_jar
* restore custom handling of participant-integration-api
- fixing scaladoc_jar isn't worth it for a single case, as with
deps vs scala_deps
* Check byKey in transaction validation
No tests because validation has no tests in general :(
part of #7622
changelog_begin
changelog_end
* Fix preprocessing of transaction root nodes
changelog_begin
changelog_end
* Swap order SEScopeExercise and SBUBeginExercise
SEScopeExercise pushes the KCloseExercise continuation. However, the
corresponding transaction node is only inserted when SBUBeginExercise
finishes. This causes an issue if SBUBeginExercise crashes/throws an
exception.
changelog_begin
changelog_end
remove weird comment
changelog_begin
changelog_end
Reenable another test
changelog_begin
changelog_end
* Update failing test
changelog_begin
changelog_end
This PR includes :
- Adding ApiCommand to distinguish between generic command (that are
accepted by the engine) and command that are accepted by the ledger
API.
- Reimplement Canton's reinterpret method using commands instead of
node.
CHANGELOG_BEGIN
CHANGELOG_END
This PR fixes a very confusing and subtle bug in the handling of
submitMustFail in scenarios:
submitMustFail did not properly unwind the transaction. Afaict, this
only matters for contract id derivation in this context.
It also only matters for errors during interpretation not errors that
happen when committing to the ledger (global duplicate key errors).
Specifically what happens is that we end up leaking the exercise
context that we were in when we hit the failure into the next
submission. I’m not sure if this actually breaks anything (if it does,
doesn’t really matter, it’s scenarios) but it is definitely not what
should be happening.
I hit this when adding uck checks to the engine since that turns some
commit errors into interpretation errors and changed contract ids in
tests that really shouldn’t have changed.
changelog_begin
changelog_end
* Move activeness check of globalKeyInputs into archive
This is semantically equivalent but it seems slightly simpler and more
importantly it leads to slightly nicer semantics by reducing the
number of cases where we have to worry about accidentally skipping an
activeness check.
changelog_begin
changelog_end
* s/optCid/keyMapping/
changelog_begin
changelog_end
* review feedback
changelog_begin
changelog_end
* Filter divulgence to an empty set of parties
As @nmarton-da noticed painfully, we currently include divulgence to
an empty set of parties. While this is arguably not wrong it is at
least confusing and useless. The whole point of divulgence is to track
visibility. Divulging to an empty set of parties does not affect
visibility so it is not meaningfully different from no
divulgence. Therefore this PR filters it out and adds a doc comment
that the list of divulgees is always non-empty.
changelog_begin
changelog_end
* Fix tests
changelog_begin
changelog_end
This looks like an attempt at moving the conversion from transactions
to Ledger API events to the Ledger API. However, that attempt is not
used anywhere (checked pkvutils, corda & canton).
I don’t think we benefit from having an unused method sitting around
here so this PR kills that method.
changelog_begin
changelog_end
* Only archive a key if it was brought into scope before
This PR implements the semantics we agreed on in
https://github.com/digital-asset/daml/pull/9472#discussion_r620097844.
Before, archive always marked the key as inactive in `keys`. This is
weirdly inconsistent with other operations. For example, a regular
fetch does not bring a key in scope in `keys`. This PR changes this to
a more consistent model where `keys` is modified exactly under the
following circumstances:
1. A create with a key always overwrites.
2. An archive will only mark a key as inactive if there was an entry
in `keys` with the same contract id.
3. Lookup/fetch by key first check if we currently have an entry in `keys`.
If so, we use that.
If not, we need to ask the ledger. We first check if we have a
ceched entry in `globalKeyInputs`. If we do use that.
If we don’t, we ask the ledger.
If the result is not active or not visible we proceed updating `keys`
and `globalKeyInputs`.
If the result was a cid, we check if it’s still active.
changelog_begin
changelog_end
* Review comments
changelog_begin
changelog_end
* Apply suggestions from code review
Co-authored-by: Remy <remy.haemmerle@daml.com>
* fmt
changelog_begin
changelog_end
Co-authored-by: Remy <remy.haemmerle@daml.com>
This PR clarifies that NodeInfo is only intended for actions by
renaming it to ActionNodeInfo and correspondingly also changes
protoNodeInfo to protoActionNodeInfo and makes it return a `Left` for
rollback nodes.
changelog_begin
changelog_end
This PR some minor clean up for #9537 and #9538.
In particular it:
- Remove the unecessary class KeyOperation (SBUKeyBuiltin is already a
somehow a key Operation)
- Rename and refactor methods handleKeyFound, handleKeyArchived,
cidToSValue and cidToSExpr to gave them a more obvious meaning.
- Replace immutable.Map#+ by immutable.Map#updated (cosmetic)
- Replace trait by abstract class (cosmetic)
CHANGELOG_BEGIN
CHANGELOG_END
* Address more exception todos
A bit of a kitchen sink PR to address a bunch of the trivial todos
that didn’t seem worth splitting out into separate PRs.
changelog_begin
changelog_end
* Revert changes to TransactionSpec
changelog_begin
changelog_end
As discussed, we don’t want to expose this via serializable values at
least for now (and it’s not exposed on the ledger API anyway) so this
PR drops the type.
changelog_begin
changelog_end
* Cache global key inputs
This PR changes speedy such that it will query at most once for a global
key. This is a prerequisite for a (sensible) duplicate key detection
and for sensible multi-key semantics.
Note that there is one outstanding issue here:
As a general rule, we only want by-key operations to bring a key in
scope not just a random operation like an exercise or a fetch that
happens to have a key.
However, for consuming exercises this is currently not the case. They
always bring the key in scope. The plan here is for an archive to not
block a further lookup and instead add an activeness check to the
lookup afterwards.
changelog_begin
changelog_end
* Update daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/speedy/SBuiltin.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
* s/CtrlImportValue/SEImportValue/
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
* Use a custom Scala type instead of Option[Value.ContractId]
changelog_begin
changelog_end
Co-authored-by: Remy <remy.haemmerle@daml.com>
No need to treat rollback nodes any differently here. A lookup for a
cid -> node mapping that does not point to a create node is borked and
we turn it into notfound.
changelog_begin
changelog_end
This is only used in one test where we can just reuse the scenario
ledger. I really don’t want more broken ledger implementations.
This also removes the need to address the TODO around rollback nodes.
changelog_begin
changelog_end
* Add Ledger API test tool tests for exceptions
changelog_begin
changelog_end
* Update daml-lf/language/daml-lf.bzl
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* Address review comments
changelog_begin
changelog_end
* Shuffle around test
changelog_begin
changelog_end
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* Document and test multi-key semantics
Canton relies on the Engine working correctly even in a setting where
we do not have multiple keys. So far this worked by accident but the
semantics of this are rather unclear. To make things worse, Canton
upgrades rely on those semantics being stable so we really do care
about the choices we make here.
This PR adds a bunch of tests as an executable documentation of the
current behavior. However, we do not provide stability guarantees for
the current behavior and therefore these tests can be changed as
needed. But at least we are aware of those changes rather than doing
them by accident.
changelog_begin
changelog_end
* Update daml-lf/engine/src/test/scala/com/digitalasset/daml/lf/engine/EngineTest.scala
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* Update daml-lf/tests/MultiKeys.daml
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* Update daml-lf/tests/MultiKeys.daml
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* Share code between SBUFetchKey and SBULookupKey
Those do almost the same and the logic is growing increasingly complex
so I really want to share it. This also makes it easier to see the
things that are different.
Along the way I’ve also discovered a bug:
SBUFetchKey called crash where it should produce a proper error that
can be caught by submitMustFail. Not going to resolve that in this PR
though so I just added a TODO.
changelog_begin
changelog_end
* review comments
changelog_begin
changelog_end
* Move builtin exceptions to a stable package.
This PR moves the built-in exceptions (GeneralError, ArithmeticError, ContractError) to a stable package in daml-prim, in preparation for removing them from LF proper.
Part of #8020.
changelog_begin
changelog_end
* Update tests
* rename the module
* couple that got away
* update hash
* Update dar reader test.
* typo
* questionable lint
* Allow missing exercise results when version >= minExceptions
imp:
- adapt encode & decode implementations
- add version dependant checks for version < `minExceptions`
testing:
- update scalacheck `ValueGenerators` to produce exercise nodes with the missing result value guarded by the versions
- update `normalizeExe` in `TransactionCoderSpec`
- update `transactionVersionGen` to be inclusive on upper bound
- unhappy-path test for encode/missing-exercise-result when not allowed
- unhappy-path test for decode/missing-exercise-result when not allowed
changelog_begin
changelog_end
Just some small changes to take into account the new ledger model transaction structure (see #9396). This PR doesn't fix all the issues with the LF spec not matching the ledger model, but it tries to avoid widening that gap.
* Removed itr, iexercise which we don't need.
* Removed the exception type and value from the rollback node.
* We're not calling rollback nodes ledger actions anymore, so separate it out.
Part of #8020
changelog_begin
changelog_end
* Switch from `@silent` to `@nowarn`.
This annotation is native to Scala 2.12.13+ and 2.13.2+. It replaces
most usages of `@silent`.
I had to get creative about a couple of use cases that didn't work.
Specifically:
1. Suppressing deprecation warnings works, but Scala 2.12 erroneously
complains that the `@nowarn` is unnecessary. I had to suppress
this warning too with `-Ywarn-unused:-nowarn`.
2. I can't seem to suppress the warning, "The outer reference in this
type test cannot be checked at run time." Instead, I have
refactored the code to remove the warning.
We still need to use the silencer plugin to suppress some warnings about
unused imports (because of compatibility between Scala 2.12 and 2.13),
but this means we no longer need the library, and therefore it is not a
transitive dependency that downstream consumers need to worry about.
CHANGELOG_BEGIN
CHANGELOG_END
* Add some comments around `@nowarn` support.
* language-support/scala: Fix a warning suppression.
* Revert to the default warnings.
Compatibility was complaining.
* Add a helper for getting all inactive contracts to Transaction
This comes up in a few places and rather than inlining the
implementation everywhere, I’d rather rely on this.
changelog_begin
changelog_end
* 2.13
changelog_begin
changelog_end
CHANGELOG_BEGIN
- [Scala bindings] ``InsertOrdMap`` is deprecated; it was never
needed for the release version of ``GenMap``. Use an ordinary
``Map``, or ``Primitive.GenMap`` for Scala codegen, instead.
CHANGELOG_END
* Fix validation of rollback nodes
To my own embarassment there is no test in this PR. We don’t have any
direct tests for validation atm. We absolutely need some and I already
discussed with Remy that this is something we should focus on once
exceptions are done but for now I’ll stick to being sad.
changelog_begin
changelog_end
* Fix bug
changelog_begin
changelog_end
