The method is not correctly implement, as it should slice the
underlying array before returning it. Since it is not possible to
implement it in constant time, we simply drop it, in favor in other
conversion.
CHANGELOG_BEGIN
CHANGELOG_END
* Escape daml-lf tracelog messages
Currently veracode complains because this allows for clrf
injection (injecting newlines to make user input look like separate
log statements).
With this change
```
debug "abc"
debug "eaiu\neaiu"
debug "def"
debugRaw "abc
```
is logged as
```
[DA.Internal.Prelude:555]: \"abc\"
[DA.Internal.Prelude:555]: \"eaiu\neaiu\"
[DA.Internal.Prelude:555]: \"def\"
[DA.Internal.Prelude:555]: abc
```
You can debate whether we should escape the quotes are necessary but
90% of the reason why people add them is because they call `debug` on
strings when they should be using `debugRaw` so this seems fine to me.
changelog_begin
changelog_end
* fix tests
changelog_begin
changelog_end
as it is normally caught in the interpreter loop. In particular the
caller of the engine should not have to catch such an error.
CHANGELOG_BEGIN
CHANGELOG_END
* Update TODOs to outdated issues.
In particular update a lot of defunct interface TODOs to the LF 1.15
issue (or add the LF 1.15 issue if it's also relevant).
changelog_begin
changelog_end
* typo
* remove numeric/bignumeric todo
* scalafmt!!
* drop choice observer TODO
changelog_begin
changelog_end
add runtime check in freeVars: determination that a variable is-free using levels instead of indexes
remove DB-indexes and runtime check; simplify freeVars computation in closure-conversion
Part of #11978. Adds typechecking for this field on the interface side,
and enforces that any template that implements A must implement B if A requires B.
CHANGELOG_BEGIN
CHANGELOG_END
As mentioned in the comment, I view these tests as a way to guard us
against bricking ledgers on upgrades so they somewhat deliberately
ignore ordering of non-cachable errors relative to each other since
those only change error messages.
changelog_begin
changelog_end
* Remove the use of nesting with `size` and `resize` approach as suggested by scalacheck.
CHANGELOG_BEGIN
CHANGELOG_END
* Biased to shorter names generator for Name
After staring at a type mismatch between `Nested` and `Nested` this
seemed like a sensible change. We could also include the package
ids. I don’t have strong feelings about this but this seems like a
clear uncontroversial improvement.
changelog_begin
changelog_end
This is part of #11691
This PR allows to limits:
- the number of signatories,
- the number of observers,
- the number of controllers,
- the number of inputContracts,
CHANGELOG_BEGIN
CHANGELOG_END
* Add skeleton for trace-based LF evaluation tests
The recent issues around wrong ordering of contract id typechecks got
me thinking about this again so I wanted to hack up a PR of how we can
test evaluation order sensibly directly at the LF level.
This is only testing a single create but I first wanted to see if we
agree that this is a sensible approach using a simple example.
If we agree that this is a sensible approach, I’d suggest to extend
those tests later in separate PRs to make them exhaustive.
changelog_begin
changelog_end
* cleanup
changelog_begin
changelog_end
* Adding missing stack-safety testcase for SEScopeExcerise.
changelog_begin
changelog_end
* run all testcases at depth of 10,000. This is plenty deep enough to ensure stack-safety
* recode free-vars computation to be stack-safe. and test
* Drop support for Daml-LF party literals from the Scala side
This PR enforces that forbidPartyLiterals is always `true` and drops
the corresponding literals from the AST. Haskell side is in #11930fixes#11581
changelog_begin
changelog_end
* Update daml-lf/interpreter/src/test/scala/com/digitalasset/daml/lf/speedy/ComparisonSBuiltinTest.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Revert "Update daml-lf/interpreter/src/test/scala/com/digitalasset/daml/lf/speedy/ComparisonSBuiltinTest.scala"
This reverts commit 55e542ce4e3a7fd15544ee703de3277ffc309b17.
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Use Absolute-indexes as keys for the Env-mapping during closure-conversion.
Do runtime check to confirm behaviour matches the existing Relative-indexes.
changelog_begin
changelog_end
* remove quadratic shift!
remove (dev)pretty-print code
remove relative-index keys from Env-mapping
remove runtime *diff* check
increase depth for stack-safety tests
* improve/simplify indexing calculation for Env-keys
* Add type rep argument for interface exercises.
(Still WIP.)
Part of #11703. Fixes the order in which errors are raised ("wrong type"
takes priority over "does not implement interface"). This PR also simplifies
ExerciseInterface by making the guard mandatory, otherwise there's too
many variations. We can revisit that later if we want.
changelog_begin
changelog_end
* haskell side
* dont throw exception in checkTemplateId
* scalafmt
* evidence security
* fix TypingSpec test
* SExpr1.SELocS - carry relBad/absGood - abs unused so far
* compute/pass SELocS.abs in ClosureConversion, and check in Anf that it matches the reconstructed value
CHANGELOG_BEGIN
CHANGELOG_END
* Remove relative stack locations. Rename as SELocAbsoluteS. Simplify Anf. Remove shiftLoc in ClosureConversion.
* Prevent wrongly typed fetch by interface.
When doing a "fetch by interface" command with a known template id,
error out with a WronglyTypedContract if the fetched contract has
a different template id. This doesn't affect daml, only affects
replays, so it's rather minor. I also enabled the engine test that
caught this.
Part of #11703, follow up to #11836.
changelog_begin
changelog_end
* strengthen test output checks
* Drop DA.Next.Set and DA.Next.Map
Fixes#11527. Also removed a bunch of unnecessary CPP, though there's
still a lot more CPP to remove.
changelog_begin
- [Daml Standard Library] DA.Next.Map and DA.Next.Set have been removed
after being deprecated since Daml-LF 1.11
changelog_end
* Drop a deriving MapKey
* update unstable-types test
* Add a guard when exercising by interface.
This fixes part of #11703, when exercising an inherited choice by
interface and you know the template id, via the command preprocessor.
It does this by inserting a "guard" in between the interface fetch and
the exercise body. The guard is a function Interface -> Bool, which
is general enough to check the template id, without complicating too
much in speedy. And can be generalized in the future to check more,
like signatories, etc.
I added the guard as an optional argument to UExerciseByInterface.
This isn't hooked up to the protobuf AST yet (or Haskell side for
that matter) -- but I'll do it in the next PR! For now you can invoke
the guarded exercise via the command preprocessor, so I can enable the
approprate engine tests. (There's still some failing fetch tests left,
but I decided to leave this for later. Fetch can be a lot simpler than
guarded choices, since you always add a fetch node. No need for fancy
continuations.)
changelog_begin
changelog_end
* scalafmt
* Feedback and fix matches
* Update comments, we are always going to abort the transaction
* Raise WronglyTypedContract in SBGuardTemplateId.
* rebase and fix parser
* restore ANF
* scalafmt
ClosureConversion -> Suffix with "Old"
CHANGELOG_BEGIN
CHANGELOG_END
ClosureConversion old-vs-new diff check
ClosureConversionNew, first cut. All tests in SBuiltinTest work.
In addition we change some Array --> List in SExpr1 (for human pp).
And we throw away ClosureConversionDup.
adapt AnfTest from Array to List change for SExpr1
all tests pass in daml-lf/interpreter
remove SExpr0.SELet1General
reorder things
testing for stack-safety of closure conversion
file/class renames
improve naming
pass cont as sep arg to commit (move out of Up/Down)
comment stack-safe closure conversion
fix bug: failed to use env1
fix 2x unmoored doc comment
comment stack safety testing
Remove old closure-conversion code & diff-check between old/new.
loose StackSafe suffix on ClosureConversion class/file
rename StackSafetyTest.scala to ClosureConversionTest.scala
prefer "sealed abstract class" to "sealed trait"
fvs.zipWithIndex --> fvs.view.zipWithIndex
(SExpr1) SEAppGeneral -> SEApp; prefer List to Array in SEApp/SECase
prefer xs.toArray to Array(xs: _*)
access SExpr0 via "source."
two more .view
improve comment and fix typo
link to Issue
switch to a continuation stack; avoids nesting in the Cont type
* Drop LF < 1.14 from supported damlc output versions
fixes#11319
We keep test coverage by depending on the most recent snapshot which
still has 1.14 support.
changelog_begin
- [Daml Compiler] Damlc can only produce Daml-LF 1.14 or
newer. Passing aynthing older to `--target` is an error. If you
need to produce older versions, use an older SDK.
changelog_end
* Switch around legacy_compiler_lf_versions
changelog_begin
changelog_end
* drop since-lf
changelog_begin
changelog_end
This fixes a bug in the typechecker (#11558) and the command
preprocessor, since those were written with this behavior of
lookupTemplateChoice in mind. Enables the engine test that
caught this.
changelog_begin
changelog_end
- rename non-default builders `apply` to `build`
* avoid confusing both
* make explicit the build can crash
- make interfaceId and templateId fields consitent
- use when possible named arguments
- check for non-repetition of inherited choices
CHANGELOG_BEGIN
CHANGELOG_END
We revert #11626, and just change the way transaction version is
computed:
- As before, Node version is calculated from the
package of the template ID action.
- Transaction version is the max of the version of all the nodes,
instead of the root nodes.
CHANGELOG_BEGIN
CHANGELOG_END
* Add engine tests for interfaces.
Adds engine tests using the new commands. This uncovered a few issues:
- getDependencies doesn't work when given an interfaceId as a templateId
(such as when exercising an interface via the ledger api). I patched
it up so it deals with interface ids. We've already discussed a way to
simplify getDependencies that will also avoid this in the future, but I'll
leave that to the future.
- issue #11703 is confirmed via some tests that don't pass (and are
currently disabled)
- PackageInterface.lookupTemplateChoice returns inherited choices, when it
should only return own choices. At least, the typechecker assumes
it doesn't -- this affects #11558 -- as does the command
preprocessor. I'll leave the cleanup to a separate PR.
changelog_begin
changelog_end
* scalafmt, add missing file
* Move interfaces tests to separate file
* scalafmt
* Update to Java 11
changelog_begin
changelog_end
* Fix RoundingMode deprecation warnings
* Fix dep-ann warning
* Integer constructor
* JavaX annotation dependency
* javax.xml.bind was removed in Java 11
Using Guava as a replacement, since it is already a project dependency.
* JDK 11 no longer has a separate JRE tree
* Remove unused jdk_nix import
* remove now redundant jdk11_nix
* Java 8 --> 9 increased Instant.now() precision
See https://bugs.openjdk.java.net/browse/JDK-8068730
The precision of `Instant.now()` increased between Java 8 and Java 9.
On Linux and MacOS this doesn't seem to be a problem, as the precision
still seems to be at micro seconds. However, on Windows this now causes
errors of the following form:
```
java.lang.IllegalArgumentException: Conversion of Instant
2021-11-05T13:58:56.726875100Z to microsecond granularity would result
in loss of precision.
```
Suggesting that it now offers sub-microsecond precision.
`TimestampConversion.instantToMicros` had a check to fail if the
conversion lead to a loss of precision. In the specific failing test
case this is not a concern, so this adds a `roundInstantToMicros`
variant that avoids this kind of error.
* TMP round timestamps
* Revert "TMP round timestamps"
This reverts commit af8e261278.
* Skip versions before 1.6.0 in migration tests
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
I’ve kept the infrastructure for versioned_scala_deps around because
I’m optimistic and hope that eventually we’ll do another Scala upgrade.
changelog_begin
changelog_end
* LF: Reintroduce TransactionVersion.asVersionedTransaction
This have been drop in #11626, but canton is using it.
CHANGELOG_BEGIN
CHANGELOG_END
* cosmetic
* cosmetic
Because a bug in the scala compiler deprecated pattern matching are
not detected (See https://github.com/scala/bug/issues/12493).
Hence some deprecated usage of Node aliases have been forgotten in
CHANGELOG_BEGIN
CHANGELOG_END
* interfaces: Preserve by_interface data for create.
Part of #10915
This was a lot more involved than fetch or exercise. The first issue is
that we need to preserve the interface id into speedy, so it needs a
separate primitive ("experimental" won't cut it). Second, because
speedy's create requires the template definition, and now the interface
id as well, we basically need to compile a separate version of "create"
for each interface that a template implements, hence the separate
`CreateByInterfaceDefRef(templateId, ifaceId)`.
changelog_begin
changelog_end
* scalafmt and refactoring
* fixx merge conflict
* fix silly mistakes
* Refactor speedy to distinuish SExpr types before/after ANF compilation phase
CHANGELOG_BEGIN
CHANGELOG_END
* remove commment/marker left in error
* make SExpr0 private to speedy
* reinstate (non-pp) print of original expression in AnfTest faiure
* avoid use of s./t. prefixes for expressions in SBuiltin; add 3 TODO markers
* inline "runtime" apply methods of SDefinitionRef into sole caller: SBCallInterface
* avoid use of t. prefix in SExpr0
* change s./t. prefix to source./target.
* add comment to summarize differences between SExpr0 and SExpr
Following up #10827 and #10921, we drop type parameter to
KeyWithMaintainers, and use the `Versioned` wrapper introduced in
CHANGELOG_BEGIN
CHANGELOG_END
* FoldableContravariant, a mapping for Foldable instances
* use FoldableContravariant to specialize several ImmArraySeq, NonEmpty methods
* folding specializations for ImmArray
* a few docs for FoldableContravariant
* specializations for FrontStack
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* LF: Rename GenTransaction to Transaction
Following up #10827 and #10921 that drop type parameters from GenNode,
we rename GenTransaction to Transaction.
CHANGELOG_BEGIN
CHANGELOG_END
* fix
* a bit more fixes
* Add by_interface field in tx nodes.
This PR adds an optional `by_interface` field in the Create, Fetch,
and Exercise nodes, as part of #10915. It also updates TransactionCoder
to support these fields.
Setting these fields (in the LF interpreter) is left to a separate PR.
changelog_begin
changelog_end
* update security-evidence.md
* Add version checks in TransactionCoder
* Remove virtual choices
* Remove choices without a body in 'interface' definition
* Remove choices in 'template ... implements' section
part of #11372
changelog_begin
changelog_end
* Remove virtual choices cont.
Switch uses of virtual choices to fixed choice with method implementation
* update snapshot after pin on windows
* Disable failing interface tests with TODO #10810
* interfaces: Improve fixed choice lookup
This PR adds a lazy map to match fixed choice names to their
interface, to improve the worst case for choice lookup.
Part of #10810
changelog_begin
changelog_end
* scalafmt
* apply improvements from remy
After #10827 that drops type parameters in Value, `CidContainer`'s
`ensureNoCid` and `assertNoCid` loose their interest.
To check a value does not contain Contract Ids one should use
`foreachCid` or `cids` methods instead.
CHANGELOG_BEGIN
CHANGELOG_END
* Replace `sandbox-classic`-based fixtures with `sandbox`-based ones
changelog_begin
changelog_end
* Replace Scenarios with Script for BasicPrimitiveTypes
* Adapt ListsSpec
* Adapt TextMapsSpec
* Adapt OptionalSpec
* Adapt more tests
* More tests passing
- removed workflow_id testing (Daml Script doesn't set one)
- reduced failure threshold for VeryLargeArchiveSpec
- solved issu in TransactionSingleTableSpec (wrong parties were used)
* Port EnumMod and make EnumSpec pass
* Port GenMapMod and make GenMapSpec pass
* Shaking things around to make Windows happy -- maybe
* Add choice collisions in scala name collision chk
This brings it in line with the LF spec.
This PR also adds collisions for inherited (interface) choices.
Closes#11137.
changelog_begin
changelog_end
* scalafmt
* Apply suggestions from code review
Co-authored-by: Remy <remy.haemmerle@daml.com>
Co-authored-by: Remy <remy.haemmerle@daml.com>
* iface: check for interface implementations precond
We change speedy to also check all preconditions defined in interface
implementations upon a create.
CHANGELOG_BEGIN
CHANGELOG_END
* Update daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/speedy/Compiler.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
* format
* interfaces: lfconversion for iface preconditions
This adds interface preconditions to the interface implementations
during completion phase in the LF conversion.
CHANGELOG_BEGIN
CHANGELOG_END
* format
* rebase
Co-authored-by: Remy <remy.haemmerle@daml.com>
After dropping type parameter from GenTransaction we do not
com.daml.lf.transaction.Transaction.Transaction type alias. We use
instead directly com.daml.lf.transaction.VersionedTransaction, wich is
anyway more informative.
CHANGELOG_BEGIN
CHANGELOG_END
In #11419 we drop the implicit call that convert ContractInstance in
CidContainer. Thie is used by Canton and should be restore.
Note that now ContractInstance is not polymorphic anymore, hence we
can inherite CidContainer instead of defining an implicit.
CHANGELOG_BEGIN
CHANGELOG_END
* Split contract-key tests from engine tests, and mark for evidence.
CHANGELOG_BEGIN
CHANGELOG_END
* remove contract key tests from their original location
* evidence for unique/non-unique contract keys
* interfaces: precondition, scala ast, decoder/encoder
This adds the interface precondition to the scala Daml LF AST, plus
decoder/encoder and the typechecker.
CHANGELOG_BEGIN
CHANGELOG_END
* Update daml-lf/archive/src/main/scala/com/digitalasset/daml/lf/archive/DecodeV1.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
* format
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Make stable packages dependent on supported LF versions
Note that this just introduces an API dependency, we are not yet
filtering the list of packages. For that, I’d really like to
autogenerate StablePackages.scala since I don’t want to make that
manually maintained list even more complex. But review seems easier if
we first change the usage sites and then switch to autogenerating than
trying to do both in one go.
changelog_begin
changelog_end
* Fix script export tests
changelog_begin
changelog_end
* fine grained test evidence for authorization
* fine grained test evidence for privacy
* fine grained test evidence for input-validation (typing)
* fix exit code of security/update.sh script (set -euo pipefail)
* add security evidence test category: Input Validation
* regenerate security-evidence.md
CHANGELOG_BEGIN
CHANGELOG_END
* fix bug in securoty evidence generation (must sort before group, or else we loose lines)
* evidence for input validation of commands
* address comments
* cleanup: remove backticks from evidence free text
We waste 2 minutes on most CI runs with the conformance test for no
good reason. Uploading the package & sending a command should be
sufficient as an integration test.
changelog_begin
changelog_end
* Generate security evidence by documenting security testcases.
CHANGELOG_BEGIN
CHANGELOG_END
* move generated file to root of repo, so links work
* formatError function instead of Show instance
* dont use Show instances for generating Markdown
* magic comment: SECURITY_TEST --> TEST_EVIDENCE
* use megaparsec and Data.Text
* remove redundant T.pack
* use: Text.Megaparsec.Char.space
* Add DA.Internal.NatSyn as a stable package
This module exposes a type NatSyn, to be used for encoding
type synonyms of Nat-kinded types
* Add generalized roundtrip test helpers in LFConversion tests
roundtripTestsBy and roundtripTestsPartialBy are like their
non-By versions, except they take an explicit equality predicate
instead of relying on the Eq instance. This allows the source
and target types to differ.
* Define encoder/decoder for type synonyms
This works by saturating the RHS of the declaration with artificial
variable names and adding the corresponding parameters on the LHS
In the case of Nat type synonyms, the Proxy-like
DA.Internal.NatSyn.NatSyn type is used to wrap the value into
something of kind star.
* Use type synonym encoder/decoder when generating/consuming DALF
This closes#11226
changelog_begin
changelog_end
* Extend type synonyms over data-dependencies test
* Add test cases for multi-param type class synonyms
This adds an experimental `toTypeRep: forall t. t -> TypeRep` builtin.
It will only work on interface payloads and crash horribly otherwise.
CHANGELOG_BEGIN
CHANGELOG_END
* Suport multi-party readAs in triggers
fixes#7640
This does not yet include the trigger service. We’ll tackle that separately.
changelog_begin
- [Daml Triggers] Triggers now support readAs parties. They can be
specified via `--ledger-readas a,b,c`. As part of this change
``testRule`` gained an extra argument to specify the `readAs`
parties. If you previously used
```
testRule trigger party acsBuilder commandsInFlight s
```
you now need to use
```
testRule trigger party [] acsBuilder commandsInFlight s
```
changelog_end
* Update triggers/tests/src/test/scala/com/digitalasset/daml/lf/engine/trigger/test/AbstractFuncTests.scala
Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
* Expand type synonyms lazily
We really only need to expand type synonyms in two cases:
1. When pattern matching e.g. to check that a projection on a struct
really happens on a struct. In this case we can first check if we
have a struct in which case we don’t need to expand anything. If we
don’t, we can expand afterwards.
2. When checking two types for alphaequivalence. This PR goes for the
simplest solution: First check alphaequivalence without expanding
anything treating type synonyms nominally. Then if that fails
expand everything and check again. We could be more clever and
interleave expansion with checking for alphaequivalence but this
seems to give us most of the performance benefits while keeping the
logic very simple.
On my local laptop this speeds up typechecking of the full DAR from
380ms to 176ms so roughly a 2x improvement.
For comparison: The Daml-LF 1.7 DAR which still relies on nonminal
typechecking of typeclasses everywhere takes 110ms. So we are still
leaving something on the table here but we’re much closer.
I think this also gets us to the point where speedy compilation is
slower than typechecking but I need to verify that again.
changelog_begin
changelog_end
* Apply suggestions from code review
Co-authored-by: Remy <remy.haemmerle@daml.com>
* fix biuld
changelog_begin
changelog_end
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Test authorization within Exercise choice.
CHANGELOG_BEGIN
CHANGELOG_END
* fix build following rebase
* 4x shouldBe a Right
* test for exercise within exercise
* implicit Name.assertFromString
* def --> val. massive speedup!
* be more private
* implicit Identifier creation
* use implicit for Party and ChoiceName creation
* comment test purpose
* delete stray text; fix build
* Typecheck nested LF type applications more efficiently
This improves performance typechecking in the Scala LF typechecker by
matching on nested foralls & type applications which allows us to
check them with only a single substitution as opposed to the previous
state of doing one for each type application.
This is about a 25% improvement on typechecking performance of
GHC.Classes from 200ms to 150ms.
changelog_begin
changelog_end
* interfaces: Do some TODOs
- Add uniqueness check between fixed choices and virtual choices in
haskell decoder.
- Encode interface methods and fixed choices in scala encoder.
- ExprIterable for interfaces.
changelog_begin
changelog_end
* scalafmt
* [Self-service error codes] Refactored for Canton integration
* Introduces ErrorCodeLoggingContext which is used to abstract away the logging tech stack
CHANGELOG_BEGIN
CHANGELOG_END
* Moved all error definitions in //ledger/error
* Formatting
* Docs and remove of implicit conversion to DamlErrorCodeLoggingContext
* Fix compilation issues
* Fix tests
* Addressed Pawel's review comments
* Addressed Robert's review comments
* interfaces: introduce TemplateOrInterface class in stdlib
The template typeclass is to strong for many applications. The new
constraint `TemplateOrInterface` only contains the methods to convert
contract IDs and choices.
CHANGELOG_BEGIN
CHANGELOG_END
* ghc-lib update, interface script tests
* pinned stackage on unix
* added missing implementation in preprocessor
* added test
* fixing tests
* remove Iface type
* pinned stackage windows
* make sure createAndExercise is not called on interfaces
* LF: clarify compilation of choice Body.
This is a purely cosmetic choice.
We move up the intro of the choice arg binder to make it clear it
affect the compilation of every expression in the body of a choices (
namely controllers, observers, and update expression).
CHANGELOG_BEGIN
CHANGELOG_END
* formatting
- enable NonUnitStatements, and find a useless statement.
- add missing `final`s,
- replace `{ ... }` by `( ... )` for one statement anonymous function
- drop useless `new` for case class.
- replace `sealed class X { ... object Y extend X` by
`class X private[Z] { ... val Y = new X` to avoid unecessary class
alocation.
- move invariant check inside the body of some case class instead of
the companion object.
- other cosmetic changes.
CHANGELOG_BEGIN
CHANGELOG_END
* interface methods: Scala Typechecker
Part of #11006.
I tried to match the behavior of the Haskell typechecker.
changelog_begin
changelog_end
* scalafmt
* Fix type
* interface methods: Scala AST
Part of #11006
This PR takes care of the Scala AST and decoder.
It leaves the encoder, type checker, and speedy for later.
changelog_begin
changelog_end
* scalafmt
* AstRewriter
* interface methods: Speedy
Part of #11006
changelog_begin
changelog_end
* scalafmt
* Remove InterfaceMethods test
* Update daml-lf/archive/src/main/scala/com/digitalasset/daml/lf/archive/DecodeV1.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
Co-authored-by: Remy <remy.haemmerle@daml.com>
* interface methods: Scala AST
Part of #11006
This PR takes care of the Scala AST and decoder.
It leaves the encoder, type checker, and speedy for later.
changelog_begin
changelog_end
* scalafmt
* AstRewriter
* Add NonUnitStatements warning in daml-lf/transaction & fix using discard or similar.
CHANGELOG_BEGIN
CHANGELOG_END
* address comment
* prefer parens to braces for single statement map/foreach bodies
* interface methods: Add protobuf
Adds protobuf definitions for interface methods and calling them.
Encoders/decoders just ignore the extra stuff or error out.
Part of #10810 (maybe)
changelog_begin
changelog_end
* Update issue numbers where appropriate
* update stable protos
(Also: generalize type of `enrichTransaction` to work for _submitted_ and _committed_ transactions & adapt existing callers)
This change paves the way to remove the `transactionNormalization` flag, and always normalize transactions coming out of the engine.
CHANGELOG_BEGIN
CHANGELOG_END
1st attempt. Causes package recompilation (bad!).
CHANGELOG_BEGIN
CHANGELOG_END
fix build
Change ValueEnricher interface to work without passing an Engine
ValueEnricher has optional preprocessor
simplify new interface to ValueEnricher: caller passes translateValue function
Part of #10810
- Implemented ToInterface, FromInterface in speedy
- Added a test that exercises and fetches an interface
- Fixed a bug in SBUChoiceInterface
Hey, interfaces work now!
changelog_begin
changelog_end
fixes#10977
Turns out assertions are good unless they’re wrong …
This only affects scenarios, the engine never looks at the callback.
changelog_begin
changelog_end
* ifaces: name collision, typecheck fetch/exercise
This adds name collision detection and adds typechecking for
fetch/exercising of interface instances.
CHANGELOG_BEGIN
CHANGELOG_END
* turn on exercises in InterfaceDesugared test case
* Update compiler/daml-lf-tools/src/DA/Daml/LF/TypeChecker/NameCollision.hs
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* interfaces: scala typechecker implementation
This is the scala side of the lf typechecker for interfaces.
CHANGELOG_BEGIN
CHANGELOG_END
* added collision check
* added exercise/fetch typechecking
* review suggestions
* added todos for collision/typing scala tests
* Extracted common error implementations
* ErrorGroups
* TransactionError hierarchy
* LedgerApiErrors
* PackageServiceError
* ProtoDeserializationError
* PruningServiceError
* SubmissionErrors
CHANGELOG_BEGIN
CHANGELOG_END
* Adapt extracted error groups and definitions to adhere to local tech stack:
* Use //ledger/error:error core API
* Use DAML SDK logging stack
* Manual rebase to latest changes from Canton
* Extracted RejectionGenerator from Canton
* Adapted RejectionGenerator
* Added //ledger/error:error to artifacts
* Pass correlationId to errors
* Pass parameters as implicits to shave some lines and improve readability
* Workaround for encountered Scala/JDK8 bug
* Addressed Ratko's review comments
* Separate exercise & fetch for interfaces from templates
part of #10810
changelog_begin
changelog_end
* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Base.hs
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Base.hs
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
99% of our usecases use Value[ContractId] so this PR just fixes it.
The few other usescases are:
1. Value[Nothing] which we use for keys. This is technically more
precise but we benefit very little from it.
2. Value[String] mostly because in a few places we are lazy.
We don’t have any code which benefits from being polymorphic in the
contract id type.
changelog_begin
changelog_end
Following #10763, we drop the ad-hoc builders for `FrontStack`.
* Building a `Fronstack` from individuals elements should be done with
standard scala buidler.
* Building a `Fronstack` from a `TraversableOne` should be done with
the scala 2.13 `.to(FrontStack)` methd
* Building a `Fronstack` from a `ImmArray` should be done with the
`toImmArray` method.
CHANGELOG_BEGIN
CHANGELOG_END
* interfaces: protobuf encoder haskell side
This is the implementation of the protobuf encoder on the haskell side.
CHANGELOG_BEGIN
CHANGELOG_END
* missing decoder cases for added interface data constructor
* Enhance InMemoryLedger to use the ValueEnricher.
Allowing SandboxServer to run Engine with the default transactionNormalization=false
CHANGELOG_BEGIN
CHANGELOG_END
* enrich only when responding to verbose API queries
Since we switch to scala 2.13, ImmArray companion object extends
`Factory`. Hence:
- the `apply` methods of `ImmArray` override the one from `Factory`
- we can use the notation `.to(ImmArray)` to convert an `Iterable` to
`ImmArray`
This PR drops those `apply` ImmArray. Conversion from Iterable to
`ImmArray` should use the `.to(ImmArray)`.
CHANGELOG_BEGIN
CHANGELOG_END
This gets us into a state where PartialTransaction always has the
authorization context which is much more sensible than having the
committers in speedy while the rest is in ptx.
The reason for having the split in the first place was the scenario
service but now that we create a new speedy machine per submission
that’s no longer a valid reason.
changelog_begin
changelog_end
Create normalized TXs when a partial TX is finalised.
Except in limited cases! (i.e for scenario-runner, sandbox)
CHANGELOG_BEGIN
CHANGELOG_END
normalize values in the engine as they are converted from speedy-values
fix 2.12 build
backout redundant change
ensure byKey field is correctly normalized when constructed by engine
rename flag: valueNormalization -> transactionNormalization
improve comment
delete commented-out code
rename: toValueNorm --> toNormalizedValue
rename: (SValue.) toValue --> toUnNormalizedValue
revert changes to ptx so that the interface to insertCreate() etc is Value-based (not SValue-based)
improve comments
respell: toUnNormalizedValue --> toUnnormalizedValue
fix build
This PR makes possible to check for contract IDs suffix during
preprocessing.
This is the first part of the task 3 described in #10504.
CHANGELOG_BEGIN
CHANGELOG_END
This PR enforces that LF Comparisons (both Ordering and Equality)
fails at runtime if comparing local vs global CID.
This corresponds to the second task of #10504.
CHANGELOG_BEGIN
CHANGELOG_END
* Upgrade to a newer canton version (post 0.27.0 snapshot version)
with canton-community configuration that supports higher throughput.
changelog_begin
changelog_end
* Disable flaky reject DeeplyNestedValueIT:Reject tests that time out half the time
As stated in #10504 the contract ID freshness check cannot be
implemented correctly in general.
This PR drops the support for this (buggy) check.
This corresponds to the fist task of #10504.
CHANGELOG_BEGIN
CHANGELOG_END
* build correctly versioned empty tx
* dont call normalizeTx from Engine.validate
* dont recalc the tx-version in normalizeTx; just assume it is correct
CHANGELOG_BEGIN
CHANGELOG_END
* Normalize transactions & values as a separate pass. Use for simpler defintiion of isReplayedBy.
CHANGELOG_BEGIN
CHANGELOG_END
normalize transaction version
* remove stray import from bad merge which breaks scala 2_12 build
* change isReplayedBy to only norm its RIGHT (replay) argument
* add forgotton normalization for ValueEmum
* switch to use existing value normalization code (remove my newly coded duplicate code)
* normalize submittedTransaction before calling engine.validate
* dont call normalizeTx from Engine.validate
* *do* call normalizeTx from Engine.validate
* Upgrade Scalatest to v3.2.9.
Because of some coupling we also have to upgrade Scalaz to the latest
v7.2 point release, v7.2.33.
The Scalatest changes are quite involved because the JAR has been broken
up into several smaller JARs. Because Bazel expects us to specify all
dependencies and doesn't allow transitive dependencies to be used
directly, this means that we need to specify the explicit Scalatest
components that we use.
As you can imagine, this results in quite a big set of changes. They
are, however, constrained to dependency management; all the code remains
the same.
CHANGELOG_BEGIN
CHANGELOG_END
* http-json-oracle: Fix a Scalatest dependency.
* ledger-api-client: Fix a Scalatest dependency.
* LF 1.6 ledger export integration test
Test that Daml ledger export script handles DALF packages in LF version
1.6. These packages don't contain metadata, meanint `--package` flags
have to use the hash rather than the package name, and they contain no
type class instances, meaning the export script needs to contain newly
defined instances for type classes required to issue ledger commands in
Daml script.
changelog_begin
changelog_end
* Expose unconstrained Daml script commands
This will be used in Daml ledger exports of contracts defined in Daml LF
versions before 1.8. These versions did not include typeclass instances,
meaning that instances such as HasTemplateTypeRep need to be recreated
in the export Script. Not all instances required in the `Template t`
constraint can be recreated at the use-site. E.g. `HasSignatory`. By
avoiding these kinds of constraints it is possible to handle these cases
in Daml ledger exports anyway.
changelog_begin
changelog_end
* Identify all templates with missing instances
changelog_begin
changelog_end
* Use internal*Cmd on templates missing instances
changelog_begin
- [Daml export] Daml ledger export now handles templates in packages
using LF versions 1.7 or older. These package versions don't include
type class instances and Daml ledger export needs to generate
replacement instances in the generated script. The generated script
uses less type-safe versions of Daml script ledger commands.
changelog_end
* Add encodeType to encode Ast.Type in ledger export
This will be required to encode HasContractKey instances.
changelog_begin
changelog_end
* Simplify newline handling in encodeExport
* Encode missing template type class instances
* ZIP entries must use forward slash as path separator
* Throw error on encode of Any
Addressing
https://github.com/digital-asset/daml/pull/10526#discussion_r685807454
* Qualify choice using argument type
Addresses review comment
https://github.com/digital-asset/daml/pull/10526#discussion_r685817745
* Use lf.language.Util
Addresses
https://github.com/digital-asset/daml/pull/10526#discussion_r685827150
* Factor out header comment in tests
Addressing review comment
https://github.com/digital-asset/daml/pull/10526#discussion_r685830832
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Expose set of stable-packages package-ids
Needed in Daml ledger export to decide whether to depend on a package
without metadata or not.
changelog_begin
changelog_end
* Handle package dependencies without metadata in Daml export
This addresses the immediate issue reported in
https://github.com/digital-asset/daml/issues/10435.
DALF before LF version 1.8 don't contain metadata such as the package
name and version. However, Daml ledger export assumed that metadata was
available to generate `--package` flags for data-dependencies.
This change generates `--package=<hash>` flags for such dependencies on
packages that don't have metadata available.
This change requires additional care when checking if a package is a
stable-package. As the package name is not available without metadata we
cannot just check against names such as `daml-stdlib`. Instead this
change uses the list of stable-package package-ids exposed by
`com.daml.lf.language.StablePackages.Ids`, which was introduced in
0da814d250 (diff-208c6948c7c7f2b582faa8e4792bfe78b231188881947ad0cdd5046e0b9a40e7)
Note that this does not fully resolve#10435. Building a generated Daml
ledger export script will fail on missing instances of standard classes
such as `HasTemplate` because packages prior to LF version 1.8 don't
contain type-class instances. This will be addressed in a separate PR.
changelog_begin
changelog_end
* Fix Scala 2.12
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
part of #9974
- rename `msg` field to `message`
- drop `engine.Error.Validation.Generic`
- rename argument `where` to `location`
- Builtin set the `dependsOnTime` before throwing the callback `SResultNeedTime`
CHANGELOG_BEGIN
CHANGELOG_END
* logging-entries: Make `LoggingEntries` a non-case class.
There's no reason for it to need `equals`, etc.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-api-domain: Convert commands into a logging value.
Instead of having a function, let's use `ToLoggingContext`.
This also adds a couple of missing items, and always logs `workflowId`.
* participant-state: Convert updates into a logging value.
Instead of having a function, let's use `ToLoggingContext`.
This changes some of the logging context structure, but otherwise
everything remains the same.
* Make sure Scaladoc is lined up for modified code.
* Stable packages bypass version restrictions.
Part of #10029
changelog_begin
changelog_end
* Add missing bypass
* scalafmt
* Fix tests
* fix script runner
* Fix scenario service
* Dont make stable packages configurable
* unused import
* revert unnecessary formatting changes
* Added a StableOnly engine mode and a conformance test that uses it
* buildifier-fix
* Inline LanguageVersions.StableOnly
* scalafmt
* Remove EngineMode in favor of representing the version range directly
* Add -unsafe to flag name
* Apply suggestions from code review
We drop this check for the following reasons:
* Its only usage was inside the transaction builder, this is subsumed
by the check we added in the SValue to Value translation (#10370)
* It seems to origianl check more that nesting (and so to be
overcomplicate)
* It was used in non-consitent way. It was used to check create
arguments and choice arguments, but niether choice result nor
contract key.
CHANGELOG_BEGIN
CHANGELOG_END
The conversion of SValue to Value already ensures the resulting value
has a serializable type. Here we add a check to ensure it does not
overpass the maximum allow nesting.
* Limit length of package ids to 64 characters
In practice, you cannot use a package id that is not a valid sha256
outside of a test anyway since we validate them. However, we didn’t
enforce it at the daml-lf level which is a bit annoying.
This PR fixes this and adds a length restriction of 64
characters (i.e. 256 bytes for sha256).
We could go a bit further and also restrict the characters to valid
sha256 hashes (i.e. 64 character hex strings). I don’t feel all that
strongly about that either way. We use other characters in tests but
fixing that shouldn’t be particularly hard either. The extra
characters don’t seem to cause problem so far, so I think it’s fine to
keep that.
No changelog entry since I don’t see how a user can be affected by this.
changelog_begin
changelog_end
* Update daml-lf/encoder/src/main/scala/com/digitalasset/daml/lf/archive/testing/DamlLfEncoder.scala
* participant-state: Remove the aliases to `Ref.LedgerString`.
Use the aliases directly in `Ref` instead.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-lf/data: Improve comments on ID aliases.
* daml-lf/data: Fix a typo in a comment.
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* Pass commitLocation along in engine warning log.
changelog_begin
changelog_end
* Use commitLocation in diagnostics.
* scalafmt
* update divulgence test
* Fix commitLocation and expected diagnostic locations
* daml-lf/data: Move ID aliases to `Ref` from _ledger-api-common_.
This allows us to remove a lot of dependencies on _ledger-api-common_,
and use these aliases in other places where that module is not used.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-integration-api: Remove an unused import.
* http-json-oracle: Remove `ledger-api-common` as a dependency.
* bindings-rxjava: Remove a now-unused dependency.
* Remove Optional[Location] field from transaction nodes.
xoptLocation
working but very messy!
cleanup PartialTransaction code
pass locationInfo in CompleteTransaction
dont pass locationInfo in SumittedTransaction
pass optLocation to CheckAuthorization
temp reinstate xoptLocation in ExerciseContextInfo
cleanup Conversions.scala
remove Optional[Location] field from transaction nodes.
CHANGELOG_BEGIN
CHANGELOG_END
* fix bad merge
* fix build after merge
* rename
* doc comment, and question
* LF: change type from Try to Either in archive module
This is the first part of restructuring errors in archive module.
This is part of #9974.
CHANGELOG_BEGIN
CHANGELOG_END
* Apply suggestions from code review
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* remove type alias
* apply stephen suggestion
* fix after rebase
* fix test
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Separate traces from warnings in engine.
I decided to separate the engine warnings from the tracelog after all,
because I think it will make testing and maintenance easier in the
long run.
Part of #9947, follow up from #10179
changelog_begin
changelog_end
* scalafmt
* Apply suggestions from code review
Co-authored-by: Remy <remy.haemmerle@daml.com>
* dont use case class for WarningLog
* revert TraceLog changes from last PR
* Scala 2.12 doesnt have ArrayBuffer.addOne :(
* remove isWarnEnabled check
Co-authored-by: Remy <remy.haemmerle@daml.com>
* LF: Make DarReader ZipEntries immulatble
CHANGELOG_BEGIN
CHANGELOG_END
* optim
* Revert "optim"
This reverts commit 3b25a8c68b.
* a function should not close a stream it didn’t open
* typo
* we do not support dar without manifest !
* LF: Simplify archive reader.
- decouple Reader and Decoder
- introduce case class to handle hash, proto payload, and version
CHANGELOG_BEGIN
CHANGELOG_END
* Address Moritz' review
* cosmetic
We probably want to start displaying the full ptx on the client side
but for now this at least moves things out of speedy and into the
rendering layer where they belong.
It also fixes IncompleteTransaction to unwind properly so the roots
are really the roots.
changelog_begin
changelog_end
* Use PartialFunction for more concise code.
CHANGELOG_BEGIN
CHANGELOG_END
* tweak to use guards
* blind fix for 2.12
* try fix 2.12 build
* loose if-guard parens
* fixes for 2.12 build (should actually work this time!)
* Add divulgence warning and test in script service.
Part of #9947, building on the key visibility checks from #10136
This PR adds a divulgence warning inside the traceLog.
I wasn't sure whether:
1. these warnings should be kept in a separate structure from the
traceLog, and therefore transmitted separately, or
2. these warnings should be kept together with traces, but the severity
should be tracked and also transmitted over grpc, and warnings should
be logged as warnings instead of as debug messages, or
3. these warnings should be kept together with traces, but
logged as warnings instead of debug messages,
4. these warnings should be treated exactly like traces
I'm leaning toward #2, but this PR implements #3.
This PR tests the warning via the script service tests.
changelog_begin
changelog_end
* scalafmt
* Address Moritz's review
* Rename traceLog.add to traceLog.addDebug
* fix test
* Add test using exercise
* add single transaction test
