* ifaces: Add exercise interface semantics to spec
This is based on the semantics of 'exercise'.
Closes#11349 (for now). Also fixes a small bug in the semantics of 'exercise'.
changelog_begin
changelog_end
* Uniformize exercise syntax
This add a first test for evaluation order of exercise_interface and
fetch_interface. More tests will come in the following PR's.
This is part of #12697.
CHANGELOG_BEGIN
CHANGELOG_END
We add tests to check that by_interface fields are set for interface
transactions. We also extend the scenario service to show the
by_interface fields in the pretty printed transactions.
Fixes#13333
CHANGELOG_BEGIN
CHANGELOG_END
* explore less bouncing; convert every: Bounce -> yesBounce/noBounce
changelog_begin
changelog_end
* remove noBounce; yesBounce -> Bounce (with curly braces)
* multi args list for better layout
* use shadow variable naming more consistently
* reduce unnecessary polymorphism
* reinstate 2x Bounce in atomizeExps which are necessary
* avoid 2x linear args.size
* avoid pointless conversion to/from Array
* move definition of Res to correct place
* remove 2x comments to type param A which has been removed
* inline 2x unnecessary val defs
* enable Anf stack-safety tests for EUpdate cases
* fix stack-safety for closures in Anf transform
* fix stack-safety for try-catch handling in Anf transform
* add missing testcases for scope-exercise/prevernt-catch ANF stack-safety, and fix the overflow
changelog_begin
changelog_end
* interfaces: tests for requires typing
We add tests for the correct typing of the interface requires field.
CHANGELOG_BEGIN
CHANGELOG_END
* security evidence
This adds the presence of interface names, as well as lists of choices
and their argument/return types for each interface.
It also changes the semantics of DefTemplate and EnvironmentInterface:
each may contain "unresolved choices", which are the choices inherited
from interfaces for each implementing template;
EnvironmentInterface#resolveChoices will copy the full type information
as a concrete choice for each implementing template, which should be
much simpler for codegen and json-api.
A type generator should take account of the astInterfaces as well as
typeDecls in EnvironmentInterface; interfaces aren't included in
typeDecls because where I is an interface only ContractId I is
serializable.
* Ast.DataInterface is empty, just use the interfaces map directly
* port json-api for interface API changes; does not solve #12689
* port Scala codegen for interface API changes; does not add support
* port Java codegen for interface API changes; does not solve #11350
CHANGELOG_BEGIN
CHANGELOG_END
This extends the Scala encoder tests to include interface definitions
and interface implementatioins. Some bugs in the encoder and ast
rewriter are fixed.
CHANGELOG_BEGIN
CHANGELOG_END
We split kv replay as follow:
- [LF] we introduce a simple protobuf to store submission entry (which
roughly correspond to TRANSACTION_ENTRY and PACKAGE_UPLOAD_ENTRY KV
submission)
- [KV] we create a tool to extract form a ledger export a sequence of
submission entry
- [LF] we move the replay tool from KV to LF, and base it on
submission entries instead of ledger export
The objectify of this split is double:
- on the one side it will simplify maintenance, as it will make API
between KV and LF more clear (depends only on a protobuf definition)
- on the other side, it will made the use of the tool handy, as it
will make possible to benchmark the engine without direct dependency
between KV and LF.
CHANGELOG_BEGIN
CHANGELOG_END
* Improve MessageDigest and Mac instance creation to solve lock contention problem [DPP-956]
changelog_begin
Scalability bottleneck in regard to hashing has been fixed in multiple places.
changelog_end
Process:
- `git ls-files -z | xargs -0 -n 100 sed -i --follow-symlinks 's/DAML/Daml/g'`
- `git add -p`
- `git restore -p`
- Check there is no unstaged change left.
To review:
- Check for false positives by carefully reviewing the diff in this PR.
- Check for false negatives with `git grep DAML`.
- Quicker check for fals positives:
```
git grep DAML | grep -v migration | grep -v DAML_
```
Fixes#13190
Note: This is the "second half" of #13191, which failed to cover all the
remaining DAMLs because of:
```
$ git ls-files | grep "'"
compiler/damlc/tests/daml-test-files/MangledScenario'.daml
```
CHANGELOG_BEGIN
CHANGELOG_END
* ifaces: tests: check for iface defs in templates
This adds a test that will fail when an interface definition is present
in a module when daml-lf < 1.dev.
CHANGELOG_BEGIN
CHANGELOG_END
* Update daml-lf/archive/src/test/scala/com/digitalasset/daml/lf/archive/DecodeV1Spec.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Update daml-lf/archive/src/test/scala/com/digitalasset/daml/lf/archive/DecodeV1Spec.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Update daml-lf/archive/src/test/scala/com/digitalasset/daml/lf/archive/DecodeV1Spec.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
* remy's suggestions
Co-authored-by: Remy <remy.haemmerle@daml.com>
* ifaces: tests: check for interface collisions in ast
We add a test to check for interface name collisions in the daml-lf AST.
CHANGELOG_BEGIN
CHANGELOG_END
* Update daml-lf/language/src/test/scala/com/digitalasset/daml/lf/language/AstSpec.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Use `this` param in method bodies in LF.
Instead of having method bodies be functions from the template, have them reuse the existing template parameter (usually `this`).
Fixes#13123
changelog_begin
changelog_end
* fix validation-test
* Update 'implements' blocks to use new syntax
* Add InterfaceSyntax test case
* Add InterfaceMultipleMethodDeclsError test case
* Add InterfaceDifferentNumArgsError test case
changelog_begin
changelog_end
* LF Spec: Add interface expressions
This PR adds type rules and operational semantics for interface
expressions (including updates). The only thing I haven't added yet is
the operational semantics for exercise interface, and the operational
semantics for interface preconditions.
changelog_begin
changelog_end
* typo
* spurious duplicate
* minor edits
* review suggestions
* Prevent iface choice & method name collisions.
This PR adds interface choices and methods to the name collision checker.
changelog_begin
changelog_end
* check the names from interfaces
* add error
* implement name collision check in scala side
* scalafmt
* interfaces:ensure that implementations are ordered
We make sure that the implementations are ordered by insertion in the
Scala AST. This is important to guarantee an evaluation order of the
interface preconditions that is determined by the order of interface
implementations of a template.
CHANGELOG_BEGIN
CHANGELOG_END
* use VectorMap instead of ListMap
* Add interfaces to LF spec.
Part of #11349. This PR adds the interface & implements declarations, deals with "requires", and name collisions. I'm leaving expression types and operational semantics to a future PR.
changelog_begin
changelog_end
* missing curly
* change T to I for interfaces
* revisions from Remy
The previous way of tracking this only worked for direct children of
rollback nodes but lost the information when there was an exercise
node in between.
Reported in https://discuss.daml.com/t/exception-rollback/4000
changelog_begin
- [Daml Studio] Fix a bug where contracts that have been rolled back
would still show up as active in the table view and in the list of
active contracts at the end of the transaction view. This only
affected display. Fetching those contracts failed and `query` also
did not return those contracts.
changelog_end
* speedy: short circuit precondition checking
Instead of evaluating all preconditions of templates and interfaces we
evaluate preconditions one by one and throw an exception as soon as we
find a failing one.
This implemented via a lazy SBCheckPrecond builtin and a foldleft
expression over the list of preconditions.
CHANGELOG_BEGIN
CHANGELOG_END
* Update daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/speedy/Compiler.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Update daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/speedy/SBuiltin.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
* fix SUnit -> Unit
* tests
Co-authored-by: Remy <remy.haemmerle@daml.com>
* wrap interfaces in SAny in speedy
* expect SAny representation for interfaces in builtins
* Update SBResolveVirtual
* SBResolveSBUInsertFetchNode uses SAnyInterface
* SBResolveSBUBeginExercise uses SAnyInterface
* Wrap 'this' in SAnyInterface for interface preconditions
changelog_begin
changelog_end
Part of #12792, this is the builtin we need to be able to effectively
store a TemplateTypeRep inside a template (i.e. you store a string
containing the template id, not a typerep).
changelog_begin
changelog_end
* Extracted kvutils Config to //leger/ledger-runner-config
changelog_begin
changelog_end
* Sandbox-on-X logback.xml does not depend on the kvutils/app one
* Remove unused dependencies
* Add copyright header back
* Remove dependency of SoX on kvutils
The computation of Node version relies on the
`packageToTransactionVersion` field of the PartialTransaction. This
is initialized during the Speedy `Machine` construction with the
compiledPackage used to initialized the Machine. However the
compiledPackage is a mutable field and can be udpated during
interpreation, potentially making the 'PartialTransaction''s
`packageToTransactionVersion` out of date.
This is not a production bug, as production code use the Machine's
`CompiledPackage` in a imutable way (with a mutable
ConcurrentCompilePackage), however this is a problem for testing code
that may use the field in a mutable way (with an immutable
`PureCompiledPackage`).
We solve the issue by making the computation of the version, outise
the PartialTransaction object.
CHANGELOG_BEGING
CHANGELOG_END
* Disable implicit party allocation in Daml Studio
fixes#12058
This matches what sandbox does in 2.0 so it reduces a source of
inconsistency and potential confusion.
changelog_begin
changelog_end
* Update compiler/scenario-service/protos/scenario_service.proto
Co-authored-by: Remy <remy.haemmerle@daml.com>
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Extend daml lf parser to support new interface primitives
* Test parser on new interface primitives
* Add daml lf decoding test cases for new interface primitives
* Add TypingSpec test cases for new interface primitives
* add SBuiltinTest test cases for interface primitives
changelog_begin
changelog_end
* Update kvutils Config and SandboxConfig
* Remove `eager-package-loading` and `scenario` from SandboxConfig
* Add `profile-dir` and `stackTraces` to kvutils Config
* Configurable authService in kvutils
changelog_begin
changelog_end
* Addressed Samir's review comments
* Add back Maximum supported deduplication duration enforcement of 1 hour
During refactoring of #9993, the "unsafe" compilation of the seedy command
has been moved out of the scope of `Engine.runSafely`, whose only purpose
was to catch exceptions thrown by the Speedy compiler.
This fixes the issue and renames the function `runSafely` to
`runCompilkerSafely` to make its purpose more obvious.
This is not a critical bug, as the command preprocessing should ensure
that (1) the command are properly typed and (2) all the dependencies
have been loaded.
CHANGELOG_BEGIN
CHANGELOG_END
from 130 to 125, reason the test is currently flaky (It fails around 20% of the time).
Surprisingly, #12678 seems to have increase the stack usage for the type checker.
CHANGELOG_BEGIN
CHANGELOG_END
* Add new primitives to proto spec
* implement E{Signatory,Observer}Interface in terms of EResolveVirtual{Signatory,Observer}
* define EToTypeRep primitive in terms of EToTypeRep Expr
* Remove experimental primitives TO_TYPE_REP and RESOLVE_VIRTUAL_{SIGNATORY,OBSERVER}
changelog_begin
changelog_end
* Upgrade to Scala 2.13.8
changelog_begin
changelog_end
* Update hash for scala in nixpkgs
* update more hashes for scala upgrade
* Fix most warnings etc.
* Fix remaining errors etc.
* Fix formatting
* Resolve last errors hopefully
* Fix ledger api common build file
* Combine imports & revert accidental change that broke the CI run
* Rename exporting vals to scriptExport & minimize diff
* Remove more wrong changes
* moved warning around
Adds a check that experimental primitive is defined in damlc, before you can use it.
This is only for the Haskell side, and only intended so we can catch bugs in the compiler more easily. (It would have caught the bug where `observer` wasn't defined correctly in LFConversion.)
I also removed RESOLVE_VIRTUAL_CREATE since we're not using it anywhere, it has been superseded by UCreateInterface.
changelog_begin
changelog_end
With this change, Daml exerciseByKey use the LF primitive
ExerciseByKey instead of the combinason of FetchByKey + Exercise.
CHANGELOG_BEGIN
CHANGELOG_END
* interfaces should not be implicitly serializable
* Add scala serializability check for interface choices
* Add scala serializability check for interface payloads
changelog_begin
changelog_end
Continues the work started in https://github.com/digital-asset/daml/pull/12543
These libraries were only needed to transition from Scala 2.12 to 2.13
and are no longer useful as all the necessary items are now available
in Scala 2.13.
changelog_begin
changelog_end
Since Scala 2.13.2, Scala introduced built-in support to
manage warnings in a more granular fashion, thus making
the silencer plugin we are currently using no longer
strictly useful. Removing compiler plugins also removes
friction from migrating to Scala 3 in the future. As a
cherry on top, the built-in warning configuration also
allows to check whether a `@nowarn` actually does
anything, allowing us to proactively remove unused
warnings should the need arise.
[Here][1] is s a blog post by the Scala team about it.
Warnings have been either solved or preserved if useful,
trying to minimize the scope (keeping it at the single
expression scope if possible). In particular, all
remaining usages of the Scala Collection API compatibility
module have been removed.
Using the silencer plugin also apparently hid a few
remaining usages of compatibility libraries that were used
as part of the transition from Scala 2.12 to Scala 2.13
that are no longer needed. Removing those warnings
highlighted those.
changelog_begin
changelog_end
[1]: https://www.scala-lang.org/2021/01/12/configuring-and-suppressing-warnings.html
* Remove user-management error cases from scenario-service proto.
changelog_begin
changelog_end
* remove Error.UserManagement
* remove UserManagememtError
* simp
* meh... done over by scala's poor excuse for a type system
As for creates, these tests somewhat ignore ordering on on-cachable
errors relative to each other since those only change error messages.
changelog_begin
changelog_end
* Split channel configuration from LedgerClientConfiguration
Fixes#12391
The channel configuration now has to be provided separately from the
configuration specific to the ledger client. In this way we avoid
situations where the builder is provided with some configuration
that gets overridden.
changelog_begin
[Scala bindings] The channel configuration has been split from the
LedgerClientConfiguration class. Provide the gRPC channel specific
configuration separately or use a builder. The channel configuration
no longer overrides the builder.
changelog_end
* Fix compilation issues in //ledger-service/...
* kvutils: Remove the DAR upload parameters from the runner.
No production ledger needs to upload DARs on participant startup; this
feature is primarily for users of Sandbox. The feature never worked in
the case of multiple participants and was only ever used in testing.
This also removes the associated functionality from Sandbox-on-X as they
share a configuration object. Hopefully this won't be an issue.
Some tests were using this feature, so I have updated them to upload the
DARs through the PackageManagementService instead.
CHANGELOG_BEGIN
CHANGELOG_END
* kvutils: Remove the `writePackagesService` factory constructor.
We no longer use it.
* daml-lf/engine: Revert an unnecessary change.
* daml-script/test: Shut down the channel after uploading the DAR.
* Clarify unhandled exception error message
The previous error doesn’t make it clear that this is an exception in
user code rather than a Scala exception in our code.
Daml-lf exception would technically be more correct but I don’t think
it’s helpful here so I went with Daml exception.
changelog_begin
changelog_end
* adjust ledger api test tool
changelog_begin
changelog_end
* Update test assertions
changelog_begin
changelog_end
* fix another assertion
changelog_begin
changelog_end
Following #12338 which merges the two implementations of the
structural record projection introduced by #7740, this PR merges the two
implementations of the structural record introduced by #7742.
However, this PR does not try to cache the field index as it is done
1- unlike what it is suggested by commit message of #7742, the update
has a linear complexity anyway, as it has to copy the whole struct.
2- the compiler does not produce any structural record updates
As result the PR is basically a revert of #7742.
CHANGELOG_BEGIN
CHANGELOG_END
Currently we have two implementations for the projection of structural
record.
1- The first implementation takes as parameter the index of the
projected field and is therefore constant. This implementations is
used when the type checking is enable, as the index cannot be directly
inferred from the AST and must hence be filled in by the DAML-LF type
checker.
2- The second implementation takes as parameter the name of the
projected field and is therefore logarithmic as the field must be
lookup by binary search at each call. This version is used when the
type checking is disable as the index cannot be inferred without type
inference.
In this PR, we modify the second implementation so it cache the index
at the first call, hence avoiding the recomputation during further
calls. In this way we reach an amortized constant complexity. The
first implementation is decommissioned in benefit of the second one.
The advantages of this approach are:
- We have a unique implementation of the projection, so the behavior
of a program is the same whenever the type checking is on or off.
- The AST for structural projection is immutable.
Benchmarks show no performance differences when the type checking is on.
Based on an idea by @sofiafaro-da.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger api: support Auth0 user names in user management
See the `IdString.UserId` and `IdString.ApplicationId` comments wrt the
character classes being introduced.
Many thanks to @cocreature for helping with deciding on the exact
restrictions of user-ids.
CHANGELOG_BEGIN
CHANGELOG_END
Somewhat error-prone, so please review carefully.
Reasons we need this:
- Some file types are not properly handled by the script.
- The only exclusion mechanism we currently have (`NO_AUTO_COPYRIGHT`)
is overly coarse.
CHANGELOG_BEGIN
CHANGELOG_END
New year, new copyright, new expected unknown issues with various files
that won't be covered by the script and/or will be but shouldn't change.
I'll do the details on Jan 1, but would appreciate this being
preapproved so I can actually get it merged by then.
CHANGELOG_BEGIN
CHANGELOG_END
* Set --enable-scenarios to False by default
changelog_begin
changelog_end
* Enable scenarios for shake test
* Add --enable-scenarios=yes for //compiler/lsp-tests
* Add enable_scenarios arg to rules_daml/daml.bzl helpers
* daml_compile
* daml_build_test
* daml_test
* Add enable_scenarios to ledger/test-common helper da_scala_dar_resources_library
* Add --enable-scenarios=yes for //ledger/test-common
* Remove unused scenario in //ledger/test-common PingPong test
* Add --enable-scenarios=yes for //daml-lf/tests
* Add --enable-scenarios=yes for //language-support/java tests
* Add --enable-scenarios=yes for group-chat example
* Add --enable-scenarios for //ledger/sandbox-perf LargeTransaction test
* Add --enable-scenarios=yes for //docs tests
* Add --enable-scenarios=yes for //daml-lf/scenario-interpreter tests
* format bazel
* Add --enable-scenario=yes for 'daml_doc_test's
* Add --enable-scenarios=yes for DamlDocTestIntegration
* Add --enable-scenarios=yes for DamlcTest
* Add --enable-scenarios=yes for Test/DataDependencies
* Add --enable-scenarios=yes for daml-ghc-deterministic.sh
* Add --enable-scenarios=yes for Test.IncrementalBuilds
* Add --enable-scenarios=yes for tests:memory-bond-trading and tests:memory-examples
* Add --enable-scenarios=yes for daml-assistant/integratio-tests
* TransactionNodesStatistics.stats now returns case class of same name
changelog_begin
changelog_end
* Rename TransactionNodesStats to Detail and make inner class
* Rename TransactionNodesStatistics.stats to apply
* Rename TransactionNodesStatistics to TransactionNodeStatistics
* Reformat
* Remove rollback from statistic action count
* Update with review comments
* Update with review comments
* Remove Extractor
Extractor is being removed after a long time being in Labs status.
This should improve the flakiness on CI.
🔥
changelog_begin
Extractor has been removed from the SDK
changelog_end
* Remove Extractor documentation
* Remove Extractor from CODEOWNERS
* Remove Extractor references in Daml-LF build files
* Remove Extractor references in the Daml SDK assistant
* Remove Extractor from the SDK
* Remove Extractor reference from CONTRIBUTING.md
The method is not correctly implement, as it should slice the
underlying array before returning it. Since it is not possible to
implement it in constant time, we simply drop it, in favor in other
conversion.
CHANGELOG_BEGIN
CHANGELOG_END
* Escape daml-lf tracelog messages
Currently veracode complains because this allows for clrf
injection (injecting newlines to make user input look like separate
log statements).
With this change
```
debug "abc"
debug "eaiu\neaiu"
debug "def"
debugRaw "abc
```
is logged as
```
[DA.Internal.Prelude:555]: \"abc\"
[DA.Internal.Prelude:555]: \"eaiu\neaiu\"
[DA.Internal.Prelude:555]: \"def\"
[DA.Internal.Prelude:555]: abc
```
You can debate whether we should escape the quotes are necessary but
90% of the reason why people add them is because they call `debug` on
strings when they should be using `debugRaw` so this seems fine to me.
changelog_begin
changelog_end
* fix tests
changelog_begin
changelog_end
as it is normally caught in the interpreter loop. In particular the
caller of the engine should not have to catch such an error.
CHANGELOG_BEGIN
CHANGELOG_END
* Update TODOs to outdated issues.
In particular update a lot of defunct interface TODOs to the LF 1.15
issue (or add the LF 1.15 issue if it's also relevant).
changelog_begin
changelog_end
* typo
* remove numeric/bignumeric todo
* scalafmt!!
* drop choice observer TODO
changelog_begin
changelog_end
add runtime check in freeVars: determination that a variable is-free using levels instead of indexes
remove DB-indexes and runtime check; simplify freeVars computation in closure-conversion
Part of #11978. Adds typechecking for this field on the interface side,
and enforces that any template that implements A must implement B if A requires B.
CHANGELOG_BEGIN
CHANGELOG_END
As mentioned in the comment, I view these tests as a way to guard us
against bricking ledgers on upgrades so they somewhat deliberately
ignore ordering of non-cachable errors relative to each other since
those only change error messages.
changelog_begin
changelog_end
* Remove the use of nesting with `size` and `resize` approach as suggested by scalacheck.
CHANGELOG_BEGIN
CHANGELOG_END
* Biased to shorter names generator for Name
After staring at a type mismatch between `Nested` and `Nested` this
seemed like a sensible change. We could also include the package
ids. I don’t have strong feelings about this but this seems like a
clear uncontroversial improvement.
changelog_begin
changelog_end
This is part of #11691
This PR allows to limits:
- the number of signatories,
- the number of observers,
- the number of controllers,
- the number of inputContracts,
CHANGELOG_BEGIN
CHANGELOG_END
* Add skeleton for trace-based LF evaluation tests
The recent issues around wrong ordering of contract id typechecks got
me thinking about this again so I wanted to hack up a PR of how we can
test evaluation order sensibly directly at the LF level.
This is only testing a single create but I first wanted to see if we
agree that this is a sensible approach using a simple example.
If we agree that this is a sensible approach, I’d suggest to extend
those tests later in separate PRs to make them exhaustive.
changelog_begin
changelog_end
* cleanup
changelog_begin
changelog_end
* Adding missing stack-safety testcase for SEScopeExcerise.
changelog_begin
changelog_end
* run all testcases at depth of 10,000. This is plenty deep enough to ensure stack-safety
* recode free-vars computation to be stack-safe. and test
* Drop support for Daml-LF party literals from the Scala side
This PR enforces that forbidPartyLiterals is always `true` and drops
the corresponding literals from the AST. Haskell side is in #11930fixes#11581
changelog_begin
changelog_end
* Update daml-lf/interpreter/src/test/scala/com/digitalasset/daml/lf/speedy/ComparisonSBuiltinTest.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Revert "Update daml-lf/interpreter/src/test/scala/com/digitalasset/daml/lf/speedy/ComparisonSBuiltinTest.scala"
This reverts commit 55e542ce4e3a7fd15544ee703de3277ffc309b17.
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Use Absolute-indexes as keys for the Env-mapping during closure-conversion.
Do runtime check to confirm behaviour matches the existing Relative-indexes.
changelog_begin
changelog_end
* remove quadratic shift!
remove (dev)pretty-print code
remove relative-index keys from Env-mapping
remove runtime *diff* check
increase depth for stack-safety tests
* improve/simplify indexing calculation for Env-keys
* Add type rep argument for interface exercises.
(Still WIP.)
Part of #11703. Fixes the order in which errors are raised ("wrong type"
takes priority over "does not implement interface"). This PR also simplifies
ExerciseInterface by making the guard mandatory, otherwise there's too
many variations. We can revisit that later if we want.
changelog_begin
changelog_end
* haskell side
* dont throw exception in checkTemplateId
* scalafmt
* evidence security
* fix TypingSpec test
* SExpr1.SELocS - carry relBad/absGood - abs unused so far
* compute/pass SELocS.abs in ClosureConversion, and check in Anf that it matches the reconstructed value
CHANGELOG_BEGIN
CHANGELOG_END
* Remove relative stack locations. Rename as SELocAbsoluteS. Simplify Anf. Remove shiftLoc in ClosureConversion.
* Prevent wrongly typed fetch by interface.
When doing a "fetch by interface" command with a known template id,
error out with a WronglyTypedContract if the fetched contract has
a different template id. This doesn't affect daml, only affects
replays, so it's rather minor. I also enabled the engine test that
caught this.
Part of #11703, follow up to #11836.
changelog_begin
changelog_end
* strengthen test output checks
* Drop DA.Next.Set and DA.Next.Map
Fixes#11527. Also removed a bunch of unnecessary CPP, though there's
still a lot more CPP to remove.
changelog_begin
- [Daml Standard Library] DA.Next.Map and DA.Next.Set have been removed
after being deprecated since Daml-LF 1.11
changelog_end
* Drop a deriving MapKey
* update unstable-types test
* Add a guard when exercising by interface.
This fixes part of #11703, when exercising an inherited choice by
interface and you know the template id, via the command preprocessor.
It does this by inserting a "guard" in between the interface fetch and
the exercise body. The guard is a function Interface -> Bool, which
is general enough to check the template id, without complicating too
much in speedy. And can be generalized in the future to check more,
like signatories, etc.
I added the guard as an optional argument to UExerciseByInterface.
This isn't hooked up to the protobuf AST yet (or Haskell side for
that matter) -- but I'll do it in the next PR! For now you can invoke
the guarded exercise via the command preprocessor, so I can enable the
approprate engine tests. (There's still some failing fetch tests left,
but I decided to leave this for later. Fetch can be a lot simpler than
guarded choices, since you always add a fetch node. No need for fancy
continuations.)
changelog_begin
changelog_end
* scalafmt
* Feedback and fix matches
* Update comments, we are always going to abort the transaction
* Raise WronglyTypedContract in SBGuardTemplateId.
* rebase and fix parser
* restore ANF
* scalafmt
ClosureConversion -> Suffix with "Old"
CHANGELOG_BEGIN
CHANGELOG_END
ClosureConversion old-vs-new diff check
ClosureConversionNew, first cut. All tests in SBuiltinTest work.
In addition we change some Array --> List in SExpr1 (for human pp).
And we throw away ClosureConversionDup.
adapt AnfTest from Array to List change for SExpr1
all tests pass in daml-lf/interpreter
remove SExpr0.SELet1General
reorder things
testing for stack-safety of closure conversion
file/class renames
improve naming
pass cont as sep arg to commit (move out of Up/Down)
comment stack-safe closure conversion
fix bug: failed to use env1
fix 2x unmoored doc comment
comment stack safety testing
Remove old closure-conversion code & diff-check between old/new.
loose StackSafe suffix on ClosureConversion class/file
rename StackSafetyTest.scala to ClosureConversionTest.scala
prefer "sealed abstract class" to "sealed trait"
fvs.zipWithIndex --> fvs.view.zipWithIndex
(SExpr1) SEAppGeneral -> SEApp; prefer List to Array in SEApp/SECase
prefer xs.toArray to Array(xs: _*)
access SExpr0 via "source."
two more .view
improve comment and fix typo
link to Issue
switch to a continuation stack; avoids nesting in the Cont type
* Drop LF < 1.14 from supported damlc output versions
fixes#11319
We keep test coverage by depending on the most recent snapshot which
still has 1.14 support.
changelog_begin
- [Daml Compiler] Damlc can only produce Daml-LF 1.14 or
newer. Passing aynthing older to `--target` is an error. If you
need to produce older versions, use an older SDK.
changelog_end
* Switch around legacy_compiler_lf_versions
changelog_begin
changelog_end
* drop since-lf
changelog_begin
changelog_end
This fixes a bug in the typechecker (#11558) and the command
preprocessor, since those were written with this behavior of
lookupTemplateChoice in mind. Enables the engine test that
caught this.
changelog_begin
changelog_end
- rename non-default builders `apply` to `build`
* avoid confusing both
* make explicit the build can crash
- make interfaceId and templateId fields consitent
- use when possible named arguments
- check for non-repetition of inherited choices
CHANGELOG_BEGIN
CHANGELOG_END
We revert #11626, and just change the way transaction version is
computed:
- As before, Node version is calculated from the
package of the template ID action.
- Transaction version is the max of the version of all the nodes,
instead of the root nodes.
CHANGELOG_BEGIN
CHANGELOG_END
* Add engine tests for interfaces.
Adds engine tests using the new commands. This uncovered a few issues:
- getDependencies doesn't work when given an interfaceId as a templateId
(such as when exercising an interface via the ledger api). I patched
it up so it deals with interface ids. We've already discussed a way to
simplify getDependencies that will also avoid this in the future, but I'll
leave that to the future.
- issue #11703 is confirmed via some tests that don't pass (and are
currently disabled)
- PackageInterface.lookupTemplateChoice returns inherited choices, when it
should only return own choices. At least, the typechecker assumes
it doesn't -- this affects #11558 -- as does the command
preprocessor. I'll leave the cleanup to a separate PR.
changelog_begin
changelog_end
* scalafmt, add missing file
* Move interfaces tests to separate file
* scalafmt
* Update to Java 11
changelog_begin
changelog_end
* Fix RoundingMode deprecation warnings
* Fix dep-ann warning
* Integer constructor
* JavaX annotation dependency
* javax.xml.bind was removed in Java 11
Using Guava as a replacement, since it is already a project dependency.
* JDK 11 no longer has a separate JRE tree
* Remove unused jdk_nix import
* remove now redundant jdk11_nix
* Java 8 --> 9 increased Instant.now() precision
See https://bugs.openjdk.java.net/browse/JDK-8068730
The precision of `Instant.now()` increased between Java 8 and Java 9.
On Linux and MacOS this doesn't seem to be a problem, as the precision
still seems to be at micro seconds. However, on Windows this now causes
errors of the following form:
```
java.lang.IllegalArgumentException: Conversion of Instant
2021-11-05T13:58:56.726875100Z to microsecond granularity would result
in loss of precision.
```
Suggesting that it now offers sub-microsecond precision.
`TimestampConversion.instantToMicros` had a check to fail if the
conversion lead to a loss of precision. In the specific failing test
case this is not a concern, so this adds a `roundInstantToMicros`
variant that avoids this kind of error.
* TMP round timestamps
* Revert "TMP round timestamps"
This reverts commit af8e261278.
* Skip versions before 1.6.0 in migration tests
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
I’ve kept the infrastructure for versioned_scala_deps around because
I’m optimistic and hope that eventually we’ll do another Scala upgrade.
changelog_begin
changelog_end
* LF: Reintroduce TransactionVersion.asVersionedTransaction
This have been drop in #11626, but canton is using it.
CHANGELOG_BEGIN
CHANGELOG_END
* cosmetic
* cosmetic
Because a bug in the scala compiler deprecated pattern matching are
not detected (See https://github.com/scala/bug/issues/12493).
Hence some deprecated usage of Node aliases have been forgotten in
CHANGELOG_BEGIN
CHANGELOG_END
* interfaces: Preserve by_interface data for create.
Part of #10915
This was a lot more involved than fetch or exercise. The first issue is
that we need to preserve the interface id into speedy, so it needs a
separate primitive ("experimental" won't cut it). Second, because
speedy's create requires the template definition, and now the interface
id as well, we basically need to compile a separate version of "create"
for each interface that a template implements, hence the separate
`CreateByInterfaceDefRef(templateId, ifaceId)`.
changelog_begin
changelog_end
* scalafmt and refactoring
* fixx merge conflict
* fix silly mistakes
* Refactor speedy to distinuish SExpr types before/after ANF compilation phase
CHANGELOG_BEGIN
CHANGELOG_END
* remove commment/marker left in error
* make SExpr0 private to speedy
* reinstate (non-pp) print of original expression in AnfTest faiure
* avoid use of s./t. prefixes for expressions in SBuiltin; add 3 TODO markers
* inline "runtime" apply methods of SDefinitionRef into sole caller: SBCallInterface
* avoid use of t. prefix in SExpr0
* change s./t. prefix to source./target.
* add comment to summarize differences between SExpr0 and SExpr
Following up #10827 and #10921, we drop type parameter to
KeyWithMaintainers, and use the `Versioned` wrapper introduced in
CHANGELOG_BEGIN
CHANGELOG_END
* FoldableContravariant, a mapping for Foldable instances
* use FoldableContravariant to specialize several ImmArraySeq, NonEmpty methods
* folding specializations for ImmArray
* a few docs for FoldableContravariant
* specializations for FrontStack
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* LF: Rename GenTransaction to Transaction
Following up #10827 and #10921 that drop type parameters from GenNode,
we rename GenTransaction to Transaction.
CHANGELOG_BEGIN
CHANGELOG_END
* fix
* a bit more fixes
* Add by_interface field in tx nodes.
This PR adds an optional `by_interface` field in the Create, Fetch,
and Exercise nodes, as part of #10915. It also updates TransactionCoder
to support these fields.
Setting these fields (in the LF interpreter) is left to a separate PR.
changelog_begin
changelog_end
* update security-evidence.md
* Add version checks in TransactionCoder
* Remove virtual choices
* Remove choices without a body in 'interface' definition
* Remove choices in 'template ... implements' section
part of #11372
changelog_begin
changelog_end
* Remove virtual choices cont.
Switch uses of virtual choices to fixed choice with method implementation
* update snapshot after pin on windows
* Disable failing interface tests with TODO #10810
* interfaces: Improve fixed choice lookup
This PR adds a lazy map to match fixed choice names to their
interface, to improve the worst case for choice lookup.
Part of #10810
changelog_begin
changelog_end
* scalafmt
* apply improvements from remy
After #10827 that drops type parameters in Value, `CidContainer`'s
`ensureNoCid` and `assertNoCid` loose their interest.
To check a value does not contain Contract Ids one should use
`foreachCid` or `cids` methods instead.
CHANGELOG_BEGIN
CHANGELOG_END
* Replace `sandbox-classic`-based fixtures with `sandbox`-based ones
changelog_begin
changelog_end
* Replace Scenarios with Script for BasicPrimitiveTypes
* Adapt ListsSpec
* Adapt TextMapsSpec
* Adapt OptionalSpec
* Adapt more tests
* More tests passing
- removed workflow_id testing (Daml Script doesn't set one)
- reduced failure threshold for VeryLargeArchiveSpec
- solved issu in TransactionSingleTableSpec (wrong parties were used)
* Port EnumMod and make EnumSpec pass
* Port GenMapMod and make GenMapSpec pass
* Shaking things around to make Windows happy -- maybe
* Add choice collisions in scala name collision chk
This brings it in line with the LF spec.
This PR also adds collisions for inherited (interface) choices.
Closes#11137.
changelog_begin
changelog_end
* scalafmt
* Apply suggestions from code review
Co-authored-by: Remy <remy.haemmerle@daml.com>
Co-authored-by: Remy <remy.haemmerle@daml.com>
* iface: check for interface implementations precond
We change speedy to also check all preconditions defined in interface
implementations upon a create.
CHANGELOG_BEGIN
CHANGELOG_END
* Update daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/speedy/Compiler.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
* format
* interfaces: lfconversion for iface preconditions
This adds interface preconditions to the interface implementations
during completion phase in the LF conversion.
CHANGELOG_BEGIN
CHANGELOG_END
* format
* rebase
Co-authored-by: Remy <remy.haemmerle@daml.com>
After dropping type parameter from GenTransaction we do not
com.daml.lf.transaction.Transaction.Transaction type alias. We use
instead directly com.daml.lf.transaction.VersionedTransaction, wich is
anyway more informative.
CHANGELOG_BEGIN
CHANGELOG_END
In #11419 we drop the implicit call that convert ContractInstance in
CidContainer. Thie is used by Canton and should be restore.
Note that now ContractInstance is not polymorphic anymore, hence we
can inherite CidContainer instead of defining an implicit.
CHANGELOG_BEGIN
CHANGELOG_END
* Split contract-key tests from engine tests, and mark for evidence.
CHANGELOG_BEGIN
CHANGELOG_END
* remove contract key tests from their original location
* evidence for unique/non-unique contract keys
* interfaces: precondition, scala ast, decoder/encoder
This adds the interface precondition to the scala Daml LF AST, plus
decoder/encoder and the typechecker.
CHANGELOG_BEGIN
CHANGELOG_END
* Update daml-lf/archive/src/main/scala/com/digitalasset/daml/lf/archive/DecodeV1.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
* format
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Make stable packages dependent on supported LF versions
Note that this just introduces an API dependency, we are not yet
filtering the list of packages. For that, I’d really like to
autogenerate StablePackages.scala since I don’t want to make that
manually maintained list even more complex. But review seems easier if
we first change the usage sites and then switch to autogenerating than
trying to do both in one go.
changelog_begin
changelog_end
* Fix script export tests
changelog_begin
changelog_end
* fine grained test evidence for authorization
* fine grained test evidence for privacy
* fine grained test evidence for input-validation (typing)
* fix exit code of security/update.sh script (set -euo pipefail)
* add security evidence test category: Input Validation
* regenerate security-evidence.md
CHANGELOG_BEGIN
CHANGELOG_END
* fix bug in securoty evidence generation (must sort before group, or else we loose lines)
* evidence for input validation of commands
* address comments
* cleanup: remove backticks from evidence free text
We waste 2 minutes on most CI runs with the conformance test for no
good reason. Uploading the package & sending a command should be
sufficient as an integration test.
changelog_begin
changelog_end
* Generate security evidence by documenting security testcases.
CHANGELOG_BEGIN
CHANGELOG_END
* move generated file to root of repo, so links work
* formatError function instead of Show instance
* dont use Show instances for generating Markdown
* magic comment: SECURITY_TEST --> TEST_EVIDENCE
* use megaparsec and Data.Text
* remove redundant T.pack
* use: Text.Megaparsec.Char.space
* Add DA.Internal.NatSyn as a stable package
This module exposes a type NatSyn, to be used for encoding
type synonyms of Nat-kinded types
* Add generalized roundtrip test helpers in LFConversion tests
roundtripTestsBy and roundtripTestsPartialBy are like their
non-By versions, except they take an explicit equality predicate
instead of relying on the Eq instance. This allows the source
and target types to differ.
* Define encoder/decoder for type synonyms
This works by saturating the RHS of the declaration with artificial
variable names and adding the corresponding parameters on the LHS
In the case of Nat type synonyms, the Proxy-like
DA.Internal.NatSyn.NatSyn type is used to wrap the value into
something of kind star.
* Use type synonym encoder/decoder when generating/consuming DALF
This closes#11226
changelog_begin
changelog_end
* Extend type synonyms over data-dependencies test
* Add test cases for multi-param type class synonyms
This adds an experimental `toTypeRep: forall t. t -> TypeRep` builtin.
It will only work on interface payloads and crash horribly otherwise.
CHANGELOG_BEGIN
CHANGELOG_END
* Suport multi-party readAs in triggers
fixes#7640
This does not yet include the trigger service. We’ll tackle that separately.
changelog_begin
- [Daml Triggers] Triggers now support readAs parties. They can be
specified via `--ledger-readas a,b,c`. As part of this change
``testRule`` gained an extra argument to specify the `readAs`
parties. If you previously used
```
testRule trigger party acsBuilder commandsInFlight s
```
you now need to use
```
testRule trigger party [] acsBuilder commandsInFlight s
```
changelog_end
* Update triggers/tests/src/test/scala/com/digitalasset/daml/lf/engine/trigger/test/AbstractFuncTests.scala
Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
* Expand type synonyms lazily
We really only need to expand type synonyms in two cases:
1. When pattern matching e.g. to check that a projection on a struct
really happens on a struct. In this case we can first check if we
have a struct in which case we don’t need to expand anything. If we
don’t, we can expand afterwards.
2. When checking two types for alphaequivalence. This PR goes for the
simplest solution: First check alphaequivalence without expanding
anything treating type synonyms nominally. Then if that fails
expand everything and check again. We could be more clever and
interleave expansion with checking for alphaequivalence but this
seems to give us most of the performance benefits while keeping the
logic very simple.
On my local laptop this speeds up typechecking of the full DAR from
380ms to 176ms so roughly a 2x improvement.
For comparison: The Daml-LF 1.7 DAR which still relies on nonminal
typechecking of typeclasses everywhere takes 110ms. So we are still
leaving something on the table here but we’re much closer.
I think this also gets us to the point where speedy compilation is
slower than typechecking but I need to verify that again.
changelog_begin
changelog_end
* Apply suggestions from code review
Co-authored-by: Remy <remy.haemmerle@daml.com>
* fix biuld
changelog_begin
changelog_end
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Test authorization within Exercise choice.
CHANGELOG_BEGIN
CHANGELOG_END
* fix build following rebase
* 4x shouldBe a Right
* test for exercise within exercise
* implicit Name.assertFromString
* def --> val. massive speedup!
* be more private
* implicit Identifier creation
* use implicit for Party and ChoiceName creation
* comment test purpose
* delete stray text; fix build
* Typecheck nested LF type applications more efficiently
This improves performance typechecking in the Scala LF typechecker by
matching on nested foralls & type applications which allows us to
check them with only a single substitution as opposed to the previous
state of doing one for each type application.
This is about a 25% improvement on typechecking performance of
GHC.Classes from 200ms to 150ms.
changelog_begin
changelog_end
* interfaces: Do some TODOs
- Add uniqueness check between fixed choices and virtual choices in
haskell decoder.
- Encode interface methods and fixed choices in scala encoder.
- ExprIterable for interfaces.
changelog_begin
changelog_end
* scalafmt
* [Self-service error codes] Refactored for Canton integration
* Introduces ErrorCodeLoggingContext which is used to abstract away the logging tech stack
CHANGELOG_BEGIN
CHANGELOG_END
* Moved all error definitions in //ledger/error
* Formatting
* Docs and remove of implicit conversion to DamlErrorCodeLoggingContext
* Fix compilation issues
* Fix tests
* Addressed Pawel's review comments
* Addressed Robert's review comments
* interfaces: introduce TemplateOrInterface class in stdlib
The template typeclass is to strong for many applications. The new
constraint `TemplateOrInterface` only contains the methods to convert
contract IDs and choices.
CHANGELOG_BEGIN
CHANGELOG_END
* ghc-lib update, interface script tests
* pinned stackage on unix
* added missing implementation in preprocessor
* added test
* fixing tests
* remove Iface type
* pinned stackage windows
* make sure createAndExercise is not called on interfaces
* LF: clarify compilation of choice Body.
This is a purely cosmetic choice.
We move up the intro of the choice arg binder to make it clear it
affect the compilation of every expression in the body of a choices (
namely controllers, observers, and update expression).
CHANGELOG_BEGIN
CHANGELOG_END
* formatting
- enable NonUnitStatements, and find a useless statement.
- add missing `final`s,
- replace `{ ... }` by `( ... )` for one statement anonymous function
- drop useless `new` for case class.
- replace `sealed class X { ... object Y extend X` by
`class X private[Z] { ... val Y = new X` to avoid unecessary class
alocation.
- move invariant check inside the body of some case class instead of
the companion object.
- other cosmetic changes.
CHANGELOG_BEGIN
CHANGELOG_END
* interface methods: Scala Typechecker
Part of #11006.
I tried to match the behavior of the Haskell typechecker.
changelog_begin
changelog_end
* scalafmt
* Fix type
* interface methods: Scala AST
Part of #11006
This PR takes care of the Scala AST and decoder.
It leaves the encoder, type checker, and speedy for later.
changelog_begin
changelog_end
* scalafmt
* AstRewriter
* interface methods: Speedy
Part of #11006
changelog_begin
changelog_end
* scalafmt
* Remove InterfaceMethods test
* Update daml-lf/archive/src/main/scala/com/digitalasset/daml/lf/archive/DecodeV1.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
Co-authored-by: Remy <remy.haemmerle@daml.com>
