Commit Graph

574 Commits

Author SHA1 Message Date
Remy
3159d60444
LF: more tests for evaluation order of fetch (#12471)
addresses remaining points of the review of #12411.

CHANGELOG_BEGIN
CHANGELOG_END
2022-01-19 17:45:58 +00:00
Remy
25ddfcdcb4
LF: Add tests for evaluation order on fetches (#12411)
As for creates, these tests somewhat ignore ordering on on-cachable
errors relative to each other since those only change error messages.

changelog_begin
changelog_end
2022-01-18 11:21:30 +01:00
Remy
33bc6fea26
LF: Wrongly typed local contract is not a internal error (#12406)
Since the introduction of COERCE_CONTRACT_ID, local contract IDs can
be wrongly typed.

CHANGELOG_BEGIN
CHANGELOG_END
2022-01-13 19:47:29 +01:00
Remy
ea2d4bc7de
LF: factorize typeAndCompile in Speedy tests (#12402)
CHANGELOG_BEGIN
CHANGELOG_END
2022-01-13 17:23:27 +00:00
Moritz Kiefer
8bc5ff06a7
Clarify unhandled exception error message (#12353)
* Clarify unhandled exception error message

The previous error doesn’t make it clear that this is an exception in
user code rather than a Scala exception in our code.

Daml-lf exception would technically be more correct but I don’t think
it’s helpful here so I went with Daml exception.

changelog_begin
changelog_end

* adjust ledger api test tool

changelog_begin
changelog_end

* Update test assertions

changelog_begin
changelog_end

* fix another assertion

changelog_begin
changelog_end
2022-01-11 17:57:07 +00:00
Remy
30300fdac6
LF: Revert #7742 (#12354)
Following #12338 which merges the two implementations of the
structural record projection introduced by #7740, this PR merges the two
implementations of the structural record introduced by #7742.

However, this PR does not try to cache the field index as it is done

1- unlike what it is suggested by commit message of #7742, the update
   has a linear complexity anyway, as it has to copy the whole struct.

2- the compiler does not produce any structural record updates

As result the PR is basically a revert of #7742.

CHANGELOG_BEGIN
CHANGELOG_END
2022-01-11 15:53:10 +00:00
Remy
7ec55d7f33
LF: Cache field index inside structural record projection builtin (#12338)
Currently we have two implementations for the projection of structural
record.

1- The first implementation takes as parameter the index of the
projected field and is therefore constant. This implementations is
used when the type checking is enable, as the index cannot be directly
inferred from the AST and must hence be filled in by the DAML-LF type
checker.

2- The second implementation takes as parameter the name of the
projected field and is therefore logarithmic as the field must be
lookup by binary search at each call. This version is used when the
type checking is disable as the index cannot be inferred without type
inference.

In this PR, we modify the second implementation so it cache the index
at the first call, hence avoiding the recomputation during further
calls.  In this way we reach an amortized constant complexity.  The
first implementation is decommissioned in benefit of the second one.

The advantages of this approach are:

- We have a unique implementation of the projection, so the behavior
  of a program is the same whenever the type checking is on or off.

- The AST for structural projection is immutable.

Benchmarks show no performance differences when the type checking is on.

Based on an idea by @sofiafaro-da.

CHANGELOG_BEGIN
CHANGELOG_END
2022-01-11 13:03:21 +01:00
Remy
c7c86d2168
LF: Benchmark for structural record projection (#12329)
CHANGELOG_BEGIN
CHANGELOG_END
2022-01-11 10:36:48 +01:00
Gary Verhaegen
d2e2c21684
update copyright headers (#12240)
New year, new copyright, new expected unknown issues with various files
that won't be covered by the script and/or will be but shouldn't change.

I'll do the details on Jan 1, but would appreciate this being
preapproved so I can actually get it merged by then.

CHANGELOG_BEGIN
CHANGELOG_END
2022-01-03 16:36:51 +00:00
Stefano Baghino
52110e31a3
Remove Extractor (#12188)
* Remove Extractor

Extractor is being removed after a long time being in Labs status.

This should improve the flakiness on CI.

🔥

changelog_begin
Extractor has been removed from the SDK
changelog_end

* Remove Extractor documentation

* Remove Extractor from CODEOWNERS

* Remove Extractor references in Daml-LF build files

* Remove Extractor references in the Daml SDK assistant

* Remove Extractor from the SDK

* Remove Extractor reference from CONTRIBUTING.md
2021-12-17 13:43:23 +00:00
Moritz Kiefer
f146bc814b
Escape daml-lf tracelog messages (#12116)
* Escape daml-lf tracelog messages

Currently veracode complains because this allows for clrf
injection (injecting newlines to make user input look like separate
log statements).

With this change
```
debug "abc"
debug "eaiu\neaiu"
debug "def"
debugRaw "abc
```

is logged as

```
[DA.Internal.Prelude:555]: \"abc\"
[DA.Internal.Prelude:555]: \"eaiu\neaiu\"
[DA.Internal.Prelude:555]: \"def\"
[DA.Internal.Prelude:555]: abc
```

You can debate whether we should escape the quotes are necessary but
90% of the reason why people add them is because they call `debug` on
strings when they should be using `debugRaw` so this seems fine to me.

changelog_begin
changelog_end

* fix tests

changelog_begin
changelog_end
2021-12-14 11:20:33 +01:00
Remy
8c87a0e327
LF: Factorize interpretation loop emulation spread in multiple tests. (#12102)
CHANGELOG_BEGIN
CHANGELOG_END
2021-12-10 17:16:27 +01:00
Remy
35beb44c3c
LF: SErrorCrash should not be thrown by Speedy callbacks (#12099)
as it is normally caught in the interpreter loop.  In particular the
caller of the engine should not have to catch such an error.

CHANGELOG_BEGIN
CHANGELOG_END
2021-12-10 12:20:38 +00:00
Sofia Faro
b99952407c
Update TODOs to outdated issues. (#12090)
* Update TODOs to outdated issues.

In particular update a lot of defunct interface TODOs to the LF 1.15
issue (or add the LF 1.15 issue if it's also relevant).

changelog_begin
changelog_end

* typo

* remove numeric/bignumeric todo

* scalafmt!!

* drop choice observer TODO
2021-12-09 21:01:27 +00:00
nickchapman-da
fdf7431ad1
Move to using DB-levels (not DB-indexes) in SExpr0. This change includes both index and level, and performs a runtime check. (#11987)
changelog_begin
changelog_end

add runtime check in freeVars: determination that a variable is-free using levels instead of indexes

remove DB-indexes and runtime check; simplify freeVars computation in closure-conversion
2021-12-09 14:17:15 +00:00
nickchapman-da
39eca49586
compileCommand takes env (#12047)
changelog_begin
changelog_end
2021-12-09 11:38:54 +00:00
Remy
4f1892b0c8
Speedy: Implement EToRequiredInterface and EFromRequiredInterface (#12046)
Part of #11978.

CHANGELOG_BEGIN
CHANGELOG_END
2021-12-08 13:34:39 +00:00
Remy
c53cf758da
LF: Add EToRequiredInterface and EFromRequiredInterface to Scala Ast. (#12042)
Part of #11978. Adds typechecking for those primitives.

CHANGELOG_BEGIN
CHANGELOG_END
2021-12-08 11:09:10 +00:00
Moritz Kiefer
c29f4d333b
Fixed broken merge in //daml-lf (#11985)
changelog_begin
changelog_end
2021-12-06 14:32:37 +00:00
Moritz Kiefer
e01fcdf143
Add exhaustive tests for evaluation order on creates (#11957)
As mentioned in the comment, I view these tests as a way to guard us
against bricking ledgers on upgrades so they somewhat deliberately
ignore ordering of non-cachable errors relative to each other since
those only change error messages.

changelog_begin
changelog_end
2021-12-06 13:06:46 +01:00
Remy
b02ed7765a
LF: Introduce configurable limits on produced transactions (#11948)
This is part of #11691

This PR allows to limits:
- the number of signatories,
- the number of observers,
- the number of controllers,
- the number of inputContracts,

CHANGELOG_BEGIN
CHANGELOG_END
2021-12-02 16:31:09 +01:00
Moritz Kiefer
4571a48c48
Add skeleton for trace-based LF evaluation tests (#11949)
* Add skeleton for trace-based LF evaluation tests

The recent issues around wrong ordering of contract id typechecks got
me thinking about this again so I wanted to hack up a PR of how we can
test evaluation order sensibly directly at the LF level.

This is only testing a single create but I first wanted to see if we
agree that this is a sensible approach using a simple example.

If we agree that this is a sensible approach, I’d suggest to extend
those tests later in separate PRs to make them exhaustive.

changelog_begin
changelog_end

* cleanup

changelog_begin
changelog_end
2021-12-02 11:38:59 +00:00
nickchapman-da
a1db59f40f
Stack safe free-vars computation. (#11946)
* Adding missing stack-safety testcase for SEScopeExcerise.

changelog_begin
changelog_end

* run all testcases at depth of 10,000. This is plenty deep enough to ensure stack-safety

* recode free-vars computation to be stack-safe. and test
2021-12-01 17:06:57 +00:00
Moritz Kiefer
0eb88d6589
Drop outdated fixme comment about struct ordering (#11945)
This has been fixed over a year ago in #7319

changelog_begin
changelog_end
2021-12-01 14:02:46 +00:00
Moritz Kiefer
8179c73763
Drop support for Daml-LF party literals from the Scala side (#11922)
* Drop support for Daml-LF party literals from the Scala side

This PR enforces that forbidPartyLiterals is always `true` and drops
the corresponding literals from the AST. Haskell side is in #11930

fixes #11581

changelog_begin
changelog_end

* Update daml-lf/interpreter/src/test/scala/com/digitalasset/daml/lf/speedy/ComparisonSBuiltinTest.scala

Co-authored-by: Remy <remy.haemmerle@daml.com>

* Revert "Update daml-lf/interpreter/src/test/scala/com/digitalasset/daml/lf/speedy/ComparisonSBuiltinTest.scala"

This reverts commit 55e542ce4e3a7fd15544ee703de3277ffc309b17.

Co-authored-by: Remy <remy.haemmerle@daml.com>
2021-12-01 10:37:42 +00:00
nickchapman-da
16a41f7da2
Avoid package validation in speedy compilation benchmark. (#11927)
changelog_begin
changelog_end
2021-11-30 16:50:42 +00:00
nickchapman-da
0ee4154972
Use Absolute-indexes as keys for the Env-mapping during closure-conversion (#11912)
* Use Absolute-indexes as keys for the Env-mapping during closure-conversion.

Do runtime check to confirm behaviour matches the existing Relative-indexes.

changelog_begin
changelog_end

* remove quadratic shift!

remove (dev)pretty-print code
remove relative-index keys from Env-mapping
remove runtime *diff* check
increase depth for stack-safety tests

* improve/simplify indexing calculation for Env-keys
2021-11-30 16:07:17 +00:00
Sofia Faro
1d7bca801e
Add optional typerep argument in UExerciseInterface. (#11910)
* Add type rep argument for interface exercises.

(Still WIP.)

Part of #11703. Fixes the order in which errors are raised ("wrong type"
takes priority over "does not implement interface"). This PR also simplifies
ExerciseInterface by making the guard mandatory, otherwise there's too
many variations. We can revisit that later if we want.

changelog_begin
changelog_end

* haskell side

* dont throw exception in checkTemplateId

* scalafmt

* evidence security

* fix TypingSpec test
2021-11-30 15:10:33 +00:00
Sofia Faro
0b9d57b95e
Add ContractDoesntImplementInterface error. (#11884)
* Add a ContractDoesntImplementInterface error

Part of #10810 and #11703

changelog_begin
changelog_end

* use inside

* rename error
2021-11-29 15:54:31 +00:00
nickchapman-da
2f4aa4797e
refactor to avoid impossible code path (#11901)
changelog_begin
changelog_end
2021-11-26 15:46:26 +00:00
Remy
58e69ade1a
LF: replace "dev" LF version by "1.dev" in bazel files (#11894)
the more consistent, as asked by Moritz in review of #11820

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-26 12:23:17 +00:00
nickchapman-da
8ef348dab0
Use absolute stack locations in SExpr1 (#11877)
* SExpr1.SELocS - carry relBad/absGood - abs unused so far

* compute/pass SELocS.abs in ClosureConversion, and check in Anf that it matches the reconstructed value

CHANGELOG_BEGIN
CHANGELOG_END

* Remove relative stack locations. Rename as SELocAbsoluteS. Simplify Anf. Remove shiftLoc in ClosureConversion.
2021-11-26 12:10:30 +00:00
Sofia Faro
024400b9e4
Error when fetching the wrong template id (via fetch by interface). (#11862)
* Prevent wrongly typed fetch by interface.

When doing a "fetch by interface" command with a known template id,
error out with a WronglyTypedContract if the fetched contract has
a different template id. This doesn't affect daml, only affects
replays, so it's rather minor. I also enabled the engine test that
caught this.

Part of #11703, follow up to #11836.

changelog_begin
changelog_end

* strengthen test output checks
2021-11-25 16:19:36 +00:00
Sofia Faro
5c12d757f4
Add a guard when exercising by interface. (#11836)
* Add a guard when exercising by interface.

This fixes part of #11703, when exercising an inherited choice by
interface and you know the template id, via the command preprocessor.

It does this by inserting a "guard" in between the interface fetch and
the exercise body. The guard is a function Interface -> Bool, which
is general enough to check the template id, without complicating too
much in speedy. And can be generalized in the future to check more,
like signatories, etc.

I added the guard as an optional argument to UExerciseByInterface.
This isn't hooked up to the protobuf AST yet (or Haskell side for
that matter) -- but I'll do it in the next PR! For now you can invoke
the guarded exercise via the command preprocessor, so I can enable the
approprate engine tests. (There's still some failing fetch tests left,
but I decided to leave this for later. Fetch can be a lot simpler than
guarded choices, since you always add a fetch node. No need for fancy
continuations.)

changelog_begin
changelog_end

* scalafmt

* Feedback and fix matches

* Update comments, we are always going to abort the transaction

* Raise WronglyTypedContract in SBGuardTemplateId.

* rebase and fix parser

* restore ANF

* scalafmt
2021-11-24 14:22:40 +00:00
nickchapman-da
0374843143
speedy compilation benchmark (#11852)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-24 13:10:18 +00:00
nickchapman-da
970243dd46
Ensure stack-safety during closure-conversion. (#11778)
ClosureConversion -> Suffix with "Old"

CHANGELOG_BEGIN
CHANGELOG_END

ClosureConversion old-vs-new diff check

ClosureConversionNew, first cut. All tests in SBuiltinTest work.

In addition we change some Array --> List in SExpr1 (for human pp).
And we throw away ClosureConversionDup.

adapt AnfTest from Array to List change for SExpr1

all tests pass in daml-lf/interpreter

remove SExpr0.SELet1General

reorder things

testing for stack-safety of closure conversion

file/class renames

improve naming

pass cont as sep arg to commit (move out of Up/Down)

comment stack-safe closure conversion

fix bug: failed to use env1

fix 2x unmoored doc comment

comment stack safety testing

Remove old closure-conversion code & diff-check between old/new.

loose StackSafe suffix on ClosureConversion class/file

rename StackSafetyTest.scala to ClosureConversionTest.scala

prefer "sealed abstract class" to "sealed trait"

fvs.zipWithIndex --> fvs.view.zipWithIndex

(SExpr1) SEAppGeneral -> SEApp; prefer List to Array in SEApp/SECase

prefer xs.toArray to Array(xs: _*)

access SExpr0 via "source."

two more .view

improve comment and fix typo

link to Issue

switch to a continuation stack; avoids nesting in the Cont type
2021-11-24 11:47:39 +00:00
Remy
f33486154f
LF: Simplify LF Syntax (#11795)
- Do not wrapped choices in `choices { ... }`
- use sytematic `;` after choice component instead of ',`

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-19 17:23:36 +01:00
Remy
cbe9c922ba
LF: Clean up Ast (#11786)
- rename non-default builders `apply` to `build`
  * avoid confusing both
  * make explicit the build can crash

- make interfaceId and templateId fields consitent

- use when possible named arguments

- check for non-repetition of inherited choices

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-19 15:40:28 +01:00
Remy
1bb2fc28a3
LF: Simplify transaction versionning for interface (#11744)
We revert #11626, and just change the way transaction version is
computed:

- As before, Node version is calculated from the
  package of the template ID action.

- Transaction version is the max of the version of all the nodes,
   instead of the root nodes.

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-18 22:46:46 +01:00
Moritz Kiefer
3b5f8a73e1
Remove precondition field from TemplateImplements (#11763)
* Remove precondition field from TemplateImplements

fixes #11635

changelog_begin
changelog_end

* Update daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/speedy/Compiler.scala

Co-authored-by: Remy <remy.haemmerle@daml.com>

Co-authored-by: Remy <remy.haemmerle@daml.com>
2021-11-18 17:10:40 +01:00
Moritz Kiefer
fa7663148a
Drop 2.12 versioned_scala_deps (#11748)
I’ve kept the infrastructure for versioned_scala_deps around because
I’m optimistic and hope that eventually we’ll do another Scala upgrade.

changelog_begin
changelog_end
2021-11-17 22:13:08 +00:00
Sofia Faro
c2d4ea4ef1
Add separate commands for create/fetch/exercise by template/interface. (#11724)
* Add separate commands wrt by template/interface

Resolves #11674 and #11675

changelog_begin
changelog_end

* remove unnecessary commands
2021-11-17 17:08:24 +00:00
nickchapman-da
a1fc5c6a25
Simplify handling for EqualList builtin. (#11676)
Step 1: SEBuiltinRecursiveDefinition(_) -> SEBuiltinEqualList

CHANGELOG_BEGIN
CHANGELOG_END

Step2: Replace SEBuiltinEqualList with SBuiltin.SBEqualList

update comment
2021-11-15 17:49:07 +00:00
Remy
15b925a116
LF: Compute transaction version as the maximum of the children (#11626)
fixes #11377

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-12 17:00:07 +01:00
nickchapman-da
8c46559163
Remove unnecessary constructors: SEDamlException, SEImportValue, from SExpr{0,1} (#11668)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-12 12:40:52 +00:00
nickchapman-da
3192d5eb74
Different types before after closure conversion (#11661)
* avoid single letter (s./t.) import prefixes

* Split type SExpr0 -> SExpr{0,1} for before/after closure conversion

CHANGELOG_BEGIN
CHANGELOG_END

* remove unnecessary constructors in SExpr{0,1}

* remove SExpr0.SExprAtomic
2021-11-12 08:54:43 +00:00
nickchapman-da
7296ba4dfb
Move closureConvert and validate compiler phases into separate files. (#11656)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-11 13:38:05 +00:00
Sofia Faro
87f282c7f3
interfaces: Preserve/provide by_interface data for create actions. (#11639)
* interfaces: Preserve by_interface data for create.

Part of #10915

This was a lot more involved than fetch or exercise. The first issue is
that we need to preserve the interface id into speedy, so it needs a
separate primitive ("experimental" won't cut it). Second, because
speedy's create requires the template definition, and now the interface
id as well, we basically need to compile a separate version of "create"
for each interface that a template implements, hence the separate
`CreateByInterfaceDefRef(templateId, ifaceId)`.

changelog_begin
changelog_end

* scalafmt and refactoring

* fixx merge conflict

* fix silly mistakes
2021-11-11 12:57:55 +00:00
nickchapman-da
2dd5289a35
different types for different speedy phases (#11605)
* Refactor speedy to distinuish SExpr types before/after ANF compilation phase

CHANGELOG_BEGIN
CHANGELOG_END

* remove commment/marker left in error

* make SExpr0 private to speedy

* reinstate (non-pp) print of original expression in AnfTest faiure

* avoid use of s./t. prefixes for expressions in SBuiltin; add 3 TODO markers

* inline "runtime" apply methods of SDefinitionRef into sole caller: SBCallInterface

* avoid use of t. prefix in SExpr0

* change s./t. prefix to source./target.

* add comment to summarize differences between SExpr0 and SExpr
2021-11-11 08:21:12 +00:00
Remy
b8f384e193
LF: Drop typeparameter from KeyWithMaintainers (#11625)
Following up #10827 and #10921, we drop type parameter to
KeyWithMaintainers, and use the `Versioned` wrapper introduced in

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-10 13:21:13 +01:00