Commit Graph

2038 Commits

Author SHA1 Message Date
Remy
96f16df3c0
remove script/trigger/cantonFixture deps on scala-bindings (#17954)
Basically we remove the dependency of different components on "//language-support/scala/bindings" by:

- replacing com.daml.ledger.api.refinements.ApiTypes.Party by com.daml.lf.data.Ref.Party
- replacing com.daml.ledger.api.refinements.ApiTypes.ApplicationId by Option[com.daml.lf.data.Ref.ApplicationId] (here we use option as ApiTypes.ApplicationId allows empty string while Ref.ApplicationId does not).
- adding rounding logic for timestamp in com.daml.lf.data.Time.Timestamp and use it instead of the one from com.daml.api.util.TimestampConversion

Note we did not clean daml-sript export as it have never pass the alpha stage and will be dropped with the 3.x fork.
2023-12-05 15:08:09 +00:00
Paul Brauner
e22f8bebd9
Run costly tests after only after merging (#17956)
* do not run pr-only tests on main, do not run main-only tests on prs

* split data dep tests into main-only and pr-only

* run non-dev conformance tests on main only
2023-12-04 10:52:33 +01:00
Moisés Ackerman
c1f7a87572
Add feature flag for scenarios (off in LFv2) (#17927)
* Add feature flag for scenarios (off in LFv2)

* whitespace

* Use rule for old-proj-2.1.dar
2023-11-29 17:01:44 +00:00
Paul Brauner
9dbb53e97e
introduce lf_version_default_or_latest in bazel and use it everywhere were I had replaced the implicit default version with 'latest' (#17930) 2023-11-29 15:53:53 +01:00
Paul Brauner
c85eb0e59b
Test LF v2 in ShakeIdeClient.hs (#17922)
* Test LF v2 in ShakeIdeClient.hs

* Handle daml3-script conversion errors in scenario.Context
2023-11-28 11:36:30 +01:00
Paul Brauner
562b303a9b
clean up some comments (#17925) 2023-11-27 15:12:54 +00:00
Samuel Williams
aa2ea1b285
Fix mangled name test to run quick (#17919)
Improve failing test assertions
2023-11-27 14:22:16 +00:00
Paul Brauner
646a74e25a
Also test LF v2 in //compiler/damlc/tests:script-service (#17904)
* Also test LF v2 in //compiler/damlc/tests:script-service

* move setEnv TASTY_NUM_THREADS outside of defaultMain

* Track end of cleanup in withResourceCps
2023-11-24 10:45:06 +01:00
dylant-da
7070966b5c
Enable multi-package by default (#17901)
* Enable --enable-multi-package by default

* Make flags for multi-package visible in help
2023-11-23 17:06:45 +00:00
Paul Brauner
2b28e59017
Introduce LF 2.1 (#17865) 2023-11-23 08:59:14 +01:00
Paul Brauner
fa8d1dc1be
Refactor LF decoding in the compiler (#17864)
* Refactor decoding in compiler

* remove obsolete comment

* factorize -decodeLf2Version and decodeLf1Version

* call the right decodeScenarioModule depending on the target LF version, update incremental build tests

* split encoding into two modules, call the right encodeScenarioModule in writeDalfFile

* delegate to EncodeV2.hs in Encode.hs

* remove noisy type signature, was there initially to guide me but I think it adds more confusions than it clears up things
2023-11-21 15:41:09 +00:00
Samuel Williams
d7b63f9c42
Multi-package CLI changes (#17850)
* Allow cyclic multi-package
Fix test --output error

* Backup damlc arg parser with build options when daml.yaml args used

* Apply review suggestions

* Add tests for changed cli behaviour
2023-11-20 14:25:39 +00:00
Remy
ff7b5c7d51
[Daml-Script] minimal support for explicit disclosure (#17818)
Co-authored-by: Samuel Williams <samuel.williams@digitalasset.com>
2023-11-17 11:30:08 +01:00
Moisés Ackerman
9e9ffea143
Remove unvalidated damlc integration tests (#17832)
* Remove tests '//compiler/damlc/tests:integration-unvalidated-v{115,1dev,2dev}'

* Use mangle_for_damlc for consistency

* remove outdated comments
2023-11-16 12:01:24 +01:00
Samuel Williams
6d05c355ce
Warn on import of internal daml-script module (#17816)
* Warn on import of internal daml-script module

* Fix tests

* Address comment
2023-11-15 14:04:46 +00:00
mziolekda
b81ee69abe
disentangle ledger api (#17831)
* make testing-utils independent of ledger-api

* remove dead files

* move non ledger-api stuff to libs-scala
2023-11-14 22:44:10 +01:00
Paul Brauner
890dee12bf
Tie right-to-left evaluation order to LF v2 (#17811)
* Tie right-to-left evaluation order to LF v2

* fix typo
2023-11-14 10:34:31 +01:00
Marton Nagy
6933514df5
Switch from Akka to Pekko (#17814)
* Automated renames by bash script

This commit is exclusively contains changes by the bash script.
For the bash script is present at the pull request.

* Manual pekko migration changes

* adapt fully qualified name references
* adapt pekko package declarations
* adapt bazel files with dependency changes
* adapt canton pekko lib shade_rule
* adapt logger configuration declarations
* pin maven dependencies
* revert incorrect changes by script to compatibility module

Workarounds for further TODOs:
* disable http-json-perf and libs-scala/gatling-utils modules to maintain clean pekko dependencies (without akka)
* disable GraphQLSchemaSpec test (sangria library needs to be upgraded)

* Formatting
2023-11-13 23:22:22 +00:00
Moisés Ackerman
e67fd306d3
Bump stackage snapshot to lts-19.33 (#17795)
* clean up stack-snapshot.yaml

* bump stack to snapshot lts-19.33 pt. 1

* bump stack to snapshot lts-19.33 pt. 2

* repin windows snapshot

* Use a copy of "@stackage-exe/hpp" in runfiles to avoid hardcoding version in haskell code

* Crash with identifiable error if hpp resource/runfile is missing but CPP is used
2023-11-13 19:42:32 +01:00
Samuel Williams
9b70a112b1
Change default multi-package behaviour to search (#17810)
* Change default multi-package behaviour to search

* Ensure new build log is only given when multi-package enabled
2023-11-13 13:32:27 +00:00
Marton Nagy
78cc750714
Scala upgrade to 2.13.11 (#17803) 2023-11-13 09:05:39 +01:00
Paul Brauner
7d1521eeca
Port daml repl to LF2 (#17750)
* add an lf-major-version flag to the repl service and pass it from the haskell client

* test LF2 in the daml-repl tests

* formatting

* formatting

* remove obsolete TODO

* fix //compiler/damlc/tests:repl

* port repl-functests to LF v2

* do not assume that LFV2 => dev

* address Moises' comments
2023-11-10 18:17:48 +00:00
Moisés Ackerman
d52a42ff52
Handle fetch node actors correctly in scenario-service (#17734)
* Add failing tests

"'bank' fetches" should be "'alice' fetches" in the ledger output when 'alice' is the divulging party

* Handle fetch node actors correctly in scenario-service

* Handle fetch node actors correctly in lf-interpreter pretty printer
2023-11-10 14:17:23 +00:00
Samuel Williams
efbe746554
Fix daml3-script 1.dev error (#17801) 2023-11-10 14:17:09 +00:00
Remy
6ef062dac9
Revert "Nmarton/pekko rebased (#17789)" (#17796)
This reverts commit d310692d1b.

@remyhaemmerle-da messed up and merged the PR too fast.
2023-11-09 21:23:18 +00:00
Marton Nagy
d310692d1b
Nmarton/pekko rebased (#17789)
* Scala upgrade to 2.13.11
* Switch from akka to pekko
* Also disable gatling based libs to start from a clean stage
2023-11-09 18:17:33 +01:00
Moisés Ackerman
495c0a0192
Fix daml3-script query question timeouts (#17792)
* Add failing test

* Add failing daml3-script+grpc test

* ScriptF:parse expects 'QueryACS' instead of 'Query' for 'query' question
2023-11-09 17:12:09 +01:00
Samuel Williams
d1f46a3e23
Implement tryCommands for internal testing (#17715) 2023-11-09 12:35:41 +00:00
Paul Brauner
32c662671e
bump proto3 suite (#17780)
* rename daml_lf_{n}.proto to daml_lf{n}.proto

* bump proto3-suite to the latest version

* pin stackage packages on windows
2023-11-08 20:12:03 +01:00
Paul Brauner
313b789731
Bump netty and grpc-netty (#17770)
* pin dependencies to json and add missing dep

* fix cyclic dep

* remove unused dep

* add missing dep to //ledger-api/testing-utils:testing-utils

* remove unused dep in //ledger/ledger-api-auth:ledger-api-auth

* remove more unused deps

* more dep fixes

* yet more dep fixing

* more fixing..

* more of the same

* hopefully the last deps to fix

* Bump the version of protobuf and fix everything that depends on it. Took shortcuts that I need to fix in a next commit, but would like to run the CI on this now that it compiles

* don't error out in the grpc-haskell patch

* remove obsolete patch

* patch absl to compile on mingw

* Add a patch to recognize the compiler

* Define _DNS_SD_LIBDISPATCH for macOS gRPC

* bump netty_tcnative_version according to https://github.com/grpc/grpc-java/blob/master/SECURITY.md#netty

* pin maven deps

* Fix macos linking errors 'dyld[xxx]: missing symbol called'

* Skip Darwin frameworks in package-app.sh

* pin stackage packages

* pin stackage windows deps

* use the netty version agreed on

* bump the windows global cache to try and debug the upb issue

* restart the CI after timeout

* clean up

* disable failing tests for now

* comment out unused code

* reset the windows machine name to 'default'

---------

Co-authored-by: Moisés Ackerman <6054733+akrmn@users.noreply.github.com>
2023-11-07 19:35:50 +01:00
Remy
a83a84b326
Add support for explicit disclosure in daml script 3 (#17723) 2023-11-07 15:46:20 +00:00
dylant-da
bb5ae76cd9
Warn instead of errors for expression changes in templates (#17609)
* Add warnings to Gamma

* Warn instead of error for changed template expressions

* lint

* Errors and warnings for key types

* add context to warnings where relevant

* Add warning logic to DamlcUpgrades, fix errors that are now warnings

* Add tests for key errors and warnings (signatories, expression, type)

* Refactor expression extraction, fix extract key expression/maintainers

* satisfy lint, trigger test restart

* Test error for dropped key and warning for added key

* Warn for extraction issue instead of erroring out

* Change `cannot change` error-wording with `has changed` warning-wording

* comments for extractors
2023-11-07 12:32:04 +00:00
Simon Maxen
ae5e9feda9
Allow for the construction of shared GlobalKeys.
* Revert "Add SharedGlobalKey"

This reverts commit 6592cb7bd5.

* Adding shared flag to GlobalKey

* Remove shared flag from GlobalKey

* Support old and new grcp error parsing

* Add tests for hashContractKey
2023-11-06 20:00:38 +00:00
Samuel Williams
e7edd942dc
Align execClean errors with execBuild (#17722) 2023-11-01 10:19:33 +00:00
Samuel Williams
79d7203f37
Implement daml clean --all (#17697) 2023-11-01 09:10:40 +00:00
Paul Brauner
1b8a676392
Revert "Cleanup after LF V2 backwards compatibility break (#17558)" (#17696)
This reverts commit 61ba0fa6fa.
2023-10-31 14:37:40 +01:00
Remy
8cfe4c01a3
Check qualified name when upgrading a contract (#17662) 2023-10-31 09:53:17 +01:00
Paul Brauner
db91552b4a
replace community canton snapshot with our bazel-built canton deploy jar (#17651)
* replace community canton snapshot with our bazel-built canton deploy jar

* add VERSION to the resources of the jar

* fix canton-test-runner-with-dependencies-script

* add resources to community_common

* Update test-common/canton/it-lib/src/main/com/daml/CantonFixture.scala

Co-authored-by: Remy <remy.haemmerle@daml.com>

---------

Co-authored-by: Remy <remy.haemmerle@daml.com>
2023-10-30 10:23:15 +00:00
Remy
422b6934d6
[Speedy] improve upgrade failure error message (#17634) 2023-10-25 15:16:11 +00:00
Moisés Ackerman
d5ec537cee
Daml.Script: Add trySubmit{,Multi}Concurrently (#17608)
* Update docs for SubmitError:TemplatePreconditionViolated

* Daml.Script: add 'trySubmit{,Multi}Concurrently'

* Make trySubmitConcurrently fully sequenced when using IdeLedger

* Add test cases for trySubmitConcurrently

* Test trySubmitConcurrently in Daml3ScriptTestRunnerDev

* Handle LOCAL_VERDICT_LOCKED_{CONTRACTS,KEYS} in gRPCi Ledger Client

* Move 'trySubmit{Multi}Concurrently' to 'Daml.Script.Questions.Submit.Internal'

* Apply review feedback

* Test 'trySubmitConcurrently' using AbstractScriptTest instead of DamlScriptTestRunner

* Apply review feedback: trySubmitConcurrently returns List instead of Seq
2023-10-25 12:12:43 +00:00
Simon Maxen
0b637ff695
Make package optional in scenario service (#17619) 2023-10-23 13:42:36 +01:00
dylant-da
9011c13579
Initial typechecking for upgrades (#17220)
* first example

* remove reliance on NameMap in del/existing/new logic

* Add testing for upgrades typechecking in `daml damlc build`

* lint

* add typecheck-upgrades flag

* rename Foo to MyLib

* Add missing field to PackageConfigFields

* Add reasonable context location for upgrade errors

Staging last changes before I leave for CX for some time.

* fix quoting for windows paths

* Check template choices specifically

* Refactor to have signatory etc. checks, improve choice/template check

* Check return types of choices

* Update DamlcUpgrades for new versions

* Check for deleted top level datatypes, add TODO for type check

Also add type signatures to some helpers

* Add tests for templates as input and outputs of choices

* lint

* Move upgrading error variants into own UpgradeError type

* Improve comment for choice input type invariant

* Replace TemplateChoiceOutput with shouldCheck
2023-10-19 17:37:11 +00:00
Moisés Ackerman
61ba0fa6fa
Cleanup after LF V2 backwards compatibility break (#17558)
* Set 'optDetailLevel = PrettyLevel (-1)' in integration tests

This means we don't need to postprocess the damlc output to remove package ids

* Move stable type constructors to DA.Daml.StablePackages

* Don't use 'rewriteStableQualified' for Tuples in LFConversion

* Don't use 'rewriteStableQualified' for DA.Internal.Erased.Erased in LFConversion

* Don't use 'rewriteStableQualified' for DA.Internal.PromotedText.PromotedText in LFConversion

* update StablePackages comment

* Future-proof renderFeatureVersionReq
2023-10-16 15:47:31 +02:00
Samuel Williams
5eb2696deb
Implement multi-package.yaml CLI (#17267)
* First working model (requires ghcide changes)

* Bug fixes, add logging

* Use pinned ghcide

* Comment up a storm

* Add multi sdk version support

* Implement project.yaml approach

* Cleanup

* Add projects to multi-package.yaml

* Add flags for build all projects in multi-project, and no caching

* Delete ghcide

* Start migration from daml multi-build to daml build

* Add non --all daml build flow

* Finish migration over to daml build

* Add comments

* Change --all and --no-cache to switches

* Fix caching issue with source set to .

* Address initial comments

* Add daml.yaml source as a file caching warning

* Forward sub-assistant call errors to multi-build

* Update comment

* Update merged ghcide
2023-10-11 13:50:55 +01:00
Paul Brauner
f4ee00361e
Make LF v2 non backwards-compatible with LF v1 in the compiler (#17492)
* initial commit

* split gen-stable-packages-v_i into two targets

* rename compatibleWith to canDependOn

* update damlc integration tests annotation to allow for 2.x

* use the right package ID for preconditionFailed when compiling to V2

* fuse stable-packages-v1 and stable-packages-v2 into one single filegroup

* Fix DA.Daml.LF.Ast.Tests

* remove leftover 'undefined' in Version.hs

* progress on fixing DataDependencies.hs

* fix Cross-SDK typeclasses test for 2.dev

* Fix the 'Typeclasses and instances' data dependency test

* Update comment

* fix //compiler/damlc/tests:packaging

* Add TODO

* parameterize the machine by the langage version, hardcode v1 in script v2, v2 in script v2, v1 in exports

* get EngineTests to pass

* fix more tests

* fix canton integration tests

* formatting

* fix more tests

* fix transactionversiontest

* fix exceptiontest

* Fix ValueEnricherSpec

* Fix EngineInfoTest

* fix PartialTransactionSpec

* fix upgragetest

* fix TransactionSnapshot

* Fix ContractKeySpec

* Fix ReinterpretTest

* fix InterfaceViewSpec

* fix InterfacesTest

* fix stable package v1 names

* fix validate.sh tests

* formatting

* Fix ChoiceAuthorityTest

* fix explicit disclosure test

* Fix SpeedyTest

* formatting

* Fix integration test

* fix data dependency tests

* fix package vetting count, increased due to metadata being added

* Redact stable package IDs in error messages in order for the ExceptionSemantics test to work for both v1 and v2

* cleanup

* fix Daml2ScriptTestRunner

* fix JsonApiIT and daml-script golden tests

* fix daml3-script runner test

* enable v2 for all integration tests

* formatting

* fix NodeSeedsTest

* fix since-lf annotations

* add comments, improve consistency

* stop hardcoding V1 in runPureExpr and runPureSExpr

* formatting

* remove harcoding of LFv1 in ConcurrentCompiledPackages.apply

* Parameterize Compiler.Config.Default with major language version

* remove global parser implicit and default package ID and language version
2023-10-11 11:40:17 +02:00
Paul Brauner
158cdd90d7
Fix the external name of daml3-script.dar to mention 2.dev (#17512)
* Fix the external name of daml3-script.dar to mention 2.dev

* make the compilation and distribution of daml3-script dars more future proof

* fix path in ci/copy-unix-release-artifacts.sh
2023-10-03 09:16:00 +00:00
Remy
623f305e9c
Move canton directory to let place for code drop (#17506) 2023-09-29 14:37:01 +00:00
Moisés Ackerman
e6bf01717f
Remove more internal uses of Scenario (#17375)
* daml-lf-test.sh no longer takes damlc argument

* Split //daml-lf/tests:BasicTests into engine and integration tests

* Remove unused //daml-lf/engine:Optional

* Convert //daml-lf/tests:AuthorizedDivulgence into integration test

* Convert //daml-lf/tests:DontDiscloseNonConsumingExercisesToObservers into integration test

* Convert //daml-lf/tests:ConjunctionChoices into integration test

* Convert //daml-lf/tests:ContractKeys into integration test LFContractKeys

* Move //daml-lf/tests:AuthTests to //daml-lf/engine:AuthTests

* Split //daml-lf/tests:LargeTransaction into ledger and non-ledger tests

* Remove scenarios in //language-support/java/codegen:ledger-tests-model

* Remove unused scenarios in //test-common:src/main/daml/model/Test.daml

* Remove 'enable_scenarios' param in da_scala_dar_resources_library

* Remove 'enable_scenarios' param in damlc_compile_test

* Remove '--enable-scenarios=yes in //compiler/damlc/tests:deterministic

* Convert /daml-lf/tests/scenario/dev/experimental to integration test

* Convert /daml-lf/tests/scenario/dev/interfaces to integration test

* Convert /daml-lf/tests/scenario/stable/big-numeric to integration test

* Convert /daml-lf/tests/scenario/stable/contract-key-through-exercises to integration test

* Convert /daml-lf/tests/scenario/stable/contract-keys to integration test

* Convert /daml-lf/tests/scenario/stable/divulge-iou to integration test

* Convert /daml-lf/tests/scenario/stable/embed-abort to integration test

* Convert /daml-lf/tests/scenario/stable/eval-agreement to integration test

* Convert /daml-lf/tests/scenario/stable/exception-auth to integration test

* Convert /daml-lf/tests/scenario/stable/gen-map to integration test

* Convert /daml-lf/tests/scenario/stable/many-fields to integration test

* Convert /daml-lf/tests/scenario/stable/mustfailcommit to integration test

* Convert /daml-lf/tests/scenario/stable/mustfailinterpretation to integration test

* Convert /daml-lf/tests/scenario/stable/mustfails to integration test

* Convert /daml-lf/tests/scenario/stable/no-contract-ids-in-keys to integration test

* Convert /daml-lf/tests/scenario/stable/pass-time to integration test

* Convert /daml-lf/tests/scenario/stable/pattern-matching to integration test

* Convert /daml-lf/tests/scenario/stable/timeout to integration test

* Remove scaffolding for daml-lf/tests/scenario

* Remove unused 'enable-scenarios' flags

* Remove unused daml-lf/tests/daml-lf-test.sh

* Remove unused daml-lf/tests/scala-test-limited-stack.sh

* Remove comments about deprecated flag 'DontDivulgeContractIdsInCreateArguments' in AuthorizedDivulgence integration test

* Add 'create' in LfStableTimeout:testScriptLoop

* add TODO for nesting limits test in LfStableMustFails

* Move LfStableMustFails expected ledger files into subdir
2023-09-28 16:29:46 +01:00
Samuel Williams
c0a18eeb7e
Fix daml3-script warning (#17481) 2023-09-26 17:07:52 +01:00
Moisés Ackerman
84ca088b2c
Check script output in damlc integration tests (#17380)
* Refactor damlc integration test input into datatype

* Factor out 'withLog' helper function

* Reduce scope of 'service' in 'testCase'

* Use 'testPassed' helper for ignored test files

* Reduce scope of 'TestCase' and 'log' in 'testCase'

* Use guards instead of nested if-then-else in 'testCase'

* Extract 'testSetup' from 'testCase' using 'withResource'

* {testCase=>damlFileTestTree} now produces a test tree

* Split jq queries into individual tests

* whitespace

* Add ledger expectation tests

* Avoid double inversion-of-control
2023-09-20 18:07:28 +00:00