Commit Graph

384 Commits

Author SHA1 Message Date
Meriam Lachkar
de9ce628d3
fix build_canton_3x_with_bazel (#18081)
* fix build_canton_3x_with_bazel

* Code pull

* fixing issues with admin-api new project

* fix the way we were passing java_conversions to scalapb plugin

* Enable `proto_gen` to correctly generate Java conversions when using the `scalapb` plugin (#18087)

* Enable `proto_gen` to correctly generate Java conversions when using the `scalapb` plugin

* Remove reference to removed parameter

---------

Co-authored-by: Rafael Guglielmetti <rafael.guglielmetti@digitalasset.com>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2024-01-05 12:10:14 +01:00
Moisés Ackerman
4794454a70
Platform independence pre-check (#17955)
* Rename //bazel_tools:match-{cabal=>golden}-file

* Generate and check hash file for contents of //compiler/damlc/tests:platform-independence.dar

* Add pre-commit hook to test //compiler/damlc/tests:platform-independence-dar-hash-file-matches

* Print info when //bazel_tools:match-golden-file fails

* platform-independence-dar-check pre-commit hook runs depending on env var

* Skip platform-independence-check if RUN_PLATFORM_INDEPENDENCE_CHECK is unset

* Document dar_hash_file rules

* move platform-independence-checks warning to .envrc

* implement generate_dar_hash_file genrule in python

in particular, this allows us to produce the same output on all platforms

* fix windows

* Update rules_daml/daml.bzl

* bytes.replace does not operate in place
2023-12-07 18:18:24 +01:00
dylant-da
40b5eb8476
Fix split release for deps (#17968)
* Resolve release version for sdk build checks

* lint

* lint

run-full-compat: true

* Simple test for using daml-script in release versions

* Fix build issues in tests using pSdkVersion

run-full-compat: true

* Fix build issues with DamlcIntegration

* fix bad sdk version being an invalid version

run-full-compat: true

* Fix the linux "mmap 4096 bytes at (nil): Cannot allocate memory" error

* Fix compat tests on Windows

run-full-compat: true

* test windows os correctly

run-full-compat: true

* temporarily disable canton_3x

run-full-compat: true

---------

Co-authored-by: Paul Brauner <paul.brauner@digitalasset.com>
2023-12-05 23:46:27 +01:00
Marton Nagy
78cc750714
Scala upgrade to 2.13.11 (#17803) 2023-11-13 09:05:39 +01:00
Remy
6ef062dac9
Revert "Nmarton/pekko rebased (#17789)" (#17796)
This reverts commit d310692d1b.

@remyhaemmerle-da messed up and merged the PR too fast.
2023-11-09 21:23:18 +00:00
Marton Nagy
d310692d1b
Nmarton/pekko rebased (#17789)
* Scala upgrade to 2.13.11
* Switch from akka to pekko
* Also disable gatling based libs to start from a clean stage
2023-11-09 18:17:33 +01:00
Paul Brauner
060932499b
Bump gRPC-haskell and delete obsolete patches (#17790) 2023-11-09 14:27:44 +01:00
Paul Brauner
313b789731
Bump netty and grpc-netty (#17770)
* pin dependencies to json and add missing dep

* fix cyclic dep

* remove unused dep

* add missing dep to //ledger-api/testing-utils:testing-utils

* remove unused dep in //ledger/ledger-api-auth:ledger-api-auth

* remove more unused deps

* more dep fixes

* yet more dep fixing

* more fixing..

* more of the same

* hopefully the last deps to fix

* Bump the version of protobuf and fix everything that depends on it. Took shortcuts that I need to fix in a next commit, but would like to run the CI on this now that it compiles

* don't error out in the grpc-haskell patch

* remove obsolete patch

* patch absl to compile on mingw

* Add a patch to recognize the compiler

* Define _DNS_SD_LIBDISPATCH for macOS gRPC

* bump netty_tcnative_version according to https://github.com/grpc/grpc-java/blob/master/SECURITY.md#netty

* pin maven deps

* Fix macos linking errors 'dyld[xxx]: missing symbol called'

* Skip Darwin frameworks in package-app.sh

* pin stackage packages

* pin stackage windows deps

* use the netty version agreed on

* bump the windows global cache to try and debug the upb issue

* restart the CI after timeout

* clean up

* disable failing tests for now

* comment out unused code

* reset the windows machine name to 'default'

---------

Co-authored-by: Moisés Ackerman <6054733+akrmn@users.noreply.github.com>
2023-11-07 19:35:50 +01:00
Remy
6af384a840
make changing default scalacopts a bit nicer (#17751) 2023-11-03 15:16:00 +00:00
Samuel Williams
637ff67ea0
Upgrades testing via daml-script (#17610)
* Implement vetDar and unvetDar
Blocked by canton not returning uploaded dars

* Upgrades testing infra/IT

* Fix HttpServiceTestFixture

* Fix some tests

* Fix cantonRunner for windows

* Add delay after vet actions so topology transactions can land

* Implement invalid data upgrades tests

* Add temporary internal setContractUpgradingEnabled flag to daml-script

* Switch to setProvidePackageId

* Write choice and multi-participant tests

* Formatting

* Remove unused import

* Address reviews

* Update errors

* Fix canton runner for windows

* Address review comments

* Add new proto to artifacts

* avoid pushing canton admin proto as maven artifact (#17742)

---------

Co-authored-by: Remy <remy.haemmerle@daml.com>
2023-11-03 14:10:15 +00:00
Moisés Ackerman
3512b5eb4d
Embed bash and zsh completion scripts into daml-assistant (#17675)
* Update optparse-applicative dep to 0.17.1.0

* Update entropy dep to 0.4.1.10

* Embed bash and zsh completion scripts into daml-assistant
2023-10-27 17:23:20 +02:00
Raphael Speyer
a4e898c252
LT-35: create JsonLfEncoders in java bindings (#17603)
No code-gen yet.

Also pulled out the common test fixtures to share between encode and decode tests.
2023-10-23 16:33:12 +11:00
Moisés Ackerman
1bee52e40c
Remove compiler support for controller..can choice syntax (#17362)
* Update GHC_REV

* Remove 'controller..can' syntax in java codegen StakeholdersTest

* Remove 'controller..can' syntax in ChoiceSyntaxes daml-test-file

* Remove daml-test-files {,No}WarnControllerCan

* Remove 'controller..can' syntax in SemanticsEvalOrder{,RightToLeft} daml-test-files

* Update {Structure,Reference}.daml code snippets for docs.daml.com

* Remove 'controller..can' syntax in TestGrammar.daml

* changelog entry for 'controller..can' removal

* Update UNRELEASED.md

Co-authored-by: carrielaben-da <91496516+carrielaben-da@users.noreply.github.com>
2023-09-08 10:30:21 +00:00
Moisés Ackerman
6ae0e0111e
Upgrade to Bazel 5 (again) (#17319)
* Update to rules_haskell v0.16
* Update comments re bazel patches
* clean up bazel overrides
* Upgrade to Bazel 5.2.0
* Remove '--distinct_host_configuration=false'
* Update buildifier to 6.3.2
* Suffix macos and ubuntu caches with yyyymm
* bump windows cache to v14
* [REVERTME] bump linux/macos/darwin timeout to 4h
2023-09-01 15:15:36 +02:00
Samuel Williams
9179b2e7d0
Jar licensing (#17299)
* Add _distribute.jar target with correct license
Use it everywhere

* Fixes from review

* Change NOTICES to NOTICES.txt for consistency
2023-08-22 15:56:29 +01:00
Moisés Ackerman
ab038ab078
Add warning for uses of 'template-let' syntax (#17266)
* Update GHC_REV

* Add daml-test-files for template-let warning and its toggle

* Adapt daml-test-files to template-let warning

* update *.desugared-daml golden files

* Adapt //daml-lf/tests to template-let warning

* Adapt //test-common to template-let warning

* Update //compiler/damlc/tests:daml-doc golden output

* Adapt //docs to template-let warning

* Update GHC_REV
2023-08-18 15:14:46 +00:00
Raphael Speyer
47d21fe717
JSON API: add PruningTest and make it pass (#17167)
If the ledger has been pruned more recently that the last cached copy, then attempting to fetch the changes since that last offset will fail, rending the relevant template(s) unqueryable. This PR detects that condition, clears the cache for the relevant template and queries again, which will refresh the cache with a fresh copy of the ACS for that template, and serve the query from that.

I also made some usability tweaks around running canton-ee tests, to help improve the dev experience for failures I came across while trying to run them. Specifically
* Use `--config=canton-ee` to opt into the tests which require canton-ee
* When downloading that EE from artifactory, provide better diagnostics if the required auth isn't set up.
2023-07-27 09:12:22 +10:00
Gary Verhaegen
0e9cb10f7d
optionally get canton EE (#17039) 2023-06-27 11:00:24 +02:00
Krisztian Pinter
fed6e1ab7a
Add cabal support for some components (#16755) 2023-06-05 17:07:31 +02:00
dylant-da
943e140bbb
Add test generator for keeping Cabal files in sync (#16864)
* wip

* Add generate_cabal{_and_track}_cabal for keeping cabal files in sync

* lint

* Disable generate_and_track_cabal on windows

* Fix build script to generate comments on separate lines

* Fix sorting, detect alias rules for ghc-lib-parser

* allow specifying file in generate_and_track_cabal

* Fully qualify path to cabal generation target

* Add handler for daml_lf_dev_archive_haskell_proto

* fix trailing T;p

---------

Co-authored-by: Remy Haemmerle <Remy.Haemmerle@daml.com>
2023-05-23 10:00:02 +01:00
Samuel Williams
434b33d1e5
Fix name clashes with choices (#16858)
* Fix name clashes with choices

* Update ghc version
2023-05-18 15:24:40 +01:00
nickchapman-da
629439d876
[engine] choice authority syntax (#16602) 2023-03-29 15:19:47 +02:00
nickchapman-da
4298cde9d2
[compiler] avoid confusing emacs (#16597) 2023-03-23 13:04:49 +00:00
Moisés Ackerman
c423e8d151
Update "Working on ghc-lib" dev docs (#16574)
* Update ghc-lib/new-working-on-ghc-lib.md

* Update GHC_REV
2023-03-21 15:34:08 +00:00
Samuel Williams
8e90882105
Add errors for self, this, and arg. Add many tests (#16459)
* Add errors for self, this, and arg. Add many tests

* All WarnUnsupportDamlNames flag, warn for non breaking names, update tests

* Rename flag, update test

* Update to merged ghc version, changelog
2023-03-09 09:14:39 +00:00
Samuel Williams
3d70f2acd9
Update ghc from last ticket (#16446) 2023-03-02 15:40:48 +00:00
Samuel Williams
e16055d876
Fix controller unused arg warning for interface archives (#16430)
* Fix controller unused arg warning for interface archives

* Run tests:daml-desugar --accept
2023-03-02 13:56:37 +00:00
mziolekda
d8d55ca412
upgrade to scala 2.13.10 (#16423)
* update build system to scala 2.13.10

* fix failing scala files

* format

* fix compatibility

* resolve conflict on maven_install_2.13.json
2023-02-28 23:21:53 +00:00
Samuel Williams
0516143831
Fix warning about unused this in exception messages (#16313)
* Fix warning about unused `this` in exception messages

* Fix desugar tests

* Update ghc version

* Update test to include context
2023-02-16 15:32:30 +00:00
Samuel Williams
0c9ba9e691
Allow name shadowing in choices (#16306)
* Add bypassReduceLambda to allow shadowing

* Fix integration test

* Update ghc, update desugared cases

* Update ghc version
2023-02-14 11:39:42 +00:00
Samuel Williams
9fdeaf2ed4
Fix list pattern match warnings to use :: (#16278)
* Fix list pattern match warnings to use ::

* Add test case

* Update ghc version
2023-02-10 16:24:46 +00:00
Samuel Williams
90a0f13829
Implement name synonyms in error prints (#16242)
* Implement interface name synonyms in error prints
* Add View type synonym test
2023-02-08 10:33:57 +00:00
Moisés Ackerman
8784ebd9ce
Add choiceController and choiceObserver functions (#16088)
* Add daml test file for choice functions

* Add HasChoice{Controller,Observer} type classes

* Add choice_{controller,observer} in LF

* Update GHC_REV

* Generate HasChoice{Controller,Observer} instances

* Remove manual instances in daml test file for choice functions

* Remove/forbid choice{Controller,Observer} before featureChoiceFuncs

* update *.desugared-daml golden files

* Apply code review suggestion

* Update GHC_REV (da-master-8.8.1)

Co-authored-by: Remy <remy.haemmerle@daml.com>
2023-01-18 14:50:00 +00:00
dylant-da
03a76f4a61
Turn Daml-specific errors into suggestions on the original error (#16017) 2023-01-11 15:44:45 +00:00
dylant-da
0701488c40
Fix return type error for methods (#15999)
* Add tests for method result mismatches & for interface synonyms

* point to updated ghc
2023-01-06 12:54:34 +00:00
Gary Verhaegen
151e12b81a
bump copyright (#16002)
This is the result of:

- Updating `./COPY` to say `2023`.
- Running `./dev-env/bin/dade-copyright-headers update .`
2023-01-04 18:21:15 +01:00
dylant-da
445104a41b
Custom error / fix error for wrong expected return type of a choice. (#15855)
* Point to updated GHC

* add test for choice result mismatch

* add test for interface

* point to updated ghc

* point to merged ghc
2022-12-13 18:08:32 +00:00
dylant-da
f9d7301024
Fix erroneous code action/lens suggestions in Daml Studio (#15731)
* point to updated ghc

* point to updated GHC

* point to updated GHCIDE

* point to updated ghc

* Make _requires_ non-writable by inserting dollar - amend detectors

* Make _choice_ declarations non-writable (add dollar)

* Add checks for dollar-gen'd code require/view/interface_instance/method

* point to updated GHC

* point to updated ghc

* satisfy changelog

CHANGELOG_BEGIN
CHANGELOG_END

* point to updated GHC

* Point to updated ghcide

* update desugar tests

* point to revised daml-ghcide

* point to updated ghc

* point to updated ghc

* replace with isDamlGenerated

* point to merged ghc and merged ghcide
2022-12-13 12:06:40 +00:00
Stephen Compall
d14a6441e3
close the gap between scalac -Xlint and our global options (#15799)
* add -Xlint options requiring no changes

* add -Xlint:recurse-with-default

- very minor code changes

* factor http-json hj_scalacopts duplication

* use lf_scalacopts_stricter in libs-scala where NonUnitStatements was

* use hj_scalacopts in api-type-signature

* add nonlocal-return and nullary-unit to hj_scalacopts

* commented-out excluded options

* add unit-special globally

* check implicit-recursion for clients code
2022-12-07 16:08:33 +00:00
Moisés Ackerman
badfb9dcea
Improve error message for attempted use of '..' in record update (#15712)
* Update GHC_REV

* Add test case
2022-12-01 10:07:29 +01:00
Claudio Bley
e766f91338
Generate CSV file from all test evidences for ledger client components (#15372)
* Add new security categories

* Use test-security framework for OAuth2 middleware

* Regenerate `security-evidence.md`

* Explicitely exit the test-evidence generator

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Add //ledger-service/http-json:integration-tests-lib to evidence generator

* Skip maven artefacts on the classpath

The checker framework artefact (`checker-2.5.4.jar`) causes an `IllegalAccessError` when included in the runpath:
```
Exception in thread "main" java.lang.IllegalAccessError: class com.sun.tools.javac.code.Scope$ImportScope$ImportEntry cannot access its superclass com.sun.tools.javac.code.Scope$Entry (com.sun.tools.javac.code.Scope$ImportScope$ImportEntry is in unnamed module of loader java.net.URLClassLoader @31000e60; com.sun.tools.javac.code.Scope$Entry is in module jdk.compiler of loader app)
	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
	at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:550)
	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458)
	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	at org.scalatest.tools.SuiteDiscoveryHelper$.isDiscoverableSuite(SuiteDiscoveryHelper.scala:204)
	at org.scalatest.tools.SuiteDiscoveryHelper$.processClassName(SuiteDiscoveryHelper.scala:243)
	at org.scalatest.tools.SuiteDiscoveryHelper$.$anonfun$processFileNames$1(SuiteDiscoveryHelper.scala:279)
	at scala.collection.Iterator$$anon$9.next(Iterator.scala:577)
	at scala.collection.Iterator$$anon$6.hasNext(Iterator.scala:474)
	at scala.collection.Iterator$$anon$9.hasNext(Iterator.scala:576)
	at scala.collection.immutable.List.prependedAll(List.scala:152)
	at scala.collection.immutable.List$.from(List.scala:684)
	at scala.collection.immutable.List$.from(List.scala:681)
	at scala.collection.IterableFactory$Delegate.from(Factory.scala:288)
	at scala.collection.immutable.Iterable$.from(Iterable.scala:35)
	at scala.collection.immutable.Iterable$.from(Iterable.scala:32)
	at scala.collection.IterableFactory$Delegate.from(Factory.scala:288)
	at scala.collection.IterableOnceExtensionMethods$.toIterable$extension(IterableOnce.scala:178)
	at org.scalatest.tools.SuiteDiscoveryHelper$.processFileNames(SuiteDiscoveryHelper.scala:285)
	at org.scalatest.tools.SuiteDiscoveryHelper$.$anonfun$discoverSuiteNames$1(SuiteDiscoveryHelper.scala:132)
	at scala.collection.immutable.List.map(List.scala:250)
```

* Regenerate `security-evidence.md`

* Convert remaining TEST_EVIDENCE stanzas of HTTP JSON

* Regenerate `security-evidence.md`

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Print warning when a test suite could not be loaded

* Fix typo

* Use test-security framework for HTTP JSON

* Read files in tests lazily

The test-evidence generator tool needs to instantiate scalatest test suites in order
to access the tagged tests and collect relavant test entries.

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Regenerate `security-evidence.md`

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Use test-security framework for trigger service

* Use structural type to call `in` and `ignore` for different classes

* Remove Authentication category from EvidenceSecurity tool

There are no TEST_EVIDENCE annotations anymore.

* Add required trigger-runner-lib


* Import `scala.language.reflectiveCalls` where it is needed

* Remove left-over comments

* Add `test_evidence_binary` scala binary rule

This rule is a customized `scala_binary` rule which also accepts a `tests` attr and
generates a runpath file which is later consumed by scalatest to detect the relevant
scalatest test suites.

* Process test suites, add transitive deps

* Support nested tests_suites in `test_evidence_binary`

* Remove debug print's

* Add missing dependencies to test-evidence:generator

* Abort if test suites cannot be loaded

* Cleanup

* Reinstate scalacopts in http-json

* Reword the test description to not drop information

* Fix typo

* Explicitly exit the JVM on exceptions

This is required since non-daemon threads also prevent JVM shutdown when an exception was thrown.

* Format test-evidence/BUILD.bazel

* Resolve file paths lazily

This avoids a `NullPointerException` on Windows where Runfiles.rlocation returns `null`.

* Document new Security properties

* Print target directory and file name

* Clarify test descriptions

* Replace duplicate Security properties

Co-authored-by: Stephen Compall <stephen.compall@daml.com>Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2022-11-29 12:35:22 +01:00
mziolekda
a02c30440a
remove deprecated warnings everywhere [DPP-1281] (#15661)
* remove deprecated warnings in /ledger

* update generated documentation
2022-11-24 12:05:08 +00:00
dylant-da
ff6e776da0
Improve type error for Numeric types with precision > 37 and mistyped/missing field access (#15626)
* point to GHC with improved error

* Point to new GHC

* Fix error in RelTimeDetailsHidden1

* Point to updated ghc

* Point to new GHC

* Satisfy changelog check

CHANGELOG_BEGIN
CHANGELOG_END

* Fix assertion in RelTimeDetailsHidden1

* Amend tests with out-of-bounds NumericScale, point to updated GHC

* Drop quotes around type error

* Add tests for field access errors

* point to merged GHC
2022-11-22 15:44:25 +00:00
dylant-da
ccde982df5
Error for implementing non-interfaces (#15435)
* point to updated GHC

* satisfy changelog

CHANGELOG_BEGIN
CHANGELOG_END

* Point to updated GHC

* Expect new interface view error message in ImplementsNonInterface

* Add test for trying to implement a non-interface

* Point to GHC with better error suppression, fix tests

* Remove unused TryImplementNonInterface

* point to rebased ghc

* point to forwarded GHC changes

* point to updated GHC

* Fix expected error in ImplementsNonInterface

* Point to merged commit
2022-11-22 11:07:21 +00:00
Moisés Ackerman
c15617e87d
Fix view in view method definition (#15595)
* Update GHC_REV

* Update *.EXPECTED.desugared-daml

* Add regression test for #15459

* Method-in-method test now checks runtime behavior
2022-11-21 09:34:51 +00:00
dylant-da
c6d10b6246
Remove GHC.Tuple.Check module (#15549)
* Prevent unused import warnings

* Remove all explicit occurences of GHC.Tuple.Check

* Update desugar tests

* Remove magic from GHC.Tuple.Check

* Add GHC.Tuple.Check back to DarReaderTest

* Point to merged changes
2022-11-14 18:29:35 +00:00
dylant-da
bb67fb7f51
Move userWrittenTuple check into own helper in GHC.Tuple.Check (#15445)
* Create tuple size check function in GHC.Tuple.Check

* Update desugar test

* lint

* Fix accidental minus in stable-packages

* Replace GHC.Types with GHC.Tuple.Check in doctest generation

* Fix DarReaderTest

* Fix DamlDocTest for GHC.Tuple.Check import

* Remove unnecessary DamlSyntax pragma

Co-authored-by: Moisés Ackerman <6054733+akrmn@users.noreply.github.com>

* Point to merged GHC changes

* Point to akrmn's merged in changes

* Point back at merged in changes

* restart CI

Co-authored-by: Moisés Ackerman <6054733+akrmn@users.noreply.github.com>
2022-11-11 15:59:29 +00:00
Remy
49f4d2979e
[LF] strengthen compilation flags (#15481) 2022-11-09 14:12:04 +01:00
dylant-da
27f0f2b014
Catch mistyped view implementations via better Daml-specific type error detection (#15426)
* point to updated GHC

* satisfy changelog

CHANGELOG_BEGIN
CHANGELOG_END

* Point to updated GHC

* Expect new interface view error message in ImplementsNonInterface

* Add test for trying to implement a non-interface

* Point to GHC with better error suppression, fix tests

* Remove unused TryImplementNonInterface

* Point to merged GHC
2022-11-08 17:57:06 +00:00
dylant-da
1f2c728113
Fix overeager tuple warning (#15418)
* Success!

* Remove dead / unnecessary code from debugging

* Remove unused `userWrittenTuple` helper

* Import `GHC.Types.magic` where necessary

* Test for example in issue #15338

* satisfy changelog check

CHANGELOG_BEGIN
CHANGELOG_END

* Remove fst/snd lint tests, which weren't correct/useful anyways

* Update DocTest to import GHC.Types where necessary

* Update desugar tests

* Fix daml repl to include GHC.Types import

* Add GHC.Types to the doctest

* style improvement: if cond then action else pure () -> when cond action

* Point to merged GHC
2022-11-04 17:16:34 +00:00