Commit Graph

730 Commits

Author SHA1 Message Date
Moritz Kiefer
8360e9f75a
Mark Java Bot tests flaky (#10322)
See https://github.com/digital-asset/daml/issues/10273 for details on
the issue.

Summary is that the logic in the Java bots is fundamentally broken
since it modifies the pending set asynchronously so it can trigger for
the same contract multiple times.

While we can certainly fix that or hack around the bug in the tests,
given that the Java bots are deprecated, marking it flaky seems like a
better use of our time.

changelog_begin
changelog_end
2021-07-20 08:50:12 +00:00
Samir Talwar
bd12170af2
participant-state: Remove the ParticipantId, PackageId, and Party aliases. [KVL-1002] (#10308)
* participant-state: Remove the `ParticipantId` alias.

This alias adds nothing. By using `Ref.ParticipantId` directly, many
packages can remove their dependency on the _participant-state_ package.

CHANGELOG_BEGIN
CHANGELOG_END

* participant-state: Remove the `PackageId` and `Party` aliases.

They don't add anything. Let's just use `Ref`.

* kvutils: Restore missing compat imports.
2021-07-19 12:31:25 +00:00
Miklos
0eba812109
Remove trace_context field from Ledger API [KVL-1021] (#10256)
CHANGELOG_BEGIN
* [Integration Kit] Removed trace_context field from Ledger API and its bindings as we now have trace context propagation support via gRPC metadata. If you are constructing or consuming Ledger API requests or responses directly, you may need to update your code.
CHANGELOG_END
2021-07-15 19:06:25 +02:00
Remy
a8f190214c
LF: change type from Try to Either in archive module (#10277)
* LF: change type from Try to Either in archive module

This is the first part of restructuring errors in archive module.
This is part of #9974.

CHANGELOG_BEGIN
CHANGELOG_END

* Apply suggestions from code review

Co-authored-by: Stephen Compall <stephen.compall@daml.com>

* remove type alias

* apply stephen suggestion

* fix after rebase

* fix test

Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2021-07-14 19:24:31 +00:00
Remy
906184ce1a
LF: Simplify UniversalDarReader (#10271)
CHANGELOG_BEGIN
CHANGELOG_END
2021-07-14 15:38:22 +02:00
Remy
caf85a2270
LF: rationalize archive Parser/Reader/Decoder (#10239)
CHANGELOG_BEGIN
CHANGELOG_END
2021-07-14 11:05:17 +02:00
Samir Talwar
6c63f96ba9
participant-integration-api: Move SeedService here. (#10261)
I don't see a reason why it's part of the participant state API, and
it definitely doesn't need to change between v1 and v2.

CHANGELOG_BEGIN
- [Integration Kit] The class ``SeedService`` has been moved from the
  *participant-state* Maven package to the *participant-integration-api*
  Maven package, under the Java package name
  ``com.daml.platform.apiserver`` to reflect its usage by the API
  server, not the participant state API. If you use this class directly,
  you will need to change your imports.
CHANGELOG_END
2021-07-14 08:35:29 +00:00
Remy
6e8ec1d618
LF: Drop old depreated code (#10251)
CHANGELOG_BEGIN
CHANGELOG_END
2021-07-13 13:10:00 +00:00
Remy
f19f5b0821
LF: Simplify DarReader (#10217)
This PR simplifies a bit DarReader and UniversalDarReader

This is a follow up of #10208.

CHANGELOG_BEGIN
CHANGELOG_END
2021-07-08 17:03:45 +02:00
Remy
41b8448b17
LF: Simplify archive reader. (#10208)
* LF: Simplify archive reader.

- decouple Reader and Decoder
- introduce case class to handle hash, proto payload, and version

CHANGELOG_BEGIN
CHANGELOG_END

* Address Moritz' review

* cosmetic
2021-07-07 19:56:40 +00:00
Samir Talwar
e7e8a5705b
daml-lf/data: Optionally truncate party names in structured logs. [KVL-996] (#10163)
* daml-lf/data: Truncate party names in log output, on request.

The party name can grow quite long, so we offer ledger implementors the
opportunity to truncate it in structured log output.

Unfortunately, because we use Logback through the global
`LoggerFactory`, there is no place to inject logging configuration. This
means we also need to use global, mutable state to configure logging
output. I have added a `LoggingConfiguration` class+object in Daml-LF
Data, which may not be the best place, but I can't think of a better
one right now. I suggest we leave it there until it has reason to grow,
at which point we may want to move it.

CHANGELOG_BEGIN
CHANGELOG_END

* logging-entries: Make `ToLoggingValue` mixin-able.

* participant-integration-api: Truncate parties in filters when logging.

* participant-integration-api: Cast to `Party` for logging.

Invalid input should not break the request at this point. No assertions.

* daml-lf/data: Move `Party to LoggingValue` to a new package.

This avoids the transitive dependency issue most of the time.

* daml-lf-data: Move the `Identifier` logging to another package.

Again, reduces the need for transitively depending on _logging-entries_.
2021-07-01 16:50:49 +00:00
Samir Talwar
5493e4e2b1
language-support/scala/codegen: Set the logging level to WARN. (#10165)
I don't find the output from Bazel very helpful most of the time. It
does, however, produce a lot of noise which I have to filter through
when something else goes wrong.

CHANGELOG_BEGIN
CHANGELOG_END
2021-07-01 16:49:46 +00:00
Samir Talwar
eda1245311
daml-lf/data: Add a conversion from Ref.Identifier to LoggingValue. (#10157)
I am getting quite tired of (Bazel + Scala)'s attitude to transitive
dependencies.

CHANGELOG_BEGIN
CHANGELOG_END
2021-06-30 20:05:06 +00:00
Stefano Baghino
f1ffd52e08
Java bindings: add DamlRecord, deprecate Record (#10132)
* Java bindings: add DamlRecord, deprecate Record

Fixes #10130

changelog_begin
[Java bindings] In order to avoid clashing with `java.lang.Record` (introduced
in Java 14), `com.daml.ledger.javaapi.data.Record` has been renamed to
`com.daml.ledger.javaapi.data.DamlRecord`. The old name has been used to
denote a sub-type of the newly renamed one, so it can still be used, but it has
been marked as deprecated.
[Java codegen] The Java codegen now uses the `DamlRecord` type wherever `Record`
was used before.
changelog_end

Boy-scout rule:
- removed references to ~old~ ancient versioning
- used `@deprecated` Javadoc annotation wherever meaningful
- some import re-arrangement performed by the linter

* Address https://github.com/digital-asset/daml/pull/10132#discussion_r659705929
2021-06-28 12:08:51 +00:00
Moritz Kiefer
46a66e2f2a
Upgrade jest to the latest version (#10107)
This ends up transitively bumping yargs-parser which has a
vulnerability but more generally I also just like being on the latest version.

changelog_begin
changelog_end
2021-06-24 11:16:29 +00:00
Moritz Kiefer
0970821ce7
Bump cpu alloc for build-and-lint (#10099)
changelog_begin
changelog_end
2021-06-23 17:57:41 +00:00
Samir Talwar
e32856ecf4
dev-env: Use nix-shell for scripts instead of sourcing dade-common. (#10088)
* Introduce shell.nix to expose the Nix tooling, if required.

* dev-env: Rewrite the ghcide script to use nix-shell.

* language-support/hs: Use `nix-shell` for export-package.sh.

* Nix: Add a shebang to the Bazel wrapper script.

CHANGELOG_BEGIN
CHANGELOG_END

* dev-env: Standardize the `set` header in `ghcide`.
2021-06-23 09:39:00 +00:00
Stephen Compall
401069ef00
prepare for upgrade to Scalaz 7.3.3 (#9997)
* upgrade scalacheck to 1.14.3

* regenerate maven_install files

* some different names and implicits

* remove some fromTryCatchNonFatal

* more porting

* port fromTryCatchNonFatal to attempt

* factor the assertions in SignatureSpec to avoid \/

* deal with invariant \/

* make partial unification do what we want

* \/, parse*, and toNel

* many uses of the .right method

* a legitimate use of fromTryCatchThrowable

* rebuild maven pins

* further invariant \/

* OneAnd and Nel interface changes

* further Either games

* \/ and reformatting

* \/ in http-json

* \/ in http-json

* deprecations

* more invariance

* cleanup unused

* more invariance; http-json compiles

* final either follies

* small 2.12 extra incompatibility

* rebuild deps

* revisit a couple earlier fixes using nicer expressions I learned later

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* repin 2.12
2021-06-15 14:41:48 -04:00
Moritz Kiefer
ab55fb2546
Handle module-prefixes in Java codegen (#9929)
* Handle module-prefixes in Java codegen

changelog_begin

- [Java Codegen] The Java codegen will now pick up the
  `module-prefixes` field from `daml.yaml` which can be used to handle
  module name collisions between different DALFs.

changelog_end

* Reference ticket in todos

changelog_begin
changelog_end

* less stupid map initialization

changelog_begin
changelog_end

* Pull out string assembly

changelog_begin
changelog_end

* Update language-support/java/codegen/src/main/scala/com/digitalasset/daml/lf/codegen/CodeGenRunner.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Update language-support/java/codegen/src/main/scala/com/digitalasset/daml/lf/codegen/CodeGenRunner.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* fewer views

changelog_begin
changelog_end

* more assert

changelog_begin
changelog_end

* more private

changelog_begin
changelog_end

* fmt

changelog_begin
changelog_end

* fix scala 2.12

changelog_begin
changelog_end

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2021-06-09 07:31:28 +00:00
Remy
cbcec32e70
LF: clean up the remaining Exception TODOs (#9913)
CHANGELOG_BEGIN
CHANGELOG_END
2021-06-08 11:49:02 +02:00
Hubert Slojewski
a0d3e80fd6
Add missing telemetryContext instances in the ledger-api-client's Ctx (#9916)
CHANGELOG_BEGIN
CHANGELOG_END
2021-06-07 18:24:04 +02:00
Remy
512f1fd87d
LF: Preview version of LF 1.14 (#9906)
+ Preview support for Exceptions.
+ byKey flag for transaction nodes.

CHANGELOG_BEGIN
CHANGELOG_END
2021-06-03 20:40:53 +02:00
Moritz Kiefer
850c7310e8
Include PackageMetadata in LF interface (#9892)
* Include PackageMetadata in LF interface

I want to use this in codegens but more generally, I think it also
fits well into the scope of the iface library in that it’s only
metadata and relatively stable.

changelog_begin
changelog_end

* Iterating more is less

changelog_begin
changelog_end

* fix tests

changelog_begin
changelog_end

* fmt

changelog_begin
changelog_end

* fix scala 2.12

changelog_begin
changelog_end
2021-06-03 09:06:23 +00:00
Hubert Slojewski
c0359ac9ae
Add trace context propagation in CommandClient.trackCommandsUnbounded [KVL-961] (#9833)
CHANGELOG_BEGIN
CHANGELOG_END
2021-06-01 13:31:58 +02:00
Moritz Kiefer
b6a89a6948
Bump ws to address security advisory (#9844)
Tested manually that Navigator does not fall over with this.

changelog_begin
changelog_end
2021-05-31 10:38:23 +02:00
Moritz Kiefer
cae429237e
Upgrade rules-nodes to version 3.5.0 (#9635)
* Upgrade rules-nodes to version 3.4.2

No particularly strong reason for doing that. I just like staying up2date.

changelog_begin
changelog_end

* Patch Windows

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end
2021-05-25 16:04:39 +02:00
Miklos
f5c84a2aa4
Daml-LF: Damlification of Scala/Haskell files (#9666)
CHANGELOG_BEGIN
CHANGELOG_END
2021-05-20 09:52:33 +02:00
Miklos
395ff58eec
Damlification of Bazel files (#9670)
CHANGELOG_BEGIN
CHANGELOG_END
2021-05-17 11:41:44 +02:00
Miklos
cd9933328a
runtime: Damlification of Scala files (#9668)
CHANGELOG_BEGIN
CHANGELOG_END
2021-05-17 10:28:50 +02:00
Leonid Rozenberg
58d4281743
Document web socket event handlers of TS Stream type. (#9634)
* Document web socket event streams.

* Address https://github.com/digital-asset/daml/pull/9634#pullrequestreview-656354109

changelog_begin
changelog_end

Co-authored-by: Stefano Baghino <stefano.baghino@digitalasset.com>
2021-05-12 09:09:14 +02:00
Stephen Compall
ca9e89b3da
check whether collection.compat is unused when compiling for Scala 2.12 (#9604)
* check whether collection.compat is unused when compiling for Scala 2.12

- Instead of always suppressing warnings for collection.compat._,
  we should only do it for Scala 2.13

- We can also reduce boilerplate by automatically adding this
  option when both silencer_plugin and collection-compat are
  present

CHANGELOG_BEGIN
CHANGELOG_END

* remove unused import

* remove another unused import

* remove even more unused imports

* missed compat dependency

* more missed compat dependencies

* missed compat dependency

* use scala_deps in scaladoc_jar

- #8423 inlined the major version expansion, but this seems to
  have been prior to proper support by scaladoc_jar

* restore custom handling of participant-integration-api

- fixing scaladoc_jar isn't worth it for a single case, as with
  deps vs scala_deps
2021-05-11 21:54:14 +00:00
Moritz Kiefer
48939b513b
Address vulnerabilities in //language-support/ts/packages (#9614)
This is a bit annoying. We have to force react-test-renderer because
otherwise we pull in both React 17 and React 16 which breaks stuff.
We also have to force hosted-git-info which is pulled in transitively
via jest. The next major version of Jest will drop that dependency but
for now there is no other way to get rid of this.

changelog_begin
changelog_end
2021-05-07 14:00:54 +02:00
Moritz Kiefer
a27b2c56bc
Address more exception todos (#9582)
* Address more exception todos

A bit of a kitchen sink PR to address a bunch of the trivial todos
that didn’t seem worth splitting out into separate PRs.

changelog_begin
changelog_end

* Revert changes to TransactionSpec

changelog_begin
changelog_end
2021-05-05 16:00:08 +02:00
Gary Verhaegen
9b749146c6
ts: add closeHandler to ByKey* on default ledger (#9565)
CHANGELOG_BEGIN
CHANGELOG_END
2021-05-04 12:37:34 +02:00
Stefano Baghino
b17f5b836f
Fixes the docstring for useStreamFetchByKeys (#9563)
changelog_begin
changelog_end

Originally reported at https://discuss.daml.com/t/usestreamfetchbykeys-is-missing-closehandler-argument/2552
2021-05-04 11:10:00 +02:00
Moritz Kiefer
4aca199bbd
Upgrade to Scala 2.13.5 (#9528)
changelog_begin
changelog_end
2021-04-29 13:05:57 +00:00
Samir Talwar
e1af564bcc
Switch from @silent to @nowarn. (#9498)
* Switch from `@silent` to `@nowarn`.

This annotation is native to Scala 2.12.13+ and 2.13.2+. It replaces
most usages of `@silent`.

I had to get creative about a couple of use cases that didn't work.
Specifically:

  1.  Suppressing deprecation warnings works, but Scala 2.12 erroneously
      complains that the `@nowarn` is unnecessary.  I had to suppress
      this warning too with `-Ywarn-unused:-nowarn`.
  2.  I can't seem to suppress the warning, "The outer reference in this
      type test cannot be checked at run time." Instead, I have
      refactored the code to remove the warning.

We still need to use the silencer plugin to suppress some warnings about
unused imports (because of compatibility between Scala 2.12 and 2.13),
but this means we no longer need the library, and therefore it is not a
transitive dependency that downstream consumers need to worry about.

CHANGELOG_BEGIN
CHANGELOG_END

* Add some comments around `@nowarn` support.

* language-support/scala: Fix a warning suppression.

* Revert to the default warnings.

Compatibility was complaining.
2021-04-26 19:46:14 +00:00
Moritz Kiefer
83e6ec8f37
Bump build and lint startup timeout (#9476)
Not quite sure what is going wrong and if this is the issue but I’m
failing to reproduce it reliably to test this so bumping and seeing if
we still hit it on `main` seems like the easiest option.

changelog_begin
changelog_end
2021-04-23 11:44:14 +02:00
Stephen Compall
5775ecc8ff
Scala codegen: use immutable.Map newtype for GenMap instead of InsertOrdMap (#9449)
* add the MapOps argument to Compat.MapLike

* use immutable.Map newtype for GenMap instead of InsertOrdMap

- bindings tests and sample-app tests compile under 2.13 and 2.12

* add changelog

CHANGELOG_BEGIN
- [Scala Codegen] ``GenMap`` no longer requires ``InsertOrdMap``;
  ordinary Scala immutable ``Map``s may be effectively used
  instead.  We still encourage use of the ``Primitive.GenMap``
  alias for proper typing, future-proofing, and deprecation
  warnings.
  See `issue #9449 <https://github.com/digital-asset/daml/pull/9449>`__.
CHANGELOG_END

* implicitly convert from Map; test several aspects of the Scala GenMap API

* test best-effort class preservation

* drop the CBF for 2.12, as with TextMap

* test conversion in a contract construction

* remove unused CanBuildFrom compats

* combine Primitive tests

* reenable 2.13-only tests

* remove InsertOrdMap references in Shrink/Show instances

* remove test that GenMap order is preserved by Scala
2021-04-22 16:11:13 +00:00
Moritz Kiefer
6ab965577e
Fix scala version in quickstart-scala (#9401)
Setting this manually has resulted in #9397 where we use a scala
version that doesn’t have a silencer version. This PR changes the
quickstart template to use the scala version we also use in our builds
which should avoid them getting out of sync in the future.

changelog_begin
changelog_end
2021-04-13 15:54:40 +02:00
Robin Krom
f84b6abc05
daml build: add a --access-token-file for remote dependencies (#9358)
* daml build: add a --access-token-file for remote dependencies

This adds a `ledger.access-token-file` field in the `daml.yaml` project
file and a `--access-token-file` flag to `daml build` to authorize
querying/fetching of remote dependencies.

CHANGELOG_BEGIN
[daml build] A new flag `--access-token-file` is added for the `daml
build` command. It allows the specify the path to an access token to
authenticate against the ledger API. This is needed if the project
depends on a remote Daml package hosted on such a ledger. Alternatively,
the path to the token can also be specified in the `daml.yaml` project
file under the `ledger.access-token-file` field.
CHANGELOG_END
2021-04-08 23:20:52 +02:00
Remy
568a8528a8
LF: release preview of LF 1.13 (#9329)
CHANGELOG_BEGIN
- Daml: (Early access) add support for BigNumeric
CHANGELOG_END
2021-04-07 12:42:54 +02:00
Moritz Kiefer
48e273dc58
Bump y18n to address security vulnerability (#9301)
changelog_begin
changelog_end
2021-03-31 12:40:21 +02:00
Stephen Compall
a6da995ecb
use representational variance for generated Scala classes (#8879)
* variance calculation without references or cycles

* an idea for handling the recursive reference case elegantly

* extra data for resolving constraints from [recursive] references later

* note on TypeVar case

* trying a different delayed resolution

* quagmire

* the problem: we were trying to solve too early and mix reference levels

* force inner delays to be pulled out; squash the delays iteratively

* make the variance result accessible

* phantoms are covariant

* scalafmt

* simplest GenMap test

* reference tests

* factoring

* don't treat covariant reference arguments as phantom

* making the wrong reference in delays was breaking solve steps

* test cons list self-recursion

* more recursion testing

* -> is better for 2-tuples here

* test cumulative variance tracking

* uninhabited types have variance too

* test mutual recursion with lots of indirect references

* trigger for failing case in incremental resolution

* redesign delayed constraints to handle nesting properly, and be much simpler

* integrate variance calculation with rest of Scala codegen

* handle concrete type and all-phantoms cases; test

* add integration test for variance inference

* add changelog

CHANGELOG_BEGIN
- [Scala Codegen] Type parameters used in GenMap keys are now supported.
  See `issue #8879 <https://github.com/digital-asset/daml/pull/8879>`__.
CHANGELOG_END

* proper doc for covariantVars

- suggested by @cocreature; thanks

* remove useless local var

- suggested by @cocreature; thanks

* add another recursive function to capture more invariants

- suggested by @cocreature; thanks

* absence in what

- suggested by @cocreature; thanks

* use Identifier instead of I

- suggested by @cocreature; thanks

* remove unused type vars; make goSdt cleaner

* also precache variance of choice and key-referenced types

- suggested by @cocreature; thanks
2021-03-17 16:47:18 +00:00
nickchapman-da
9b2158508b
Add new variant to Value.scala for builtin-exceptions. (#9084)
* Add new variant to Value.scala for builtin-exceptions.

final case class ValueBuiltinException[+Cid](tag: String, value: Value[Cid]) extends Value[Cid]

And push through the code consequences.
Most places fixed up.
A couple more things to do in this PR (marked NICK)
A couple of things which can be left for later (marked 8020)

fix build

fix another scala match

changelog_begin
changelog_end

* fix any match

* add marker of code which needs attending to in the PR

* extend ledger-api value.proto & fix LfEngineToApi

* undo/comment-out the change to value.proto

* add tests in HashSpec for BuiltinException

* code but dont yet enable value-gen for builtin exceptions

* address comments which suggest we crash in various places

* support BuiltinException in scenario_service.proto

* one more TODO 8020 tag
2021-03-16 13:15:33 +00:00
Kamil Bożek
32d4bf92ec
Do not require a JWT token for Health and Reflection services [DPP-277] (#8969)
* Do not require a JWT token for Health and Reflection services

CHANGELOG_BEGIN
- A JWT token is no longer required to call methods of Health and Reflection services
CHANGELOG_END

* Let service's authorizer decide about rejections

* Updated authorization test

* Added integration test for unsecured authorisation test for the Health service

* Added integration test for unsecured authorisation test for the Server Reflection service

* Updated Claims doc comments

* Minor change

* Reduced code duplication with SecuredServiceCallAuthTests and UnsecuredServiceCallAuthTests

* Added copyrights

* Move response status handling logic to Authorizer
2021-03-03 11:05:35 +00:00
Remy
a9b035e92e
LF: release LF 1.12 (#8856)
CHANGELOG_BEGIN

    - [LF] Release LF 1.12. This version reduce the size of transaction

    - [Compiler]: Change the default LF output from 1.8 to 1.11.

CHANGELOG_END
2021-02-24 21:17:33 +01:00
Remy
d17dd7e5f1
Ledger API test tool: test against legacy and preview version. (#8913)
* Ledger API test tool: test against legacy and preview version.

+ use dictionary instead of alias to map version keywords to LF version 

CHANGELOG_BEGIN
CHANGELOG_END
2021-02-23 09:19:17 +01:00
Moritz Kiefer
e82410bbae
Fix and simplify readActiveContracts in Java codegen tests (#8915)
changelog_begin
changelog_end
2021-02-22 18:17:08 +00:00
Remy
d92f2c7003
Ledger: refactor bazel packaging of tedger test tool (#8894)
improve previous generalization from #8695

- use lf version instead keyword (like 'stable', 'latest', 'dev') to
  tag actual target.  This will allow two keywords to map to the same
  versions without doing twice the compilation/test work.

- use alias to map keywords tag target to versioned tag target.

- move package manage dar to test_commong.

CHANGELOG_BEGIN
CHANGELOG_END
2021-02-22 11:41:19 +01:00