Commit Graph

2464 Commits

Author SHA1 Message Date
associahedron
e8a5db5de3 Add Numeric to basic types table. (#3595) 2019-11-27 09:10:33 +00:00
Remy
997e206779 daml-lf: type synonyms are also taken into account for collisions (#3644) 2019-11-27 08:36:59 +00:00
Robin Krom
a2aacaa79b language: better error message for the script runner (#3637) 2019-11-26 21:27:35 +00:00
Moritz Kiefer
afe25febbf
Remove debug printf from daml script (#3640) 2019-11-26 20:45:33 +01:00
Rohan Jacob-Rao
094214aa55 Notify IDE users about new daml-driven blog posts (#3623)
* Successfully parse rss xml and find latest blog post object

* Show notification with link to blog (on every ide startup...)

* Only show blog post if not seen before

* Formatting

CHANGELOG_BEGIN

- [SDK] Notify IDE users about new daml-driven blog posts

CHANGELOG_END

* Add setting for user to opt out of notifications

* Use await rather than then

* Catch and swallow exceptions

* Another await and minor cleanup
2019-11-26 19:34:56 +00:00
Stefano Baghino
4d922e763b Add authentication to Java identity client (#3630)
* Add authentication to Java identity client

Contributes to #3626

No changelog added right now, it will be added with the PR that
effectively closes the ticket.

Moves `LedgerCallCredentials` to a shared package.

Establishes the test framework for other Java clients.

* Address https://github.com/digital-asset/daml/pull/3630#discussion_r350663731

* Address https://github.com/digital-asset/daml/pull/3630#discussion_r350663731

* Rebase and fix compilation errors

* Fix type of jar in artifacts.yaml

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* Fix more errors emerged after rebase

* Add dependency for DAML assistant ITs

* Add missing dependency

* Fix wrong dependency

* fix release
2019-11-26 18:51:09 +00:00
Stefano Baghino
d951d738ee
Add support for authenticated ledger (#3639)
Closes #3627

- Pass a token at startup to read packages
- Re-read the token every time new packages are fetched to ensure token
freshness
- If the token is expired, new packages will not be retrieved

CHANGELOG_BEGIN

- [JSON API] Accept a path to a file containing a token at startup for
package retrieval. See `issue #3627 <https://github.com/digital-asset/daml/issues/3627>`__.

CHANGELOG_END
2019-11-26 19:45:47 +01:00
Robin Krom
d90eb357ad
language: upgrades: prefix stdlib imports everywhere. (#3633) 2019-11-26 19:28:11 +01:00
Gary Verhaegen
b72fc19b6e
add comments to unreleased.sh (#3575) 2019-11-26 18:27:35 +00:00
Martin Huschenbett
09a681d4f3 Add a readme for daml2ts (#3638) 2019-11-26 17:49:40 +00:00
Moritz Kiefer
f0a111ca1a
Get grpc from nix on unix (#3632)
* Get grpc from nix on unix

The one from Bazel seems to cause linking issues when trying to run
things in GHCi. I’ve spent some time trying to use rules_foreign_cc to
build gRPC using CMake but decided that for my own sanity it’s better
to not pursue that further.

* Address review comments

* Add missing module load

* Cleanup GHCI_SCRIPT

* use the correct file ending on macos

* Import is_linux

* Switch back to grpc-1.23

The newer version seems to cause issues in combination with the java libraries.

* Try to fix package_app on macos

* more debugging

* Maybe this is not necessary, we will never know

* linkers are the worst

* Remove debugging output again

* readd rpaths

* treat libdispatch specially

* remove hack

* more fooling around

* lalala
2019-11-26 18:47:39 +01:00
Moritz Kiefer
41958b1430 Fix typo in synonym spec (#3636) 2019-11-26 17:15:58 +01:00
Martin Huschenbett
167232bd81
Ban usage of generic templates (#3631)
We don't drop full support from the compiler yet but rather ban their use by
adding a check to the preprocessor. We'll remove the actual support as we go
along with fixing the upgrading story.

CHANGELOG_BEGIN

- [DAML Compiler] Make the experimental feature "generic templates"
  unavailable. The current implementation is at odds with other, more important
  language features still under development.

CHANGELOG_END
2019-11-26 17:15:17 +01:00
Moritz Kiefer
f402acf19b Add specification for type synonyms in DAML-LF (#3635)
* Add specification for type synonyms in DAML-LF

* Fix identifiers for type synonyms

* Use S for type synonyms

* Address review comments

* Split up TyTypeSynonym and DefTypeSynonym
2019-11-26 15:01:44 +00:00
Remy
06b7a106a1 DAML-LF: generic map values protobuf format (#3583)
* DAML-LF: generic map values protobuf format

* Apply suggestions from code review

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* Address Gerolf's comment

* Apply suggestions from code review

Co-Authored-By: Stephen Compall <stephen.compall@daml.com>

* fix test

* Address Stephen's last comment
2019-11-26 13:34:33 +00:00
Stefano Baghino
3d3322a06a
Fix wording in authorization tests (#3629)
Address previously unaddressed https://github.com/digital-asset/daml/pull/3598#discussion_r350581428
2019-11-26 11:12:30 +01:00
Stefano Baghino
160da16227
Move AuthorizationIT to sandbox integration tests (#3598)
* Move AuthorizationIT to Sandbox ITs

* Use refactored code in Extractor

* Apply a few improvements, see body

- Run previously ignored tests
- Make token signing private
- Fix stream observer utility comment more accurate
- Add read-only token helper

* Add r/w token split tests

* Fix formatting and imports

* Address https://github.com/digital-asset/daml/pull/3598#discussion_r349769546

* Address https://github.com/digital-asset/daml/pull/3598#discussion_r349769801

* Fix inaccurate test

A test was incorrectly trying to get a token with an invalid signature

* Address reviews
2019-11-26 10:13:36 +01:00
Martin Huschenbett
e9be9419f8
Remove documentation for generic templates (#3619)
Unfortunately, our current implementation of generic templates is at odds
with cross-SDK imports of DARs and hence the whole upgrading story for
DAML models, particularly around type classes, which generic templates
heavily rely upon. Thus, we've decided to drop support for generic templates
for now.

This PR does the first step by dropping the documentation. I've deliberately
not added a changelog entry here. I'll do that once we have dropped support
from the compiler.
2019-11-26 10:10:13 +01:00
Andreas Herrmann
15f12eea6a Remove Bazel override (#3603)
* Remove Bazel override

The extended default shell env is no longer required. See
https://github.com/digital-asset/daml/pull/3597#pullrequestreview-322096242

* Import diff from rules_sh posix toolchain
2019-11-26 09:06:44 +00:00
Samir Talwar
8818ea7cbb kvutils: Clean up the InMemoryKVParticipantState integration tests a little. (#3621)
* kvutils: Nest all InMemoryKVParticipantState tests inside the block.

* kvutils: Simplify InMemoryKVParticipantStateIT where possible.

* kvutils: In InMemoryKVParticipantStateIT, move assertions to the end.

It's confusing when the assertions are before the submissions.

* kvutils: Move test helpers into a companion object.

* kvutils: Use BeforeAndAfterEach in testing to remove duplication.

* kuvtils: In testing config updates, always increment the generation.

* kvutils: Assert something in the "initial conditions" integration test.
2019-11-25 17:49:56 -05:00
Stefano Baghino
25f65f746b
Move DamlLedgerClient to builder-based constructors (#3620)
With the intent of adding more functionality to the Java bindings, in
particular authentication, this PR deprecates existing constructors in
favor of a more flexible builder based approach.

CHANGELOG_BEGIN

- [Java Bindings] Deprecated existing constructors for
``DamlLedgerClient``, please use the static ``newBuilder`` method to
instantiate a builder and use it to create the client, starting from
either a ``NettyChannelBuilder`` or a plain host/port pair.

CHANGELOG_END
2019-11-25 23:19:14 +01:00
Moritz Kiefer
770a9ae5e4 Support empty packages in daml build (#3624)
We still error out if the directory in `src` doesn’t exist to catch
typos.
2019-11-25 20:44:34 +00:00
nickchapman-da
0f0a12db22 Sdk access token file (#3607)
* support --access-token-file in SDK

* rename: HostAndPort -> LedgerArgs; HostAndPortFlags -> LedgerFlags

* integration test for --access-token-file

* doc & more help

* address comments: use FilePath

* adapt to renamed arg: --auth-jwt-hs256-unsafe

* avoid use of single quotes on command line
2019-11-25 17:38:41 +00:00
Remy
3fa6043056 fix typo in java codegen doc (#3618)
Thanks Miklos !
2019-11-25 16:34:14 +00:00
Samir Talwar
8df4b9ec33 ledger-api-test-tool: Use the default Akka execution context. (#3608)
* ledger-api-test-tool: Use the default Akka execution context.

Akka will use the global fork-join pool by default, which is fine for
our purposes. There's no longer a need to use a specific execution
context, because we no longer use it to limit the work in progress.
That's now done by the parallelism parameter to Akka Streams'
`mapAsyncUnordered`.

* ledger-api-test-tool: Shut down the ActorSystem.
2019-11-25 15:57:17 +00:00
Samir Talwar
603ee9367b ledger-api-test-tool: The endpoints are mandatory. (#3611)
We should fail fast if no endpoints are provided, rather than timing out
every test.
2019-11-25 15:32:35 +00:00
Ognjen Maric
48174c8ee4
Fixed a stale reference to a MustPaint contract (#3613) 2019-11-25 16:29:32 +01:00
Robert Autenrieth
4c3f6014f6
Implement support for RSA-signed JWT tokens (#3526)
* Improve Navigator output

* Fix Navigator not using the access token

* Add RSA signatures for JWT tokens

* Remove unused method

* Add timeouts for reading JWKS

* Fix test

* Rename method for consistency

* Improve comment

* More renaming for consistency

* CHANGELOG

CHANGELOG_BEGIN
- [Sandbox] Add CLI options to start the sandbox with JWT based authentication with RSA signed tokens.
  See `issue #3155 <https://github.com/digital-asset/daml/issues/3155>`__ .
- [Navigator] Fixed a bug where the `--access-token-file` option did not work correctly.
CHANGELOG_END

* Make JwksVerifier limits configurable

* Make SimpleHttpServer private
2019-11-25 16:29:24 +01:00
Stefano Baghino
552bf6497d Obscure shared-secret-based authenticated sandbox (#3609)
The shared-secret-based authentication is there exclusively for testing,
this PR makes sure that this is correctly reported through the CLI and
hides this until further notice.

Makes sure that the flag clearly says that this option is not meant for
production use cases.

A warning is already printed when this feature is used, both at startup
and every time this verifier is used.

CHANGELOG_BEGIN

- [Sandbox] The ``--auth-jwt-hs256`` is renamed to
``--auth-jwt-hs256-unsafe``: you are advised to _not_ use this JWT token
signing way in a production environment.

CHANGELOG_END
2019-11-25 14:27:21 +00:00
Robert Autenrieth
243ca0f10a Print usage for the unreleased.sh script (#3585) 2019-11-25 13:26:08 +00:00
Remy
d152c7cbfd daml-lf: rename Map to TextMap in archive proto (#3589)
* daml-lf: rename Map to TextMap in archive proto
+ in Scala/haskell AST

* a bit more renamming

* Update compiler/daml-lf-tools/src/DA/Daml/LF/TypeChecker/Serializability.hs

Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>

* fix test

* Apply suggestions from code review

Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>

* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Base.hs

Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
2019-11-25 12:14:57 +00:00
Moritz Kiefer
27051160eb
Support multi-participant DAML script (#3605)
* Support multi-participant DAML script

fixes #3555

CHANGELOG_BEGIN

- [DAML Script - Experimental] DAML script can now run be used in distributed topologies.

CHANGELOG_END

* Fix ports in multiparticipants tests
2019-11-25 12:43:47 +01:00
Oliver Seeliger
26b569f73d Properly drop invalid actor name chars in standalone indexer actor name (#3604)
Closes #3327

CHANGELOG_BEGIN

[Ledger]
  * Allow non-alphanumeric characters in ledger api server participant ids (space, colon, hash, slash, dot). Proper fix for change originally attempted in v0.13.36. See issue `issue #3327 <https://github.com/digital-asset/daml/issues/3327>`__.

CHANGELOG_END

Proper fix replacing f39bf7fc5a which does not work
2019-11-25 11:16:59 +00:00
Samir Talwar
159104adb1 ledger-api-test-tool: Don't discard the cause on fail. (#3600)
* ledger-api-test-tool: On `fail`, don't discard the cause.

* ledger-api-test-tool: Delete a redundant `new`.
2019-11-25 10:43:01 +00:00
Oliver Seeliger
d35961d857
Ledger api test tool canton run config fix (#3602)
Missing from 15a7f72eae
2019-11-25 11:13:27 +01:00
Andreas Herrmann
10031a614c Remove all instances of use_default_shell_env = True (#3597)
* use_default_shell_env = False in proto_gen

* use_default_shell_env = False in scala_source_jar

* use_default_shell_env = False in scaladoc_jar

* use_default_shell_env = False in dar_to_scala

* use_default_shell_env = False in _real_pkg_tar

* use_default_shell_env = False in client_server_build

* use_default_shell_env = False in npm_package
2019-11-25 09:15:00 +00:00
Samir Talwar
1bd2ec5b1a grpc-utils: Simpler extractors. (#3594)
* grpc-utils: Simpler GrpcStatus extractors when you know the code.

And don't care about the description.

* grpc-utils: Simpler GrpcException extractors when you know the code.

And don't care about the description.

* Apply suggestions from code review

Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Add `new` to `SpecificGrpc{Exception,Status}` constructor calls.

Also, don't blindly apply GitHub suggestions.
2019-11-22 21:09:17 +00:00
Andreas Herrmann
16b6f1b4cb Hide warnings of external Haskell packages (#3593) 2019-11-22 18:27:33 +00:00
Andreas Herrmann
f1f41b6e3e Windows: Take stack from dadew (#3596)
rules_haskell looks for stack in PATH. On Windows it is provided by
dadew (i.e. scoop). rules_haskell then symlinks (copies on Windows) the
stack binary. Unfortunately, this breaks with scoop as the shim file is
then not found.
2019-11-22 17:27:42 +00:00
Stefano Baghino
6d16409699 Move ReflectionIT out of LAPI IT (#3582)
* Move ReflectionIT out of LAPI IT

* Address https://github.com/digital-asset/daml/pull/3582#discussion_r349221189

* Address https://github.com/digital-asset/daml/pull/3582#discussion_r349507268

* Delete previous incarnation of the test
2019-11-22 16:30:56 +00:00
associahedron
367a1d8ceb Set Sdk-Version in DAR Manifest to compiler SDK version. (#3587)
* Set Sdk-Version in DAR Manifest to compiler SDK version.

CHANGELOG_BEGIN

- [DAML Compiler] Bugfix: The Sdk-Version field in a DAR manifest file
now matches the SDK version of the compiler, not the sdk-version field
from daml.yaml. These are usually the same, but they could be different
if you set the DAML_SDK_VERSION environment variable before running
``daml init`` or ``daml build``.

CHANGELOG_END

* Fallback to daml.yaml if env var not set

* add yaml dependency

* Always require sdk-version, and emit warning on mismatch with env var

* More explicit about where override comes from

* Add packaging regression test
2019-11-22 16:25:58 +00:00
Andreas Herrmann
02f806bf22 Manually run stack update (#3590)
* Add stack to dev-env

* Manually run stack update before build

To work around multiple concurrent attempts to take the
hackage-security-lock.
2019-11-22 15:16:48 +00:00
Andreas Herrmann
f4d0eb636e Replace Hazel by stack_snapshot (#2743)
* Unmangled libz.so and libbz2.so

* Use stack_snapshot instead of Hazel

* Remove Hazel

* Define stack_snapshot

* Update rules_haskell

* Document stack_snapshot

* Clean stack's lock file from aborted builds
2019-11-22 14:24:08 +00:00
Samir Talwar
0ff716df2a Ledger API: Add healthcheck endpoints. (#3573)
* grpc-definitions: Delete health_service.proto

We can use the version in io.grpc:grpc-services instead.

* ledger: Delete ledger/API.md.

* sandbox: Fix warnings in ApiServices flagged by IntelliJ.

* sandbox: Implement a dummy grpc.health.v1.Health.Check endpoint.

* sandbox: Implement a dummy grpc.health.v1.Health.Watch endpoint.

* sandbox: Drop repeated elements from grpc.health.v1.Health.Watch.

* sandbox: Wrap the HealthService in basic tests.

* sandbox: Stop streaming the server health too.

* ledger-api-test-tool: Health check tests.

* Add a changelog entry for the health check endpoints.

CHANGELOG_BEGIN

- [Ledger API] Add healthcheck endpoints, conforming to the
  `GRPC Health Checking Protocol <https://github.com/grpc/grpc/blob/master/doc/health-checking.md>`_.
  It is always ``SERVING`` for now.

- [DAML Ledger Integration Kit] Add conformance test coverage for the
  ``grpc.health.v1.Health`` service.

CHANGELOG_END

* ledger-api-integration-tests: Increment the number of services.

* Apply suggestions from code review

Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* sandbox: Use `AkkaBeforeAndAfterAll` in the HealthServiceSpec.

In an attempt to get it working on CI.

* sandbox: Change `dropRepeated` to `DropRepeated()`.

Keep it in one file.

* test-common: Use `Delayed.by` in `TimeBoundObserver`.

* test-common: Close the source when `TimeBoundObserver` completes.

* ./fmt.sh

That'll teach me not to `--no-verify` just because it's a merge commit.

* sandbox: Inline `HealthService.suppress`.

At some point it was being used twice.

* sandbox: Increase the timeout for HealthServiceSpec.

* sandbox: Reimplement HealthService using the Scala protobuf types.

* sandbox: Generate an Akka-compatible trait for the health service.

And refactor a lot of test code to make it easy to test.

* ledger-api-common: Move the HealthService here.

* rs-grpc-testing-utils: Publish to Maven.

* rs-grpc-testing-utils: Add Maven coordinates.
2019-11-22 14:02:05 +00:00
Oliver Seeliger
ad3adbb7aa
Ledger api test tool --load-scale-factor option to test different ledgers (#3586)
CHANGELOG_BEGIN

[DAML Integration Toolkit]
  * Add ledger api test tool `--load-scale-factor` option that allows dialing up
     or down the workload applied by scale tests (such as the
     `TransactionScaleIT` suite). This allows improving the performance of
     different ledger over time.

CHANGELOG_END
2019-11-22 14:51:17 +01:00
Remy
3f8bd0da5e DAML-LF: rename ValueMap to ValueTextMap. (#3584) 2019-11-22 11:35:32 +00:00
Robert Autenrieth
87c8f274e1
Split rw claims (#3548)
* Split read and write claims

Fixes #3398

CHANGELOG_BEGIN
- [Sandbox] Added support JWT tokens that only authorize to read data, but not to act on the ledger.
CHANGELOG_END
2019-11-22 11:24:50 +01:00
Remy
b506e27bde Java-codegen: add support for Generic Maps. (#3549)
* java-bindings: use LinkedHashMap asunderlying Map for DamlGenMap

* Address Samir's comments

* java-codegen: make DamlList, DamlMap and DamlGenMap immutable

* Address Samir's comments

* Adress Gerolf's comments

* more cleanup

* fix value generator

* Address Gerolf's comments:

CHANGELOG_BEGIN

[Java Bindings]
  * Rename ``DamlMap`` to ``DamlTextMap``
  * ``DamlCollectors`` class provides Collectors to build more easily ``DamlList`` and ``DamlTextMap``
  * Change the recommended method to convert ``DamlValue`` containers from/to Java Bindings containers.
    See `docs/source/app-dev/bindings-java/codegen.rst` for more details the new methodology.

CHANGELOG_END

* fix tests
2019-11-22 08:07:53 +00:00
Rohan Jacob-Rao
56820447d5 Fix link (#3579) 2019-11-21 18:54:50 +00:00
Stefano Baghino
9f9d2d0458 Move and improve LedgerIdentityService integration tests (#3581) 2019-11-21 17:08:10 +00:00