Commit Graph

2479 Commits

Author SHA1 Message Date
Remy
d465787dde Extractor: Add support for GenMap (#3614)
* Ledger-service: fix json encoding for genMap

* Extractor: Add support for GenMap

* Address Leo's comments
2019-11-28 12:31:28 +00:00
nickchapman-da
885bbefdf3 rename structural records: tuple -> struct (#3660)
* rename structural records: tuple -> struct

* add missing renames (tuple -> struct) in comments, var-names and error messages

* exposition: structural vs nominal; change history note

* remove accidentilly checked-in file
2019-11-28 10:00:24 +00:00
Stefano Baghino
7c3542fc1d Use scalafmt's diff mode (#3654)
Limits the set of files scalafmt will run on to those in the
git diff with master. This should greatly speed up language
agnostic checks, especially for developers to test them before
the contribution hits CI.

Worth mentioning: we use scalafmt 1.5.x, scalafmt latest release
is 2.2.x and the option is deprecated in favor of .
2019-11-28 09:33:13 +00:00
Stefano Baghino
c199c20730 Document HTTP JSON API authentication (#3657)
* Document HTTP JSON API authentication

* Update docs/source/json-api/index.rst

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* Update docs/source/json-api/index.rst

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* Address review https://github.com/digital-asset/daml/pull/3657#pullrequestreview-323840183

* Update docs/source/json-api/index.rst

Co-Authored-By: Stephen Compall <stephen.compall@daml.com>

* Update docs/source/json-api/index.rst

Co-Authored-By: Stephen Compall <stephen.compall@daml.com>
2019-11-28 09:25:22 +00:00
Martin Huschenbett
fbac97bda7 Remove the FooInstance rewriting hack used for generic templates (#3659)
This hack is not necessary anymore. A future addition of generic templates
must be significantly less hacky than this.
2019-11-28 08:50:40 +00:00
nickchapman-da
d3d6891021
Rename daml lf tuples (#3658)
* Rename DAML-LF tuples as structs (structural records)
2019-11-28 07:58:30 +00:00
Leonid Shlyapnikov
f9f9672fd5 ContractsService#search to return stream of ActiveContracts (#3625)
* Changing search so it returns `Source[ActiveContract]`

* wip

* function to convert Source[JsVal] into result Source[ByteString]

* compiles now

* cleanup

* tests passing

* test case for contracts search streaming error handling

* in memory search relying on acsFollowingAndBoundary, so we fetch all `following` updates

* Using ContractsFetch.acsFollowingAndBoundary

* Addressing code review comments, thanks @S11001001
2019-11-27 21:36:20 +00:00
Samir Talwar
ae9bb35b85 ledger-api-test-tool: Increase the duration when watching health. (#3645)
This should hopefully stop CI from flaking out.
2019-11-27 13:23:38 -05:00
Brian Healey
02186ef068 Ledger configuration indexing changes (rebased to master) (#3553)
* Add participant-state configuration protobuf

- Move the "DamlConfiguration" from kvutils into participant-state/protobuf/ledger_configuration.proto.
- Add version number and spec (ala transaction.proto)

This is a preparation for indexing the configuration and having one canonical serialization for it.

* Initial thoughts on indexing the configuration

* Implement indexing of ledger configuration changes

* Add record time to all Updates. Wire through participant id.

and rename V7__Add_configuration to V8.

* Add ledger_configuration_java_proto to release artifacts

* Fix up release of ledger_configuration_java_proto

* Suggestions from review

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* Apply suggestions from code review

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* address rebase issues

* Resolve compilation errors after rebase/merge

* happy formatting for scala and bazel and friends

* Drop "openWorld" setting from configuration

And refactor the tests to allocate parties explicitly.

* Fix up migration and tests

* Drop authorizedParticipantIds from configuration

Implement configuration authorization in kvutils using the previous
the participant id of the previous configuration.

* Post-rebase fixes

* Add missing migrations

* Apply suggestions from code review

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* Add missing mavenUpload to ledger-api-server

* Remove stateUpdateRecordTime

* Address code review

- Address PR review
- Merge TimeModelImpl and the traits. Remove TimeModel from ledger-api-common.
- Throw `Err` from KeyValueConsumption on parse errors instead of assert/sys.error

* Reformat

* Add missing protobuf file

* Fix compilation after TimeModel changes. Add version logs to participant-state{,-kvutils}.

* Fix TestUtil.scala build

* Apply suggestions from code review

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* Address review

- synchronized access to ledgerConfiguration in InMemoryLedger
- store rejection if configuration generation is wrong

* Update ledger/participant-state/protobuf/ledger_configuration.rst

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2019-11-27 17:41:23 +01:00
Martin Huschenbett
677b30687f Improve wording of daml2ts readme (#3652) 2019-11-27 16:16:10 +00:00
Stefano Baghino
a53caafc98
Address review comment (#3649)
Addresses https://github.com/digital-asset/daml/pull/3639/files#r350923206
2019-11-27 17:13:50 +01:00
Robin Krom
faa150b310
language: reenable migration checks (#3650)
This reenables the migration checks. The migration command now only
generates stumbs for conversions instead of using the formerly (generic)
upgrade template.
2019-11-27 17:10:54 +01:00
associahedron
aebf7452f6
Add generic maps to stdlib in DA.Map (#3651)
* Add generic maps to stdlib in DA.Map

* Update copyright headers

* Add doctests and fix memory tests
2019-11-27 14:29:06 +00:00
Remy
5b155409a0 DAML-LF iface reader: rename Map to TextMap (#3610)
CHANGELOG_BEGIN

 [DAM-LF Interface Reader]: **Rename** ``PrimTypeMap`` to ``PrimTypeTextMap`` and ``PrimType.Map`` to ``PrimType.TextMap``

CHANGELOG_END
2019-11-27 14:01:35 +00:00
Stefano Baghino
3482ffac19 Fix wrong token usage in auth tests (#3648) 2019-11-27 10:48:39 +00:00
associahedron
e8a5db5de3 Add Numeric to basic types table. (#3595) 2019-11-27 09:10:33 +00:00
Remy
997e206779 daml-lf: type synonyms are also taken into account for collisions (#3644) 2019-11-27 08:36:59 +00:00
Robin Krom
a2aacaa79b language: better error message for the script runner (#3637) 2019-11-26 21:27:35 +00:00
Moritz Kiefer
afe25febbf
Remove debug printf from daml script (#3640) 2019-11-26 20:45:33 +01:00
Rohan Jacob-Rao
094214aa55 Notify IDE users about new daml-driven blog posts (#3623)
* Successfully parse rss xml and find latest blog post object

* Show notification with link to blog (on every ide startup...)

* Only show blog post if not seen before

* Formatting

CHANGELOG_BEGIN

- [SDK] Notify IDE users about new daml-driven blog posts

CHANGELOG_END

* Add setting for user to opt out of notifications

* Use await rather than then

* Catch and swallow exceptions

* Another await and minor cleanup
2019-11-26 19:34:56 +00:00
Stefano Baghino
4d922e763b Add authentication to Java identity client (#3630)
* Add authentication to Java identity client

Contributes to #3626

No changelog added right now, it will be added with the PR that
effectively closes the ticket.

Moves `LedgerCallCredentials` to a shared package.

Establishes the test framework for other Java clients.

* Address https://github.com/digital-asset/daml/pull/3630#discussion_r350663731

* Address https://github.com/digital-asset/daml/pull/3630#discussion_r350663731

* Rebase and fix compilation errors

* Fix type of jar in artifacts.yaml

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* Fix more errors emerged after rebase

* Add dependency for DAML assistant ITs

* Add missing dependency

* Fix wrong dependency

* fix release
2019-11-26 18:51:09 +00:00
Stefano Baghino
d951d738ee
Add support for authenticated ledger (#3639)
Closes #3627

- Pass a token at startup to read packages
- Re-read the token every time new packages are fetched to ensure token
freshness
- If the token is expired, new packages will not be retrieved

CHANGELOG_BEGIN

- [JSON API] Accept a path to a file containing a token at startup for
package retrieval. See `issue #3627 <https://github.com/digital-asset/daml/issues/3627>`__.

CHANGELOG_END
2019-11-26 19:45:47 +01:00
Robin Krom
d90eb357ad
language: upgrades: prefix stdlib imports everywhere. (#3633) 2019-11-26 19:28:11 +01:00
Gary Verhaegen
b72fc19b6e
add comments to unreleased.sh (#3575) 2019-11-26 18:27:35 +00:00
Martin Huschenbett
09a681d4f3 Add a readme for daml2ts (#3638) 2019-11-26 17:49:40 +00:00
Moritz Kiefer
f0a111ca1a
Get grpc from nix on unix (#3632)
* Get grpc from nix on unix

The one from Bazel seems to cause linking issues when trying to run
things in GHCi. I’ve spent some time trying to use rules_foreign_cc to
build gRPC using CMake but decided that for my own sanity it’s better
to not pursue that further.

* Address review comments

* Add missing module load

* Cleanup GHCI_SCRIPT

* use the correct file ending on macos

* Import is_linux

* Switch back to grpc-1.23

The newer version seems to cause issues in combination with the java libraries.

* Try to fix package_app on macos

* more debugging

* Maybe this is not necessary, we will never know

* linkers are the worst

* Remove debugging output again

* readd rpaths

* treat libdispatch specially

* remove hack

* more fooling around

* lalala
2019-11-26 18:47:39 +01:00
Moritz Kiefer
41958b1430 Fix typo in synonym spec (#3636) 2019-11-26 17:15:58 +01:00
Martin Huschenbett
167232bd81
Ban usage of generic templates (#3631)
We don't drop full support from the compiler yet but rather ban their use by
adding a check to the preprocessor. We'll remove the actual support as we go
along with fixing the upgrading story.

CHANGELOG_BEGIN

- [DAML Compiler] Make the experimental feature "generic templates"
  unavailable. The current implementation is at odds with other, more important
  language features still under development.

CHANGELOG_END
2019-11-26 17:15:17 +01:00
Moritz Kiefer
f402acf19b Add specification for type synonyms in DAML-LF (#3635)
* Add specification for type synonyms in DAML-LF

* Fix identifiers for type synonyms

* Use S for type synonyms

* Address review comments

* Split up TyTypeSynonym and DefTypeSynonym
2019-11-26 15:01:44 +00:00
Remy
06b7a106a1 DAML-LF: generic map values protobuf format (#3583)
* DAML-LF: generic map values protobuf format

* Apply suggestions from code review

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* Address Gerolf's comment

* Apply suggestions from code review

Co-Authored-By: Stephen Compall <stephen.compall@daml.com>

* fix test

* Address Stephen's last comment
2019-11-26 13:34:33 +00:00
Stefano Baghino
3d3322a06a
Fix wording in authorization tests (#3629)
Address previously unaddressed https://github.com/digital-asset/daml/pull/3598#discussion_r350581428
2019-11-26 11:12:30 +01:00
Stefano Baghino
160da16227
Move AuthorizationIT to sandbox integration tests (#3598)
* Move AuthorizationIT to Sandbox ITs

* Use refactored code in Extractor

* Apply a few improvements, see body

- Run previously ignored tests
- Make token signing private
- Fix stream observer utility comment more accurate
- Add read-only token helper

* Add r/w token split tests

* Fix formatting and imports

* Address https://github.com/digital-asset/daml/pull/3598#discussion_r349769546

* Address https://github.com/digital-asset/daml/pull/3598#discussion_r349769801

* Fix inaccurate test

A test was incorrectly trying to get a token with an invalid signature

* Address reviews
2019-11-26 10:13:36 +01:00
Martin Huschenbett
e9be9419f8
Remove documentation for generic templates (#3619)
Unfortunately, our current implementation of generic templates is at odds
with cross-SDK imports of DARs and hence the whole upgrading story for
DAML models, particularly around type classes, which generic templates
heavily rely upon. Thus, we've decided to drop support for generic templates
for now.

This PR does the first step by dropping the documentation. I've deliberately
not added a changelog entry here. I'll do that once we have dropped support
from the compiler.
2019-11-26 10:10:13 +01:00
Andreas Herrmann
15f12eea6a Remove Bazel override (#3603)
* Remove Bazel override

The extended default shell env is no longer required. See
https://github.com/digital-asset/daml/pull/3597#pullrequestreview-322096242

* Import diff from rules_sh posix toolchain
2019-11-26 09:06:44 +00:00
Samir Talwar
8818ea7cbb kvutils: Clean up the InMemoryKVParticipantState integration tests a little. (#3621)
* kvutils: Nest all InMemoryKVParticipantState tests inside the block.

* kvutils: Simplify InMemoryKVParticipantStateIT where possible.

* kvutils: In InMemoryKVParticipantStateIT, move assertions to the end.

It's confusing when the assertions are before the submissions.

* kvutils: Move test helpers into a companion object.

* kvutils: Use BeforeAndAfterEach in testing to remove duplication.

* kuvtils: In testing config updates, always increment the generation.

* kvutils: Assert something in the "initial conditions" integration test.
2019-11-25 17:49:56 -05:00
Stefano Baghino
25f65f746b
Move DamlLedgerClient to builder-based constructors (#3620)
With the intent of adding more functionality to the Java bindings, in
particular authentication, this PR deprecates existing constructors in
favor of a more flexible builder based approach.

CHANGELOG_BEGIN

- [Java Bindings] Deprecated existing constructors for
``DamlLedgerClient``, please use the static ``newBuilder`` method to
instantiate a builder and use it to create the client, starting from
either a ``NettyChannelBuilder`` or a plain host/port pair.

CHANGELOG_END
2019-11-25 23:19:14 +01:00
Moritz Kiefer
770a9ae5e4 Support empty packages in daml build (#3624)
We still error out if the directory in `src` doesn’t exist to catch
typos.
2019-11-25 20:44:34 +00:00
nickchapman-da
0f0a12db22 Sdk access token file (#3607)
* support --access-token-file in SDK

* rename: HostAndPort -> LedgerArgs; HostAndPortFlags -> LedgerFlags

* integration test for --access-token-file

* doc & more help

* address comments: use FilePath

* adapt to renamed arg: --auth-jwt-hs256-unsafe

* avoid use of single quotes on command line
2019-11-25 17:38:41 +00:00
Remy
3fa6043056 fix typo in java codegen doc (#3618)
Thanks Miklos !
2019-11-25 16:34:14 +00:00
Samir Talwar
8df4b9ec33 ledger-api-test-tool: Use the default Akka execution context. (#3608)
* ledger-api-test-tool: Use the default Akka execution context.

Akka will use the global fork-join pool by default, which is fine for
our purposes. There's no longer a need to use a specific execution
context, because we no longer use it to limit the work in progress.
That's now done by the parallelism parameter to Akka Streams'
`mapAsyncUnordered`.

* ledger-api-test-tool: Shut down the ActorSystem.
2019-11-25 15:57:17 +00:00
Samir Talwar
603ee9367b ledger-api-test-tool: The endpoints are mandatory. (#3611)
We should fail fast if no endpoints are provided, rather than timing out
every test.
2019-11-25 15:32:35 +00:00
Ognjen Maric
48174c8ee4
Fixed a stale reference to a MustPaint contract (#3613) 2019-11-25 16:29:32 +01:00
Robert Autenrieth
4c3f6014f6
Implement support for RSA-signed JWT tokens (#3526)
* Improve Navigator output

* Fix Navigator not using the access token

* Add RSA signatures for JWT tokens

* Remove unused method

* Add timeouts for reading JWKS

* Fix test

* Rename method for consistency

* Improve comment

* More renaming for consistency

* CHANGELOG

CHANGELOG_BEGIN
- [Sandbox] Add CLI options to start the sandbox with JWT based authentication with RSA signed tokens.
  See `issue #3155 <https://github.com/digital-asset/daml/issues/3155>`__ .
- [Navigator] Fixed a bug where the `--access-token-file` option did not work correctly.
CHANGELOG_END

* Make JwksVerifier limits configurable

* Make SimpleHttpServer private
2019-11-25 16:29:24 +01:00
Stefano Baghino
552bf6497d Obscure shared-secret-based authenticated sandbox (#3609)
The shared-secret-based authentication is there exclusively for testing,
this PR makes sure that this is correctly reported through the CLI and
hides this until further notice.

Makes sure that the flag clearly says that this option is not meant for
production use cases.

A warning is already printed when this feature is used, both at startup
and every time this verifier is used.

CHANGELOG_BEGIN

- [Sandbox] The ``--auth-jwt-hs256`` is renamed to
``--auth-jwt-hs256-unsafe``: you are advised to _not_ use this JWT token
signing way in a production environment.

CHANGELOG_END
2019-11-25 14:27:21 +00:00
Robert Autenrieth
243ca0f10a Print usage for the unreleased.sh script (#3585) 2019-11-25 13:26:08 +00:00
Remy
d152c7cbfd daml-lf: rename Map to TextMap in archive proto (#3589)
* daml-lf: rename Map to TextMap in archive proto
+ in Scala/haskell AST

* a bit more renamming

* Update compiler/daml-lf-tools/src/DA/Daml/LF/TypeChecker/Serializability.hs

Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>

* fix test

* Apply suggestions from code review

Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>

* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Base.hs

Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
2019-11-25 12:14:57 +00:00
Moritz Kiefer
27051160eb
Support multi-participant DAML script (#3605)
* Support multi-participant DAML script

fixes #3555

CHANGELOG_BEGIN

- [DAML Script - Experimental] DAML script can now run be used in distributed topologies.

CHANGELOG_END

* Fix ports in multiparticipants tests
2019-11-25 12:43:47 +01:00
Oliver Seeliger
26b569f73d Properly drop invalid actor name chars in standalone indexer actor name (#3604)
Closes #3327

CHANGELOG_BEGIN

[Ledger]
  * Allow non-alphanumeric characters in ledger api server participant ids (space, colon, hash, slash, dot). Proper fix for change originally attempted in v0.13.36. See issue `issue #3327 <https://github.com/digital-asset/daml/issues/3327>`__.

CHANGELOG_END

Proper fix replacing f39bf7fc5a which does not work
2019-11-25 11:16:59 +00:00
Samir Talwar
159104adb1 ledger-api-test-tool: Don't discard the cause on fail. (#3600)
* ledger-api-test-tool: On `fail`, don't discard the cause.

* ledger-api-test-tool: Delete a redundant `new`.
2019-11-25 10:43:01 +00:00
Oliver Seeliger
d35961d857
Ledger api test tool canton run config fix (#3602)
Missing from 15a7f72eae
2019-11-25 11:13:27 +01:00