Commit Graph

2497 Commits

Author SHA1 Message Date
Martin Huschenbett
ea89dbb122 Don't make UndecidableSuperClasses a default extension for DAML (#3673)
We needed this for our generic template hacks. Since they are gone now,
I'm very much in favour of removing this extension since "undecidable"
always sounds a bit scary, even if it is only at compile time.

CHANGELOG_BEGIN

- [DAML Compiler] Don't make `UndecidableSuperClasses` a default language
  extension for DAML anymore. If you really need this feature for a module,
  you can reenable it using a `LANGUAGE` pragma at the top.

CHANGELOG_END
2019-11-29 15:51:40 +00:00
Moritz Kiefer
75c9b1bf91
Simplify template desugaring (#3670)
* Introduce a simpler template desugaring without support for generic templates

This adapts the LF conversion to the new template desugaring
introduced in our GHC fork. The guiding principle is that we use the
typeclasses directly to avoid generating, typechecking and converting
redundant code caused by indirections. I updated the template
desugaring documentation so that is probably a good starting point for
reviewing this.

* Address review comments

* Fix daml doc tests

* Fix data dependency tests

* Switch to new ghc-lib release
2019-11-29 16:13:15 +01:00
Samir Talwar
5dd38d54e8 sandbox: PostgreSQL health checks. (#3655)
* ledger-api-test-tool: Increase the duration when watching health.

This should hopefully stop CI from flaking out.

* reference-v2/sandbox: Avoid unnecessary companion object constructors.

I like indirection… when it does something.

* ledger: Propagate empty health checks throughout the services.

* reference: Remove duplication from the ReferenceServer object.

* ledger-api-common: Actually query a "reporter" in the health service.

* ledger-api-common: Report health per-component when required.

* ledger-api-health: Use a Map to represent components for health checks.

* sandbox: Fix warnings in SqlLedgerSpec.

* ledger-api-common: Throw GrpcHealthService errors inside the Future.

* ledger: Implement health checks against the PostgreSQL connection.

Without proper testing, because I am not great at this.

* sandbox: Remove duplication and fix warnings in PostgresAround.

* sandbox: Test the SQL Ledger's health reporting on failure.

* sandbox: Don't report as unhealthy until 3 connections fail.

* ledger-api-health: Remove unused parts of the API.

Bit of premature design there.

* sandbox: Rename the "ledger" health check to "write".

* participant-state: Add the ReportsHealth trait to ReadService.

* ledger-api-common: `Future.fromTry(Try(…))` -> `Future(…)`.

* ledger-api-common: Make it clearer that StubReporter closes over health.

* ledger-api-common: Explain the HealthService watch tests with comments.

* sandbox: Clean up SqlLedger a bit.

* sandbox: Don't try and stop PostgreSQL twice in PostgresAround.

* bazel_tools: Windows rlocation lookups need to be with forward slashes.

* release: Fix case of "true".

* ledger-api-common: Make `GrpcHealthService::matchResponse` return a Try.

* ledger-api-common: Make `GrpcHealthServiceSpec` async.

* sandbox: Make a couple of DB classes final.

* sandbox: Avoid importing `X._` in PostgresAround.

* sandbox: Add clues to the SqlLedgerSpec's multiple assertions.

* sandbox: If PostgreSQL doesn't come back up, keep retrying.

* sandbox: Remove duplication in SqlLedgerSpec.

* sandbox: In SqlLedgerSpec, actually wait for the health to change.

* sandbox: In PostgresAround, make stopping PostgreSQL idempotent.

* sandbox: Simplify the SqlLedgerSpec to make it work on CI.

It's worth a shot.

* ledger-api-common: Simplify the GrpcHealthServiceSpec a little.

And add a changelog.

CHANGELOG_BEGIN

- [Ledger API Server] Add a health check endpoint conforming to the
  `GRPC Health Checking Protocol <https://github.com/grpc/grpc/blob/master/doc/health-checking.md>`_.
- [Ledger API Server] Add health checks for index database connectivity.
- [Participant State API] Add a mandatory ``currentHealth()`` method to
  ``IndexService``, ``ReadService`` and ``WriteService``.

CHANGELOG_END

* sandbox: Improve the Javadoc layout for DbDispatcher.

* sandbox: Capitalize constants in SqlExecutor.

* ledger-api-health: Convert HealthStatus to an abstract class.
2019-11-29 15:07:43 +00:00
associahedron
2713d6978d
Reduce SDK tarball size by 70% by deduplicating Scala dependencies. (#3678)
* Start moving all the jars into a single jar.

* Fold navigator into daml-sdk jar

* include sandbox

* Remove unnecessary compileDeps.

CHANGELOG_BEGIN

- [DAML SDK] Reduced the size of the DAML SDK by about
60% uncompressed, 70% compressed, by deduplicating Scala
dependencies.

CHANGELOG_END

* update copyright header

* buildifier fix
2019-11-29 13:26:47 +00:00
Martin Huschenbett
9f18a83570
Fix error message for missing --ledger-party in daml trigger (#3677)
Who is "Ledger Pary"?
2019-11-29 12:31:19 +01:00
Martin Huschenbett
952184298e Release DAML SDK 0.13.38 (#3676) 2019-11-29 12:16:20 +01:00
Oliver Seeliger
ef08936c05 Ledger api CommandService conformance test fixes (#3675)
Adding missing synchronize to two tests
2019-11-29 09:57:14 +00:00
Oliver Seeliger
f11a3ce9a7
Release triggers to maven central (#3674) 2019-11-29 10:25:02 +01:00
Martin Huschenbett
0e8c2bbc6f Improve the error message for polymorphic numeric literals (#3672)
Display the actual number the user tried to use instead of 1.2345.

The logic to display the number is not perfect but definitely better than
we have now. If somebody knows a _simple_ way to do this better, please
tell me about it.
2019-11-29 08:51:32 +00:00
associahedron
aafa107a80
Update DataDependencies.hs for GenMap (#3671) 2019-11-29 08:29:48 +00:00
Martin Huschenbett
bf5310060c Don't break long party names in scenario view weirdly (#3669)
Until now, we broke long party names in the table view in the scenario at
spaces. This looked quite ugly.

Now, we don't break them at spaces. At the same time we also moved the CSS
into a file rather than having it in the code to make changes like this one
easier in the future.

This fixes #3642.
2019-11-29 08:10:44 +00:00
Stefano Baghino
0123c659f2 Add authentication support to CommandClient (#3666) 2019-11-29 07:20:39 +00:00
associahedron
405ae5a8d8 Implement DAML constraint tuples as LF structs. (#3667)
* Constraint tuples

* LF Tuples -> Structs

* A couple that got away

* Dead code elim

* Better documentation for constraint tuple projection functions

* strict -> struct

* readability on constraint tuple projection conversion

* Add mkIndexedField
2019-11-28 17:18:55 +00:00
Remy
5e63db7a40 speedy: Rename SMap to STextMap (#3664)
+ rename MAP to TEXTMAP in decoder/encoder
2019-11-28 17:05:26 +00:00
Samir Talwar
326f427e8f scala-protoc-plugins: Don't use two versions of scalapb. (#3665)
This really confuses IntelliJ, because it's importing two versions of
scala-library-*.jar.

Turns out it's not being used anyway.
2019-11-28 16:26:59 +00:00
Remy
54d98fd186 navigator: add GenMap support (#3525)
* navigator: add GenMap support

* fix after rebase

* change missing 'map' to 'textmap'

* more fixes

* navigator: fix query for TextMap and GenMap
2019-11-28 16:13:36 +00:00
Moritz Kiefer
8b289eb080
Document how to iterate on changes to template desugaring (#3663) 2019-11-28 16:33:24 +01:00
Stefano Baghino
9e9eb68e38 Make ActiveContractsClient work against auth ledgers (#3653)
* Make ActiveContractsClient work against auth ledgers

* Address review comments

- Address https://github.com/digital-asset/daml/pull/3653#discussion_r351667485
- Address https://github.com/digital-asset/daml/pull/3653#discussion_r351667588
2019-11-28 12:36:50 +00:00
Remy
d465787dde Extractor: Add support for GenMap (#3614)
* Ledger-service: fix json encoding for genMap

* Extractor: Add support for GenMap

* Address Leo's comments
2019-11-28 12:31:28 +00:00
nickchapman-da
885bbefdf3 rename structural records: tuple -> struct (#3660)
* rename structural records: tuple -> struct

* add missing renames (tuple -> struct) in comments, var-names and error messages

* exposition: structural vs nominal; change history note

* remove accidentilly checked-in file
2019-11-28 10:00:24 +00:00
Stefano Baghino
7c3542fc1d Use scalafmt's diff mode (#3654)
Limits the set of files scalafmt will run on to those in the
git diff with master. This should greatly speed up language
agnostic checks, especially for developers to test them before
the contribution hits CI.

Worth mentioning: we use scalafmt 1.5.x, scalafmt latest release
is 2.2.x and the option is deprecated in favor of .
2019-11-28 09:33:13 +00:00
Stefano Baghino
c199c20730 Document HTTP JSON API authentication (#3657)
* Document HTTP JSON API authentication

* Update docs/source/json-api/index.rst

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* Update docs/source/json-api/index.rst

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* Address review https://github.com/digital-asset/daml/pull/3657#pullrequestreview-323840183

* Update docs/source/json-api/index.rst

Co-Authored-By: Stephen Compall <stephen.compall@daml.com>

* Update docs/source/json-api/index.rst

Co-Authored-By: Stephen Compall <stephen.compall@daml.com>
2019-11-28 09:25:22 +00:00
Martin Huschenbett
fbac97bda7 Remove the FooInstance rewriting hack used for generic templates (#3659)
This hack is not necessary anymore. A future addition of generic templates
must be significantly less hacky than this.
2019-11-28 08:50:40 +00:00
nickchapman-da
d3d6891021
Rename daml lf tuples (#3658)
* Rename DAML-LF tuples as structs (structural records)
2019-11-28 07:58:30 +00:00
Leonid Shlyapnikov
f9f9672fd5 ContractsService#search to return stream of ActiveContracts (#3625)
* Changing search so it returns `Source[ActiveContract]`

* wip

* function to convert Source[JsVal] into result Source[ByteString]

* compiles now

* cleanup

* tests passing

* test case for contracts search streaming error handling

* in memory search relying on acsFollowingAndBoundary, so we fetch all `following` updates

* Using ContractsFetch.acsFollowingAndBoundary

* Addressing code review comments, thanks @S11001001
2019-11-27 21:36:20 +00:00
Samir Talwar
ae9bb35b85 ledger-api-test-tool: Increase the duration when watching health. (#3645)
This should hopefully stop CI from flaking out.
2019-11-27 13:23:38 -05:00
Brian Healey
02186ef068 Ledger configuration indexing changes (rebased to master) (#3553)
* Add participant-state configuration protobuf

- Move the "DamlConfiguration" from kvutils into participant-state/protobuf/ledger_configuration.proto.
- Add version number and spec (ala transaction.proto)

This is a preparation for indexing the configuration and having one canonical serialization for it.

* Initial thoughts on indexing the configuration

* Implement indexing of ledger configuration changes

* Add record time to all Updates. Wire through participant id.

and rename V7__Add_configuration to V8.

* Add ledger_configuration_java_proto to release artifacts

* Fix up release of ledger_configuration_java_proto

* Suggestions from review

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* Apply suggestions from code review

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* address rebase issues

* Resolve compilation errors after rebase/merge

* happy formatting for scala and bazel and friends

* Drop "openWorld" setting from configuration

And refactor the tests to allocate parties explicitly.

* Fix up migration and tests

* Drop authorizedParticipantIds from configuration

Implement configuration authorization in kvutils using the previous
the participant id of the previous configuration.

* Post-rebase fixes

* Add missing migrations

* Apply suggestions from code review

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* Add missing mavenUpload to ledger-api-server

* Remove stateUpdateRecordTime

* Address code review

- Address PR review
- Merge TimeModelImpl and the traits. Remove TimeModel from ledger-api-common.
- Throw `Err` from KeyValueConsumption on parse errors instead of assert/sys.error

* Reformat

* Add missing protobuf file

* Fix compilation after TimeModel changes. Add version logs to participant-state{,-kvutils}.

* Fix TestUtil.scala build

* Apply suggestions from code review

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* Address review

- synchronized access to ledgerConfiguration in InMemoryLedger
- store rejection if configuration generation is wrong

* Update ledger/participant-state/protobuf/ledger_configuration.rst

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2019-11-27 17:41:23 +01:00
Martin Huschenbett
677b30687f Improve wording of daml2ts readme (#3652) 2019-11-27 16:16:10 +00:00
Stefano Baghino
a53caafc98
Address review comment (#3649)
Addresses https://github.com/digital-asset/daml/pull/3639/files#r350923206
2019-11-27 17:13:50 +01:00
Robin Krom
faa150b310
language: reenable migration checks (#3650)
This reenables the migration checks. The migration command now only
generates stumbs for conversions instead of using the formerly (generic)
upgrade template.
2019-11-27 17:10:54 +01:00
associahedron
aebf7452f6
Add generic maps to stdlib in DA.Map (#3651)
* Add generic maps to stdlib in DA.Map

* Update copyright headers

* Add doctests and fix memory tests
2019-11-27 14:29:06 +00:00
Remy
5b155409a0 DAML-LF iface reader: rename Map to TextMap (#3610)
CHANGELOG_BEGIN

 [DAM-LF Interface Reader]: **Rename** ``PrimTypeMap`` to ``PrimTypeTextMap`` and ``PrimType.Map`` to ``PrimType.TextMap``

CHANGELOG_END
2019-11-27 14:01:35 +00:00
Stefano Baghino
3482ffac19 Fix wrong token usage in auth tests (#3648) 2019-11-27 10:48:39 +00:00
associahedron
e8a5db5de3 Add Numeric to basic types table. (#3595) 2019-11-27 09:10:33 +00:00
Remy
997e206779 daml-lf: type synonyms are also taken into account for collisions (#3644) 2019-11-27 08:36:59 +00:00
Robin Krom
a2aacaa79b language: better error message for the script runner (#3637) 2019-11-26 21:27:35 +00:00
Moritz Kiefer
afe25febbf
Remove debug printf from daml script (#3640) 2019-11-26 20:45:33 +01:00
Rohan Jacob-Rao
094214aa55 Notify IDE users about new daml-driven blog posts (#3623)
* Successfully parse rss xml and find latest blog post object

* Show notification with link to blog (on every ide startup...)

* Only show blog post if not seen before

* Formatting

CHANGELOG_BEGIN

- [SDK] Notify IDE users about new daml-driven blog posts

CHANGELOG_END

* Add setting for user to opt out of notifications

* Use await rather than then

* Catch and swallow exceptions

* Another await and minor cleanup
2019-11-26 19:34:56 +00:00
Stefano Baghino
4d922e763b Add authentication to Java identity client (#3630)
* Add authentication to Java identity client

Contributes to #3626

No changelog added right now, it will be added with the PR that
effectively closes the ticket.

Moves `LedgerCallCredentials` to a shared package.

Establishes the test framework for other Java clients.

* Address https://github.com/digital-asset/daml/pull/3630#discussion_r350663731

* Address https://github.com/digital-asset/daml/pull/3630#discussion_r350663731

* Rebase and fix compilation errors

* Fix type of jar in artifacts.yaml

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* Fix more errors emerged after rebase

* Add dependency for DAML assistant ITs

* Add missing dependency

* Fix wrong dependency

* fix release
2019-11-26 18:51:09 +00:00
Stefano Baghino
d951d738ee
Add support for authenticated ledger (#3639)
Closes #3627

- Pass a token at startup to read packages
- Re-read the token every time new packages are fetched to ensure token
freshness
- If the token is expired, new packages will not be retrieved

CHANGELOG_BEGIN

- [JSON API] Accept a path to a file containing a token at startup for
package retrieval. See `issue #3627 <https://github.com/digital-asset/daml/issues/3627>`__.

CHANGELOG_END
2019-11-26 19:45:47 +01:00
Robin Krom
d90eb357ad
language: upgrades: prefix stdlib imports everywhere. (#3633) 2019-11-26 19:28:11 +01:00
Gary Verhaegen
b72fc19b6e
add comments to unreleased.sh (#3575) 2019-11-26 18:27:35 +00:00
Martin Huschenbett
09a681d4f3 Add a readme for daml2ts (#3638) 2019-11-26 17:49:40 +00:00
Moritz Kiefer
f0a111ca1a
Get grpc from nix on unix (#3632)
* Get grpc from nix on unix

The one from Bazel seems to cause linking issues when trying to run
things in GHCi. I’ve spent some time trying to use rules_foreign_cc to
build gRPC using CMake but decided that for my own sanity it’s better
to not pursue that further.

* Address review comments

* Add missing module load

* Cleanup GHCI_SCRIPT

* use the correct file ending on macos

* Import is_linux

* Switch back to grpc-1.23

The newer version seems to cause issues in combination with the java libraries.

* Try to fix package_app on macos

* more debugging

* Maybe this is not necessary, we will never know

* linkers are the worst

* Remove debugging output again

* readd rpaths

* treat libdispatch specially

* remove hack

* more fooling around

* lalala
2019-11-26 18:47:39 +01:00
Moritz Kiefer
41958b1430 Fix typo in synonym spec (#3636) 2019-11-26 17:15:58 +01:00
Martin Huschenbett
167232bd81
Ban usage of generic templates (#3631)
We don't drop full support from the compiler yet but rather ban their use by
adding a check to the preprocessor. We'll remove the actual support as we go
along with fixing the upgrading story.

CHANGELOG_BEGIN

- [DAML Compiler] Make the experimental feature "generic templates"
  unavailable. The current implementation is at odds with other, more important
  language features still under development.

CHANGELOG_END
2019-11-26 17:15:17 +01:00
Moritz Kiefer
f402acf19b Add specification for type synonyms in DAML-LF (#3635)
* Add specification for type synonyms in DAML-LF

* Fix identifiers for type synonyms

* Use S for type synonyms

* Address review comments

* Split up TyTypeSynonym and DefTypeSynonym
2019-11-26 15:01:44 +00:00
Remy
06b7a106a1 DAML-LF: generic map values protobuf format (#3583)
* DAML-LF: generic map values protobuf format

* Apply suggestions from code review

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* Address Gerolf's comment

* Apply suggestions from code review

Co-Authored-By: Stephen Compall <stephen.compall@daml.com>

* fix test

* Address Stephen's last comment
2019-11-26 13:34:33 +00:00
Stefano Baghino
3d3322a06a
Fix wording in authorization tests (#3629)
Address previously unaddressed https://github.com/digital-asset/daml/pull/3598#discussion_r350581428
2019-11-26 11:12:30 +01:00
Stefano Baghino
160da16227
Move AuthorizationIT to sandbox integration tests (#3598)
* Move AuthorizationIT to Sandbox ITs

* Use refactored code in Extractor

* Apply a few improvements, see body

- Run previously ignored tests
- Make token signing private
- Fix stream observer utility comment more accurate
- Add read-only token helper

* Add r/w token split tests

* Fix formatting and imports

* Address https://github.com/digital-asset/daml/pull/3598#discussion_r349769546

* Address https://github.com/digital-asset/daml/pull/3598#discussion_r349769801

* Fix inaccurate test

A test was incorrectly trying to get a token with an invalid signature

* Address reviews
2019-11-26 10:13:36 +01:00