Commit Graph

2591 Commits

Author SHA1 Message Date
Moritz Kiefer
6716947931 Bump ghcide (#3800) 2019-12-10 13:05:20 +00:00
Remy
a309b4a50b
DAML-LF: spec cleanup (#3801) 2019-12-10 13:38:55 +01:00
Remy
5f0976343c
DAML-LF: fix subsitution bug (#3798)
* damlc lf typechecker: expose subtitution bug

* Fix substitution bug

* Rename variables

* fix subsitution bug in scala side

* reactive some test

* Update daml-lf/validation/src/main/scala/com/digitalasset/daml/lf/validation/TypeSubst.scala

Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>

* Update daml-lf/validation/src/main/scala/com/digitalasset/daml/lf/validation/TypeSubst.scala

Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
2019-12-10 13:36:55 +01:00
Martin Huschenbett
0b63fa432b
daml2ts: Add instructions on how to develop (#3799) 2019-12-10 12:52:03 +01:00
Remy
0107228c49
daml-lf: distribute daml_lf_1.7_archive_java_proto (#3792) 2019-12-10 12:24:55 +01:00
mziolekda
3c48bd37eb
Clean up kvutils logs (#3780)
* clean up

* clean up config committer

* remove unnecessary constructor parameters in the ConfigCommitter

* address review comments
2019-12-10 11:12:19 +01:00
Moritz Kiefer
9510a5a382 Release SDK 0.13.40 (#3797)
* Release SDK 0.13.40

* rst is stupid
2019-12-10 09:56:21 +00:00
Moritz Kiefer
e769264ddf Move all datatypes out of daml-prim (#3791)
* Move all datatypes out of daml-prim

This moves the remaining two modules DA.Types and GHC.Tuple to
separate LF packages with stable identifiers.

The only data types remaining are the ones for typeclasses which will
disappear once we move this to type synonyms.

CHANGELOG_BEGIN

- [DAML Compiler] The modules DA.Types and GHC.Tuple from daml-prim
have been moved to separate packages.

CHANGELOG_END

* Fix codegen tests

* Fix DarReader test

* Fix kvutils tests

* Fix jdbcdao tests

* Fix hs ledger bindings tests
2019-12-10 09:19:16 +01:00
Rohan Jacob-Rao
37fa12c943
Remove blog post notifications (#3794)
* Disable IDE notifications about new blog posts

* Remove notification setting
2019-12-09 15:41:56 -05:00
Stefano Baghino
6e17cdba90
Fix typo in JavaDoc (#3793) 2019-12-09 17:58:33 +01:00
Robert Autenrieth
bf2098f038
Check ledger and participant ID in claims (#3781)
* Add ledger and participant ID to claims

CHANGELOG_BEGIN
- [Ledger] AuthService implementations can now restrict the validity of access tokens to a single ledger or participant.
- [Sandbox] The sandbox JWT authentication now respects the ledgerId and participantId fields of the token payload.

CHANGELOG_END

* Add tests for ledger and participant in claims

* Address review comment

* Address review comment

* Fix tests

* Fix tests
2019-12-09 17:55:17 +01:00
associahedron
c97a5e7aaf Workaround and test for #3777 (#3790) 2019-12-09 16:50:54 +00:00
associahedron
398a1237ca Perform name collision check against other names in package. (#3770)
* Full package name collision check

* Handle type synonyms appropriately

* Better comment

* Make isAscendant case-insensitive

* Document isAscendant and explain case-insensitivity

* Add a package-wide name collision test.
2019-12-09 16:36:55 +00:00
Samir Talwar
639eba2c17 ledger-api-scala-logging: Fix errors in IntelliJ IDEA. (#3785)
* ledger-api-scala-logging: Fix errors in IntelliJ IDEA.

The Bazel plugin for IntelliJ doesn't seem to be smart enough to be able
to handle a Scala library that is part `src` directory and part
generated code from another Bazel rule. It just ignores the second part.
This means that IntelliJ cannot find the *ServiceLogging classes, as
they're not represented on the Bazel-generated classpath, and so
complains with lots of errors when working on the equivalent Api*Service
files.

To fix this, we can split these in two, compiling the base traits to
`ledger-api-scala-logging-base` and then the generated code separately.

It does result in an extra Bazel dependency for the users of
ledger-api-scala-logging, as Bazel doesn't realise transitive
dependencies for us.

* Release: Add `ledger-api-scala-logging-base` to the Maven list.
2019-12-09 16:02:59 +00:00
Stefano Baghino
18824016c1 Ensure the access token is initialized when constructing a client (#3788)
* Ensure the access token is initialized when constructing a client

CHANGELOG_BEGIN
- [Java Client] Ensure the access token is initialized when using a
deprecated constructor.
CHANGELOG_END

* Improve phrasing and grammar
2019-12-09 14:14:45 +00:00
Martin Huschenbett
0211e1337d Move the bazel aliases for yarn and java to the root BUILD file (#3786)
Replicating the yarn alias in multiple places doesn't make sense.
2019-12-09 13:22:21 +00:00
Brian Healey
23edf52d36
resolve jackson transitive clash (#3775) 2019-12-09 07:47:53 -05:00
Samir Talwar
1033e48cb4
Ledger: Documentation for health checks. (#3779)
* ledger: Document the health checks.

* sandbox: Build a Docker image.

* sandbox: Create a sample Kubernetes YAML file.

* sandbox: Add health probes to the sample Kubernetes configuration file.

Startup and liveness are tested with a simple TCP connection to port
6865. Readiness checks are done with `grpc-health-probe`, which is added
to the Sandbox container image.

* sandbox: Link to kubernetes.yaml in the README and provide a disclaimer.

// changelog removed as it's not actually relevant to users

* sandbox: Don't try and build `sandbox-image-base` on Windows.

* Apply suggestions from code review

Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2019-12-09 12:43:20 +01:00
Samir Talwar
1d1c7e9033 sandbox: Increase the timeout in StaticTimeIT. (#3774)
It's currently quite flaky in CI; I think increasing it a little will
help.

Also fixes some warnings in the same file.
2019-12-09 07:53:40 +00:00
Martin Huschenbett
19d2a22b0b
daml2ts: Add E2E tests (#3776)
* daml2ts: Add E2E tests

* Attempt to fix failing test

* Don't try to delete the temp directory

* Cleanup block can't be empty

* Leave TMP_DIR before removing it
2019-12-07 18:34:40 +01:00
Samir Talwar
732c675e76 Sandbox: Perform JDBC health checks on a timer. (#3773)
* sandbox: HikariJdbcConnectionProvider.start(), rather than construction.

It's too easy to construct things; I'd like it to be obvious that we're
starting something (in this case a connection pool).

* sandbox: In SqlLedgerSpec, stop ledgers before shutting down PostgreSQL.

* sandbox: Perform JDBC health checks on a timer.

The problem with hooking into existing requests is that if the Sandbox
is running in a load balancer and reports itself as unhealthy, it might
be taken out of the load balancer. Without any requests going through,
it'll never realize it's healthy again, and so will remain thinking it's
unhealthy forever.
2019-12-06 17:25:11 +00:00
Gerolf Seitz
9d926d81b0 RxJava Bindings: Allow bot to run on a scheduler (#3404)
* RxJava Bindings: Allow bot to run on a scheduler

It seems that having many bots results in a some sort of deadlock
or blocking of data flow within the flowable network.

Adding some async boundaries to allow for concurrent processing
seems to help.

Fixes #2356

CHANGELOG_BEGIN
- [RxJava Bindings] Added a method to the ``Bot`` class allowing users to specify a ``Scheduler`` to use for running the bot. See `issue #2356 <https://github.com/digital-asset/daml/issues/2356>`__.
CHANGELOG_END

* Go easy, test!
2019-12-06 16:30:25 +00:00
Remy
596a80f012 daml-lf: test equality and hashing of values (#3715)
* daml-lf: test equality and hashing of values

* daml-lf: remove comment about test

* format

* address reviews
2019-12-06 15:57:27 +00:00
Gerolf Seitz
658365e5eb Java codegen fixes (#3762)
* Fix compiler warnings in generated enums

* Fix warnings in generated equals method for parameterized types.

* Remove warning in equals for records without fields.

CHANGELOG_BEGIN
- [Java Bindings] Removed warnings in code emitted by the Java Codegen.
CHANGELOG_END

* fix compilation error in tests
2019-12-06 15:10:38 +00:00
Samir Talwar
b7fdf2236c sandbox: Merge DbDispatcher and SqlExecutor. (#3753)
* sandbox: Inline SqlExecutor into DbDispatcher.

There's no clear delineation of responsibilities. I'll try and separate
some of this back out again later.

* sandbox: Make it clear that we're "starting" the DbDispatcher.

* sandbox: Simplify the promise-based behavior in DbDispatcher.

* sandbox: Rename `noOfShortLivedConnections` to `maxConnections`.
2019-12-06 15:09:58 +00:00
Gerolf Seitz
e1a59776a7 Increase party allocation timeout and mark test as flaky (#3768)
The timeout for witnessing the party allocation is now increased
to 30 seconds (from 10 seconds). Additionally the LotsOfParties
conformance test is marked as flaky.
2019-12-06 14:34:23 +00:00
Remy
e055b2bd9b DAML LF: expose generic equality in LF (#3752)
* daml-lf: expose generic equality

* drop unnecessary parentheses

* fix issue number

* address Fran's comments
2019-12-06 14:12:48 +00:00
Moritz Kiefer
321032a0f2 Create a new type for package imports (#3765)
My brain can’t handle 3-tuples with strings and booleans, so this PR
introduces a new record type to make those fields clearer.
2019-12-06 14:02:59 +00:00
Nemanja
bec60c69b7 added tagmanager to docs (#3149)
* added tagmanger to docs
- 3rd party integration
- event tracking

* Removed the duplicate TagManager script

* Pushed the wrong script, sorry. This PR has the propper one
2019-12-06 13:52:06 +00:00
Samir Talwar
1f6529aea7 Upgrade Scalafmt to v2.3.1. (#3764)
* Upgrade Scalafmt to v2.3.1.

* Reformat daml-assistant/daml-sdk/.../SdkMain.scala with Scalafmt.
2019-12-06 12:56:07 +00:00
associahedron
158d1357c2 Fix code syntax in latest release notes (#3760) 2019-12-06 11:59:29 +00:00
associahedron
2e7932d409
daml-assistant: Add a 2s timeout to the version check (#3763)
* Add a 2s timeout to the version check

* Fix error message.
2019-12-06 11:34:04 +00:00
Gerolf Seitz
17435d6e99 Sandbox: Remove streaming connections (#3682)
* Sandbox: Remove streaming connections

The separate database connection pool for streaming connections
was only used for the active contracts stream. However, a single
db connection was being occupied until the last active contract was
streamed over the Ledger API to the client. This effectively means
that only ever 2 concurrent active contract streams could exist.
No need to say that this is bad design.

The following changes happened:

- remove the db connection pool for streaming connections
- replace the streaming mechanism for active contracts with
  the already existing pagination mechanism in JdbcLedgerDao
- change the pagination mechanism to actually use database level
  limit and offset instead of doing the pagination "client side"
- configure the HikariDataSource with the metric registry

CHANGELOG_BEGIN
- [Sandbox] Improve loading of active contracts for the Sandbox SQL backend.
CHANGELOG_END

* Extract PaginatingAsyncStream from JdbcLedgerDao for testing

* Reset metrics registry before each test
2019-12-06 10:58:10 +00:00
Oliver Seeliger
655d3d6716 Disable multi-participant ledger api test by default (#3761) 2019-12-06 10:56:32 +00:00
Remy
ac420a483b Daml LF Encoder: reactivate and fix tests (#3748)
* daml-lf-encoder: reactivate and fix tests

* lf encoder: fix test

* address reviews
2019-12-06 10:37:03 +00:00
Moritz Kiefer
ab59376cb3 Fix Sdk-Version field in daml-sdk-head (#3759)
In #3706 we fixed the SDK versions of the daml-trigger an daml-script
libraries and (rightfully) stopped filtering out 0.0.0 from the SDK
version check in `damlc build`. However, this broke daml-sdk-head
since we still distributed the DARs in daml-sdk-head with the
Sdk-Version set to whatever the current release is rather than 0.0.0.
2019-12-06 10:08:37 +00:00
Moritz Kiefer
3f12e60b12 Refactor packaging logic (#3749)
* Refactor packaging logic

This is a first step towards cleaning up the packaging logic and
adding some comments to make it clearer what is going on. There are no
functional changes in this PR.

There is more stuff here that we can and should cleanup but I will
leave that for separate PRs.

* Update compiler/damlc/lib/DA/Cli/Damlc/Packaging.hs

Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>

* Update compiler/damlc/lib/DA/Cli/Damlc/Packaging.hs

Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>

* Update compiler/damlc/lib/DA/Cli/Damlc/Packaging.hs

Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>

* Update compiler/damlc/lib/DA/Cli/Damlc/Packaging.hs

Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>

* Document topological sorting

* Undo requiredE change
2019-12-06 09:55:27 +00:00
Brian Healey
71205e1f78 Include party allocation related events in integrity check (#3758) 2019-12-06 09:22:52 +01:00
associahedron
9a27ed092f Prepare release 0.13.39 (#3750) 2019-12-05 20:08:12 +00:00
Gerolf Seitz
0c00b28c83
Party management (#3744)
* Implement Async party allocation

* Address Jussi's review.

- Remove comments about implicit party entries
- Remove PollingUtils#pollSingleUntilPersisted
- Remove ill-placed comment in SqlLedger

Review: https://github.com/digital-asset/daml/pull/3744#pullrequestreview-327453144

* Address Samir's review

- Slight improvement in InMemoryKVParticipantStateIT
- Fix typo in scaladoc
- Fix SqlLedgerSpec: remove unused param and rephrase clues

Review: https://github.com/digital-asset/daml/pull/3744#pullrequestreview-327461368

* Address Stefano's review

Review: https://github.com/digital-asset/daml/pull/3744#pullrequestreview-327455206

* Only poll from the ledger end just before submitting the party allocation request.
2019-12-05 17:50:21 +01:00
Stephen Compall
1388ebd695 compile ValuePredicates to SQL and run json-api queries in Postgres (#3599)
* toSqlWhereClause WIP

* literal cases for SQL predicate generation

* fill in the remaining Literals' sql-equal data

* add other likely elements of the toSqlWhereClause fold

* SQL ListMatch case

* SQL VariantMatch case

* partial SQL RecordSubset case

* SQL Range case

* comments on JSON encoding for DB

* new relationship for the 3 paths in toSqlWhereClause Rec

- conversion to Fragment is further delayed to allow mixed-mode matching
  of records

* handle new Rec semantics in ListMatch (can't drop @> on the floor anymore)

* compile RecordSubset to SQL

* compile MapMatch to SQL

* note optimization for record & map

* optimize = when @> unavailable for record subsets

* optimize range when degenerately =

* don't discard the @> safety of empty test sets

* unnested SQL optional matching

* converting DB JSON to response body JSON

* compiling nested optionals to SQL

* add missing length check to ListMatch

* remove unneeded parens from generated predicates

* remove 1 = 1 leader from generated predicates

* test the generated SQL from toSqlWhereClause

* searchDb WIP

* searchDb integrated

* coerce party to text in selectContracts; log that query

* fixing query

* fixing scala formatting

* removing unused functions

* removing unused type alias

* checking a that search returned exactly what we searched for,

also checking that all contracts got stored in the DB

* factor commonalities of MapMatch and RecordSubset

* cleanup

* cleanup

* changelog

CHANGELOG_BEGIN

- [JSON API - Experimental] Queries will always run against Postgres if Postgres is
  configured.  See `issue #3388 <https://github.com/digital-asset/daml/issues/3388>`_.

CHANGELOG_END

* fix record =-unsafe detection
2019-12-05 11:15:01 -05:00
Moritz Kiefer
e9d7e783f2
Fix incremental builds (#3751)
Sadly, I haven’t managed to come up with an isolated test case for
this but I’ve tested this on a large internal codebase and it fixes an
issue with damlc build --incremental=yes.
2019-12-05 17:07:20 +01:00
Remy
96b3e33c27 daml-lf: specified generic equality (#3740)
* daml-lf: specified generic equality

* polishing

* address reviews

* more polishing

* address Samir's comments
2019-12-05 15:47:49 +00:00
associahedron
8127a415d9
Add unstable experimental text primitives in DA.Text and LF 1.dev (#3734)
* Add experimental text primitives

* Implement SBTextSlice

* Implemented SBTextSliceIndex

* Implement toUpper / toLower

* Implement SBTextContainsOnly

* Implement SBTextReplicate

* Implement SBTextReplicate

* Implement SBTextSplitOn

* Implement SBTextIntercalate

* Add unstable primitives in LFConversion

* Add unstable text primitives in DA.Text

* Remove all the ASCII infixes

* Fix typing mishap

* Change numbering for unstable primitives

* Deal with UTF8

* More careful slice

* Fix typo

* Missing decoder
2019-12-05 14:35:50 +00:00
Samir Talwar
cb378017d9 sandbox: If the Flyway migrations fail, crash the process. (#3731)
* sandbox: If the Flyway migrations fail, crash the process.

Otherwise running it in the background can make it look like
everything's OK.

* reference-v2: If the Flyway migrations fail, crash the process.

Otherwise running it in the background can make it look like
everything's OK.

* sandbox: Remove an errant debugging `println`.

* sandbox: Use `DirectExecutionContext.implicitEC`.
2019-12-05 13:44:13 +00:00
Moritz Kiefer
022c60ccb3 Publish the thin trigger library instead of the fat jar (#3739)
This still contains the main class so you can use it like you would
use the fat jar but publishing fat jars to maven central is apparently
bad practise and some peple have asked for the library.

This includes some slight tweaks to the scala_docs rule to make it
capable of coping with the generated source file and a hack in the
release script to avoid it complaining about the scenario proto
library not being published to Maven even though it is included in the
transitive deps.
2019-12-05 13:04:20 +00:00
Stefano Baghino
5254520b80 Spin off TokenHolder into a new library (#3741)
* Spin off TokenHolder into a new library

Avoids having weird dependencies between different packages, makes TokenHolder available on Maven

* Fix auth-utils path
2019-12-05 12:31:14 +00:00
Darko Pilav
d8eebc866d Change range of snippet to be displayed (#3746)
Fixes #3745
2019-12-05 13:25:13 +01:00
Robin Krom
3f666792b1 language: suffix all dalfs dependencies in a dar with the pkgid. (#3668)
* language: suffix all dalfs dependencies in a dar with the pkgid.

This makes sure that dalf dependencies are not accidentally overwritten
when two packages with equally named dalfs are imported.

* factor out parseUnitId
2019-12-05 11:47:45 +00:00
Stefano Baghino
da9306414b Add authentication support to DamlLedgerClient (#3743)
Add documentation

CHANGELOG_BEGIN
- [Java Bindings] Added authentication support. See `issue #3626 <https://github.com/digital-asset/daml/issues/3626>`__.
CHANGELOG_END
2019-12-05 10:57:51 +00:00