Commit Graph

250 Commits

Author SHA1 Message Date
Gary Verhaegen
878429e3bf
update copyright notices to 2020 (#3939)
copyright update 2020

* update template
* run script: `dade-copyright-headers update .`
* update script
* manual adjustments
* exclude frozen proto files from further header checks (by adding NO_AUTO_COPYRIGHT files)
2020-01-02 21:21:13 +01:00
Stefano Baghino
84e6064d51 Remove unreleased.rst (#3547)
* Start working on getting rid of unreleased.rst

Document new process in CONTRIBUTING.md,
.github/pull_request_template.md and unreleased.rst (for good measure)

Report the previous changelog additions here so that they're not lost in
the mist of times.

CHANGELOG_BEGIN

- [DAML Stdlib] Added the ``NumericScale`` typeclass, which improves the type inference for Numeric literals, and helps catch the creation of out-of-bound Numerics earlier in the compilation process.

- [DAML Triggers] ``emitCommands`` now accepts an additional argument
  that allows you to mark contracts as pending. Those contracts will
  be automatically filtered from the result of ``getContracts`` until
  we receive the corresponding completion/transaction.

- [Navigator] Fixed a bug where Navigator becomes unresponsive if the ledger does not contain any DAML packages.

- [Ledger-API] Add field ``gen_map`` in Protobuf definition for ledger
  api values. This field is used to support generic maps, an new
  feature currently in development.  See issue
  https://github.com/digital-asset/daml/pull/3356 for more details
  about generic maps.

  The Ledger API will send no messages where this field is set, when
  using a stable version of DAML-LF.  However the addition of this
  field may cause pattern-matching exhaustive warnings in the code of
  ledger API clients. Those warnings can be safely ignored until
  GenMap is made stable in an upcoming version of DAML-LF.

- [JSON API - Experimental] CLI configuration to enable serving static content as part of the JSON API daemon:
  ``--static-content "directory=/full/path,prefix=static"``
  This configuration is NOT recommended for production deployment. See issue #2782.

- [Extractor] The app can now work against a Ledger API server that requires client authentication. See `issue #3157 <https://github.com/digital-asset/daml/issues/3157>`__.
- [DAML Script] This release contains a first version of an experimental DAML script
   feature that provides a scenario-like API that is run against an actual ledger.
- [DAML Compiler] The default DAML-LF version is now 1.7. You can
  still produce DAML-LF 1.6 by passing ``--target=1.6`` to ``daml
  build``.

- [JSON API - Experimental] The database schema has changed; if using
  ``--query-store-jdbc-config``, you must rebuild the database by adding
  ``,createSchema=true``.
  See `issue #3461 <https://github.com/digital-asset/daml/pull/3461>`_.

- [JSON API - Experimental] Terminate process immediately after creating schema. See issue #3386.

- [DAML Stdlib] ``fromAnyChoice`` and ``fromAnyContractKey`` now take
  the template type into account.

CHANGELOG_END

* Document new release process to gather changelog additions

* Change the release script to ignore unreleased.rst

* Remove spurious unreleased.rst lines

* Transition to use tags

* Document new way to get changelog additions with tags

* Update release/RELEASE.md

Co-Authored-By: Gary Verhaegen <gary.verhaegen@digitalasset.com>

* Address https://github.com/digital-asset/daml/pull/3547#discussion_r348438786

* Document correction process

* Add copyright header to unreleased.sh

* Update CONTRIBUTING.md

Co-Authored-By: Gary Verhaegen <gary.verhaegen@digitalasset.com>

* Modify CONTRIBUTING.md after @garyverhaegen-da's proposal

* Make unreleased.sh run per commit and treat tags as case-insensitive

* Fix documentation for replacements
2019-11-20 15:16:57 +00:00
Moritz Kiefer
1bc4bb76a4 Make AnyChoice and AnyContractKey take template type into account (#3541)
* Make AnyChoice and AnyContractKey take template type into account

fixes #3540

* Update template desugaring

* Switch to proper ghc-lib release
2019-11-20 10:40:14 +00:00
Leonid Shlyapnikov
a849bb86e4 JSON API: quit immediately after creating schema (#3534)
* quitting after create schema

* better error codes

* quitting after create schema

* better error codes

* explicitly creating schema during integration testing

* Updating release notes and CLI help

* Updating release notes

* minor update

* Fixing formatting

* update
2019-11-19 21:39:18 +00:00
Stephen Compall
1b6e4523ed replace JSON witnessParties column with a text[] column (#3461)
* replace JSON witnessParties column with a PG text[] column

* notes about the concurrent behavior of updateOffset

* include witness party in selectContracts query

* fmt

* fix witness_parties fetching in http-json tests

* release note

* typo in prior release note

* move new release note to bottom
2019-11-19 19:53:43 +00:00
Moritz Kiefer
8615266230 Switch the default DAML-LF version to 1.7 (#3530) 2019-11-19 19:18:07 +00:00
Moritz Kiefer
7deca903bb Add documentation for DAML script and bundle it in the SDK (#3527) 2019-11-19 17:19:57 +00:00
Stefano Baghino
c51ecd3c58
Extractor with authentication (#3514)
* Make Extractor work against Ledger API servers protected by authentication

* Update changelog

* Address https://github.com/digital-asset/daml/pull/3514#pullrequestreview-318811500

- https://github.com/digital-asset/daml/pull/3514#discussion_r347753205
- https://github.com/digital-asset/daml/pull/3514#discussion_r347753629
- https://github.com/digital-asset/daml/pull/3514#discussion_r347754462

* Address https://github.com/digital-asset/daml/pull/3514#discussion_r347755484

* Address https://github.com/digital-asset/daml/pull/3514#discussion_r347752424

* Address https://github.com/digital-asset/daml/pull/3514#discussion_r347756328

* Fix docs

* Fix artifacts.yaml
2019-11-19 17:12:25 +01:00
Leonid Shlyapnikov
e1cbd38f48 CLI configuration to enable serving static content as part of the JSON API daemon (#3460)
* StaticContentEndpoint

* cleanup

* release notes

* Add test

* SDK doc update

* Copying the directory content explicitly

to avoid creating a symlink, I believe that is the cause of the Windows issue

* creating a static content dir from the test

* fixing release notes

* creating a tmp dir

* fixing release notes

* Using Gen.Identifier to generate random strings
2019-11-18 19:31:33 +00:00
Remy
fedc49dd12 leger-api: add GenMap support (#3506)
* leger-api: add GenMap support

* cosmetic

* Address Gerolf's  comment

* add a point in the release notes

* Address Fran's comments
2019-11-18 18:13:25 +00:00
Robert Autenrieth
388b204e92 Fix Navigator crashing on ledgers with no packages (#3400) 2019-11-18 16:15:57 +00:00
Moritz Kiefer
5458053ea9 Add pending set to DAML triggers (#3502)
fixes #3360
2019-11-18 14:25:40 +00:00
associahedron
d654d7591b Improve Numeric type inference and safety by adding a scale constraint. (#3505)
* Add NumericScale typeclass

* Add release note
2019-11-18 13:53:17 +00:00
Gerolf Seitz
e6651ccdc2 Re-release 0.13.35 as 0.13.36 (#3478) 2019-11-14 19:54:06 +00:00
Oliver Seeliger
7744b886dd Release 0.13.35 (#3464) 2019-11-14 15:00:57 +00:00
Moritz Kiefer
4eb72d496e
Fix conversion from Ledger API to Speedy in DAML triggers (#3458)
Previously, we use SValue.fromValue for the conversion. However, this
breaks in cases like Numeric where the scale information is lost. By
using the ValueTranslator instead, we avoid this issue.

There is a similar problem in DAML script but I’ll fix that in a
separate PR.

Since the ValueTranslator is package private, this PR moves the
triggers in the engine package.
2019-11-14 09:26:36 +01:00
Stephen Compall
c36696f3a7 json-api: in-memory comparison query (#3405)
* comparison query parser given scalar parser

- written in half-error-propagation style to better suit other potential
  error features

* factor dupes in RangeExpr

* text range parsing

* interpreting Range into in-memory predicate; points out bad dedupe from earlier

* make reuse of the scalar extractors much nicer

* express date, time, int64 as factored-out range exprs

* express numeric as factored-out range expr

* factor \&/ usage

* refactor LF value extractors for reuse in range queries

* factor mkRange usage

* totally deconstruct the int64 and text cases

* totally deconstruct the date and timestamp cases

* totally deconstruct the numeric case

* document comparison queries

* use Utf8.Ordering for text comparison queries

* int64 range query tests

* more int64 range query tests

* date, string, numeric range query tests

* include line # in query test successes table

* timestamp range query tests

* add release note

* remove duplicate changelog entry from #3425
2019-11-12 23:02:33 +00:00
Leonid Shlyapnikov
cb86dc4029
Retry on PostgreSQL UNIQUE_VIOLATION (#3425)
* Retry on UNIQUE_VIOLATION

* Update release notes

* Hiding method that is not supposed to be used directly

* using connection.raiseError

* Force re-run on stale offset update
2019-11-12 14:49:13 -05:00
Moritz Kiefer
45d2b7048c
[daml-triggers] Rename getTemplates to getContracts (#3430)
Given that this returns the contracts of a given template rather than
returning templates, this name makes a lot more sense.
2019-11-12 14:48:24 +01:00
Moritz Kiefer
11044f3683 Add exerciseByKey to DAML triggers (#3413)
fixes #3317
2019-11-11 18:12:25 +00:00
Moritz Kiefer
028dfcf6d4 Expose projection from contract key to maintainers (#3396) 2019-11-09 18:00:00 +00:00
Gerolf Seitz
d032415d1b
Extractor: fixed bug in setting up TLS to the ledger (#3394)
The parameters for the private key and the certificate were swapped,
resulting in extractor not being able to establish a secure connection
to the ledger.
2019-11-08 17:22:45 +01:00
Robin Krom
6cf3a5276f
language: introduce data-imports (#3399)
* language: introduce data-imports

Right now the user experience for importing dalfs and dars from
different sdks is quiet confusing. This PR tries to solve this. We add
an additional field `data-imports` to daml.yaml. These imports can come
from different SDK's and we will generate interface files containing the
data types and their Template instances.

This also simplifies the migration command, as it now always imports the
respective packages as `data-imports`.
2019-11-08 16:32:30 +01:00
Gerolf Seitz
d30b6eb94b
Sandbox: fix excessive memory consumption (#3362)
* Implement proper stream pagination

The previous pagination mechanism for streaming ledger entries was implemented
as a recursive method call with manually concatenating akka Sources.
However, this didn't work properly and resulted in all the subsources being
forced immediately, resulting in parallel requests for all pages
(0-100, 100-200, 200-300, ...) instead of the expected mechanism of loading
the first page and on upstream request (i.e. client requests for more data
over grpc) loading the next page of data.

The current mechanism uses Source.unfoldAsync which makes the paging
mechanism work as expected: sequential loading of pages on demand.

* Serialize and deserialize Transactions outside the SQL Executor

This frees up the SQL Executor threads sooner for other work
and the conversion work only happens on a by need basis when
the consumer requests more data.
2019-11-08 10:25:29 +01:00
Robert Autenrieth
89d6c7375c
Add a JWT authentication to sandbox (#3283)
Fixes #3363
2019-11-07 23:04:16 +01:00
Leonid Shlyapnikov
b22bc2a0a7
RecordID by Choice + TemplateID resolution (#3389) 2019-11-07 15:54:05 -05:00
Moritz Kiefer
d7f5b3b436 Add support for on-disk incremental builds in damlc build (#3378)
* Add support for on-disk incremental builds in damlc build

* Normalise file paths of internal modules because Windows

* stop stealing my $s hlint

* Apparently jars are also called exe

* Address review comments

* Bump to proper ghcide revision
2019-11-07 16:56:42 +00:00
Oliver Seeliger
0df351b2f7
Multi domain ledger api server fixes (#3359)
* Divulged contract visibility in multi-participant environments #3351
* Ledger api server time service optional ability for testing #3225
* Allow ledger api server to share DAML-on-X DAML engine #2975
* Allow ledger api server participant ids with LedgerString chars #3327
* Ledger api server includes SQL description in errors #3324
2019-11-07 16:15:17 +01:00
Remy
fa6b339507
Release 0.13.34 (#3373) 2019-11-07 15:02:06 +01:00
Remy
ce36674ea0 daml-lf: clone dev protobuf for LF 1.7 (#3371) 2019-11-07 12:32:38 +00:00
Moritz Kiefer
c57a31d630
Compile DAML trigger library with --target 1.7 (#3372) 2019-11-07 12:08:44 +01:00
Remy
d39d7974fc daml-lf: freeze version 1.7 (#3340)
* daml-lf: freeze daml-lf 1.7

* speedy: remove useless double checks
2019-11-07 09:51:18 +00:00
Rohan Jacob-Rao
2eea995840 Release (#3369) 2019-11-06 23:30:00 +00:00
Rohan Jacob-Rao
0ae016ddf1 Display release notes using webview (#3321)
* Display release notes using webview

* Use const and fix string

* Check for version upgrade before showing release notes

* Changelog entry

* Use node-fetch instead of web-request

* Remove spurious state update
2019-11-06 17:01:35 -05:00
Stephen Compall
d0ec381925 json-api: store contracts similar to Extractor's single-table (#3192)
* new library ledger-service/db-backend

* borrow contracts table schema from extractor

* borrow contract insertion, removing some data to be unused

* match contract schema with insert function

* factor insertContract arguments

* offset table declarations

* CLI argument for query store

* surrogate template IDs

* compute surrogate template IDs on-the-fly

* database init action

* incoherent typeclasses, eh

* newtype SurrogateTpId

* offset fetch/update functions

* bad sql

* bulk insert contracts, function for selecting contracts

* expose contract column name for query's usage

* Initializing DB on startup if configured

* dropping existing tables as part of initialization

* fix some query syntax errors

* createSchema flag

* function for streaming transactions with jwt party selected

* formatting

* usage

* collect acs contracts and the ledger offset at the end

* lastOffset

* fixing merge conflicts, updating the way 3rd party deps are specified

* Moving ContractDao into http-json module

so it can take domain AST as an input

* cleanup

* injecting new dependencies

* split transaction batches into inserts and deletes

* generate sql for deleting contracts

* `fetch_sources = True` for java_deps

* make the delete-constructed fragment more efficient; handle empty list here

* pass logHandler for insertContracts

* ContractDao returns ConnectionIO, it's up to the caller to wrap query into a transaction

* fixing typo

* minor cleanup, moving fromLedgerApi factory function into corresponding companion objects

* don't need it any more

* GetActiveContractsResponse => domain.Contract factory

* make concatFragment private

* add partition graph; move other contract-fetching experiments to ContractsFetch

* experimenting with akka sources

* introducing domain.Offset to work around API's empty/null offset cases

* minor cleanup

* decompose fetchActiveContractsFromOffset

* missed via

* ACS splitting graph

* finish doc for ACS splitting graph

* remove unneeded stages

* WIP

* lazily read a stream of ConnectionIO into a single ConnectionIO

* cancel on IO error

* figuring out how to put all the pieces together

* graph WIP

* Removing workflowId from the JSON API

* simplify acsAndBoundary; describe other flow pieces

* WIP

* use Vector in InsertDeleteStep; add variant for ACS (no deletes)

* `org.wartremover.warts.NonUnitStatements` enforced in `http-json` module

* evaluate InsertDeleteStep to a ConnectionIO

* database variant of LfValueCodec, using numbers for numbers

* convert input to JSON, combine insert plans, connect rest of contractsToOffsetIo

* remove strict contractsToOffset sink

* moving dao methods into an object

* putting pieces together

* contractsFromOffset WIP

* should be it

* cleanup

* cleanup

* contractsIo that takes List[domain.TemplateId.RequiredPkg])

* contractsIo that takes List[domain.TemplateId.RequiredPkg])

* cleanup

* put all pieces together, testing

something does not work yet

* diff is not required to return anything

that is why Sink.lastOption that gives Option[domain.Offset]

* factor out tuple split

* use traverse syntax in contractsIo2

* factor explicit flow steps out of graph DSL; remove aggregate

* locally model the Absolute/Begin distinction for offset bookmarking in DB

* Adding test cases to run HTTP Service with Postgres backend

the same set of test cases, run with and without DB backend

* make better use of domain.Offset in OffsetBoundary

* monomorphize InsertDeleteStep#append

* Disabling a test that fails with DB backend

* add release note

* add release note about workflowId

* a test case that checks the number or stored contracts

* trying to figure out why Postgres test fails on Windows with NPE
2019-11-06 13:53:16 -05:00
associahedron
8a7c6cd94d Add support for single-constructor enum types. (#3345)
* Allow data A = A by prepending DamlEnum$ to type name.

* Single con enums basically work.

* Fix export lists for single constructor enum types.

* Revert "Fix export lists for single constructor enum types."

This reverts commit 7475a3dfbe3531d3ef62fdbcfe64c01a9e22d7af.

* Switch to a "stupid theta" approach

* Clean up enum type preprocessor

* Run enum preprocessor on generated code.

* Add daml-docs golden test for single constructor enums

* s/genPreprocessor/generatedPreprocessor/g

* Update copyright header

* Update release notes

* Remove unnecessary OverloadedStrings
2019-11-06 16:03:52 +00:00
Moritz Kiefer
bb15df1c7d
Add Applicative functions that ignore their results (#3354)
fixes #3315
2019-11-06 15:45:33 +01:00
Gerolf Seitz
7a15706101
Added a missing index to the disclosures table (#3343) 2019-11-06 13:48:36 +01:00
Moritz Kiefer
90679c93f8
Use regular ContractIds in DAML triggers (#3337)
The only reason for having AbsoluteContractId was that we could get
some more instances in particular `Ord` and `MapKey` but given that
`ContractId` will be a valid key type for the new DAML-LF maps, we can
just use slower implementations for now and switch to Map-based
implementations once that has landed.

fixes #3336
2019-11-05 15:15:07 +01:00
Martin Huschenbett
dff3edfd26 Add assertEq and assertNotEq to DA.Assert (#3318)
* Add assertEq and assertNotEq to DA.Assert

I prefer named functions over infix operators.

* Adapt doctest integration test
2019-11-04 22:48:46 +00:00
Stefano Baghino
96622c9fe7 Expose DB wait/exec times over JMX as well as logs (#3310)
* Expose DB wait/exec times over JMX

* Fix compilation errors

* Fix more compilation errors

* Update unreleased.rst

* Address https://github.com/digital-asset/daml/pull/3310#pullrequestreview-310346159

* Address https://github.com/digital-asset/daml/pull/3310#discussion_r341641717
2019-11-01 17:34:44 +00:00
Moritz Kiefer
bd19291632 Fix superclass detection for Numeric (#3316)
For numeric, the superclass dicts can be of the for `dict @10` so only
handling variables doesn’t work. Now we walk down applications and
check the name on the left.
2019-11-01 16:15:23 +00:00
Gerolf Seitz
510ecb7aeb
Extractor stores exercise events in single table data format. (#3303)
This also fixes a bug where extractor would set archived_by_transaction_id even
if the exercise event was non-consuming.

Fixes #3274.
2019-11-01 14:39:28 +01:00
Martin Huschenbett
6211d74193 Add filter function to TextMap and Next.Map (#3305)
This closes #2760.
2019-10-31 15:41:32 +00:00
Gerolf Seitz
9750ccf716
Navigator properly sets the max incoming message size again. (#3302)
The symptom for receiving messages larger than the configured maxInboundMessageSize is a
gRPC error like:

Oct 31, 2019 1:52:37 PM io.grpc.internal.AbstractClientStream$TransportState inboundDataReceived
INFO: Received data on closed stream

Fixes #3301.
2019-10-31 14:56:48 +01:00
Stefano Baghino
d9ae487fec Disallow empty command submission (#3270)
* Disallow empty command submission

Fixes #592
2019-10-31 08:37:58 +01:00
Jussi Mäki
819f8a8af5 Bring back daml integration kit docs (#3292)
* Bring back daml integration kit docs

This just revives the documentation, without updating it yet.

* Updated URLs and remvoe references to the IndexService

* Add release note for revival of integration kit docs

* Update unreleased.rst

* update release notes
2019-10-30 17:15:58 +00:00
Gerolf Seitz
8650d38c78
Fixed a bug in the DB migration script 10.1 (#3287)
The script would throw an exception for contracts without observers.

Fixes #3284.
2019-10-30 10:58:21 +01:00
Moritz Kiefer
e7280f96b3 Add dedupCreate/dedupExercise helpers to the DAML trigger API (#3280)
Given how common this is, it seems worthwile to add helpers for this.
2019-10-29 18:06:43 +00:00
Martin Huschenbett
8cfbd50c17 Don't cache the GHC Core produced during compilation (#3269)
* Don't cache the GHC Core produced during compilation

In our experiments, this reduced the memory footprint by ca. 18% on a very
big code base.

* Adapt integration tests

* Fix integration tests
2019-10-29 15:30:59 +00:00