Commit Graph

3600 Commits

Author SHA1 Message Date
Remy
1216fd1a6f
Engine: fix missing usage of Enum Svalue rank (#5083)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-19 14:27:13 +01:00
Robert Autenrieth
60a845ec72
Add new protobuf fields for new ledger time model (#5036)
* Add new protobuf fields for new ledger time model

Fixes #4344.

CHANGELOG_BEGIN
CHANGELOG_END

* Adapt haskell bindings
2020-03-19 13:53:49 +01:00
Martin Huschenbett
d9f2eed632
daml2ts: Simplify generated build config (#5085)
We don't use TypeScript's project references. Hence there's no point in
passing `--build` to `tsc`. Let's stop doing that hence.

By default, the TypeScript compiler type checks the `.d.ts` of all
dependencies of a project. This is painfully slow. For the packages
generated by `daml2ts` it also doesn't make a lot of sense since we control
all dependencies except for `@mojotech/json-type-validation`, which is
written in TypeScript itself and hence has very sane typings anyway.

This default behaviour can be disabled by setting `skipLibCheck` to `true`.
Doing that decreases the compilation time of _every_ single package generated
by `daml2ts` for the DAVL project from ~10s to ~2s. Let's get that time back!

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-19 12:21:05 +00:00
Moritz Kiefer
13816c75b3
Support uploading DARs to the trigger service (#5084)
* Support uploading DARs to the trigger service

This PR adds a new `upload_dar` endpoint that accepts a DAR as a
multi-part form request and adds it to the list of compiled packages.

I’ve also made the DAR passed in on startup optional now given the new
endpoint.

There is no endpoint for deleting a DAR so far but there is none on
the ledger API either so I think this not particularly urgent.

changelog_begin
changelog_end

* Address review comments
2020-03-19 13:02:04 +01:00
Andreas Herrmann
8527ec4c4b
Infer daml-script package id from script (#5076)
* Factor out common identifier generation

For `DA.Types`, `DA.Internal.Any`, and `Daml.Script`.

* Factor out Script type for DAML scripts

* Adapt DAML script test runners

* Adapt REPL

CHANGELOG_BEGIN
CHANGELOG_END

* ./fmt.sh

* Avoid `unapply`

addressing
https://github.com/digital-asset/daml/pull/5076#discussion_r394526881

* Pure Script.fromIdentifier

* Pure Script.fromDar

* Simplify test script discovery

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-03-19 11:59:06 +00:00
Nemanja
8dc99b583b
Follower terminology in the docs (#5067)
* Follower terminology in the docs

Changed the docs to a more twitter/Instagram like terminology where we use 'follower' relationship instead of 'friend'. Resolves #5015

CHANGELOG_BEGIN
CHANGELOG_END

* Update docs/source/getting-started/index.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Update docs/source/getting-started/index.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Update docs/source/getting-started/index.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Update docs/source/getting-started/app-architecture.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Update docs/source/getting-started/app-architecture.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Update docs/source/getting-started/first-feature.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Update docs/source/getting-started/first-feature.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Update docs/source/getting-started/first-feature.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Update docs/source/getting-started/first-feature.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Update docs/source/getting-started/first-feature.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
2020-03-19 12:00:09 +01:00
Shayne Fletcher
9fbd3da302
Use type application for 'decode' (#5081)
changelog_begin
changelog_end
2020-03-19 06:53:04 -04:00
Remy
c097266edc
java-binding: fix protobuf testing generator (#5082)
* exclude com.daml.ledger.javaapi.data.ValueSpec test

* Revert "exclude com.daml.ledger.javaapi.data.ValueSpec test"

This reverts commit 9b737bc452.

* java-binding: fix testing Generators

* revert minor change

* changelog

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-19 09:59:36 +00:00
Stefano Baghino
499fd1b75a
Add method to look up flat transaction by id (#5080)
* Add method to look up flat transaction by id

CHANGELOG_BEGIN
CHANGELOG_END

* Limit visibility of helper
2020-03-19 10:01:09 +01:00
Oliver Seeliger
e50a33019c
StandaloneApiServer facility for lf-seeding opt-out (#5078)
Until all DAML-on-X implementations have a chance to implement
seeding support for daml-lf contract-ids.

Closes #5077

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-19 08:50:15 +00:00
nickchapman-da
d81caa9568
rework daml_compile bazel rule (#5070)
Avoiding `damlc compile/package` commands (which we would like to deprecate), and replace with plain `damlc build` together with a post dar->dalf extraction step in the couple of places where we actually want the .dalf for testing.

changelog_begin
changelog_end
2020-03-18 23:06:55 +00:00
Leonid Shlyapnikov
29c340fed0
2nd part of stabilizing flaky websocket integration test (#5079)
* 2nd part of stabilizing flaky websocket integration test,

handling empty events, which we may receive a bunch in the end of the test
when there is no activity

changelog_begin
changelog_end

* fixing `Offset.semigroup`, thanks @S11001001!
2020-03-18 17:04:01 -04:00
Remy
3cbba8a444
Engine: Use enum and variant svalues' enum and variant s rank (#5068)
CHANGELOG_BEGIN
CHANGELOG_END

We use the rank recently introduced in the enum and variant svalues for:
* pattern matchin
* value hashCode
* value ordering (so the oredering match the DAML-LF spec)
* value equality
2020-03-18 20:44:17 +01:00
Gary Verhaegen
7b9932a7f3
specify which channels to post releases to (#5075)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-18 17:22:02 +00:00
Stefano Baghino
856ed80728
Add tests for completion reader in JdbcLedgerDaoSpec (#5072)
* Add tests for completion reader in JdbcLedgerDaoSpec

CHANGELOG_BEGIN
CHANGELOG_END

* Add test for completion, make offset checks more strict

* Fix badly typed check

* Comment reason for weird matcher use

* Address https://github.com/digital-asset/daml/pull/5072#discussion_r394447235

* Address https://github.com/digital-asset/daml/pull/5072#discussion_r394447981

* Address https://github.com/digital-asset/daml/pull/5072#discussion_r394448506

* Address https://github.com/digital-asset/daml/pull/5072#discussion_r394451069
2020-03-18 16:34:36 +00:00
Shayne Fletcher
76a37b9560
Update instructions (#5069)
changelog_begin
changelog_end
2020-03-18 14:22:24 +00:00
Remy
c2070170b3
Engine: add rank to Enum and Variant SValue (#5048)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-18 14:37:44 +01:00
Shayne Fletcher
8a9e5e778a
Notes on updating release version (#5066)
* Notes on updating release version

changelog_begin
changelog_end

* Update release/RELEASE.md

Co-Authored-By: Gary Verhaegen <gary.verhaegen@digitalasset.com>

* Corrections

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2020-03-18 13:06:38 +00:00
Stefano Baghino
f93333a503
Run JdbcLedgerDaoSpec on both H2 and Postgres (#5065)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-18 13:50:38 +01:00
Shayne Fletcher
e9938c4da2
0.13.56-snapshot (#5064)
changelog_begin
changelog_end
2020-03-18 08:45:38 -04:00
Moritz Kiefer
0471b26d70
Get openssl from msys2 (#5063)
* Get openssl from msys2

changelog_begin
changelog_end

* Try to fix opensl config location
2020-03-18 13:09:03 +01:00
associahedron
6ea118d614
Update DA.Map to reflect generic order. (#5060)
* Update DA.Map to reflect generic order.

changelog_begin
changelog_end

* Improve example built-in types
2020-03-18 11:07:49 +00:00
Moritz Kiefer
3493b33de5
Release 0.13.55 (#5058)
changelog_begin
changelog_end
2020-03-18 11:53:57 +01:00
Stefano Baghino
0cefe98945
Modularize JdbcLedgerDaoSpec (#5061)
* Modularize JdbcLedgerDaoSpec

Adding more tests, but breaking it up a bit to make sense of it beforehand.

CHANGELOG_BEGIN
CHANGELOG_END

* Remove unnecessary suppressed warnings
2020-03-18 10:41:31 +00:00
Moritz Kiefer
42b4441159
Bump openssl (#5054)
* Bump openssl

The previous one has stopped working for some reason :sadpanda:

changelog_begin
changelog_end

* daml-assistant: Add `--wall-clock-time` to the Sandbox Next test.

Missed this due to doing two things at once.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-03-18 10:24:05 +01:00
Moritz Kiefer
bd6db5be68
Handle failures during trigger initialization synchronously (#5046)
Previously the http endpoint for starting a trigger would always
return immediately. Based on the recent refactorings, we now do the
non-IO trigger initialization synchronously and return a failed http
status code with an error message.

This also refactors the code to only have one (mutable) set of
compiled packages which is a prerequisite for dynamic package uploads.

changelog_begin
changelog_end
2020-03-18 10:23:41 +01:00
Samir Talwar
7510d0d404
daml-assistant: Add daml sandbox-next. (#5045)
* daml-assistant: Add `daml sandbox-next`.

CHANGELOG_BEGIN
- [DAML Assistant] You can now run a pre-release version of Sandbox with
  ``daml sandbox-next`` so you can test it out and verify everything is
  working as expected. Running this will launch Sandbox rebuilt on a
  more modern architecture. An upcoming release of DAML will switch over
  to the new implementation by default.
CHANGELOG_END

* daml-assistant: Explain that sandbox-next is experimental.

Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* daml-assistant: Copy-pasta an integration test for `daml sandbox-next`.

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-03-18 08:25:08 +00:00
Samir Talwar
cbeeb5aafc
sandbox: Fail to start if a time mode is not explicitly specified. (#5033)
* sandbox: Fail to start if a time mode is not explicitly specified.

CHANGELOG_BEGIN
- [Sandbox] Sandbox is switching from Static Time mode to Wall Clock
  Time mode as the default. To ensure that our users know about this,
  for one version, there will be no default time mode. Instead, users
  will have to explicitly select their preferred time mode by means of
  the `--static-time` or `--wall-clock-time` switches. In the next
  release, Wall Clock Time will become the default, and users who are
  happy with the defaults will no longer need to specify the time mode.
CHANGELOG_END

* daml-script|triggers: Specify time mode when testing against Sandbox.

* daml-assistant: Default the Sandbox to wall clock time.

CHANGELOG_BEGIN
- [DAML Assistant] Initializing a new DAML project adds a switch to
  ``daml.yaml`` to ensure Sandbox can continue to start with ``daml
  start``::

      sandbox-options:
        - --wall-clock-time
CHANGELOG_END

* docs: Update the DAML Script and Triggers docs to use Wall Clock time.

It's now what Sandbox will use by default when using `daml init`.

* docs: Change the Quickstart to run Sandbox in wall clock time.

This explains why the contract IDs may vary.

It also updates the manual release testing script to match.
2020-03-18 08:25:03 +00:00
Gerolf Seitz
cf032a02cf
Introduce Stable Offsets (#4953)
A "stable offset" in the context of the Participant Server is the offset
that was provided by the ledger backend (be it kvutils, corda, daml on sql).

The Participant Server does not keep a participant-local offset anymore.
In a single domain/kvutil setup, this makes offsets stable across participants,
since all participants will see the same offset for the same transaction.

The following changes were needed to achieve this:
- The participant server always uses the offset provided by the backend
  AS IS (no more +1 magic).
- Offsets provided to the Ledger API in requests must be treated as
  startExclusive and endInclusive (previously beginInclusive and
  endExclusive).

CHANGELOG_BEGIN
[Ledger API]: Offsets have been redefined. Instead of being represented
by a number or a structured string, an offset is now an opaque string
that can be compared lexicographically.
[DAML Integration Kit]: The bounds for ``Dispatcher`` are now
startExclusive and endInclusive.
CHANGELOG_END

---------
ledger api:
ledger_offset.proto
  Changed definition of offsets, since they can now be compared
  lexicographically.

---------
participant-state-api:
Offset:
  Changed from Array[Long] to ByteString. Ledgers need to make sure that the
  offsets produced are strictly monotonically increasing according to
  lexicographical order.

---------
akka-streams:
Dispatcher, DispatcherImpl, SubSource:
  Changed interval handling to exclusive/inclusive.

---------
ledger-on-memory:
InMemoryLedgerReaderWriter, InMemoryState:
  Changed interval handling to exclusive/inclusive.

---------
ledger-on-sql:
CommonQueries, SqlLedgerReaderWriter:
  Change interval in query and boundary handling.

---------
kvutils:
KeyValueParticipantStateReader, KVOffset:
  Convenience functions for kvutils to add or remove sub-indexes for
  offsets.
  KV ledger implementations can use KVOffset to construct a structured offset.

---------
Participant Server:
JdbcLedgerDao:
  Use Offset instead of Long.
  Fetch offsets directly as Offset from the database with proper anorm
  integration.
  Change interval handling to exclusive/inclusive.

CommandCompletionsReader, CommandCompletionsTable:
  Change interval handling to exclusive/inclusive.

BaseLedger:
  Use Offset instead of Long.
  Change interval handling to exclusive/inclusive.

Conversions:
  Anorm integration for using Offset in queries and result parsers.

JdbcIndexer:
  Remove references to "extenalLedgerEnd" and participant-local Long
  offset (headRef).

---------
sandbox:
In general:
  Use the Offset type everywhere instead of Long.

SQL migrations:
  Change all offset columns to bytea or BINARY.

LedgerBackedIndexService:
  Proper bounds checking has been pushed down to Dispatcher, which
  allowed simplifying the acceptedTransactions implementation.

InMemoryLedger, LedgerEntries:
  Change interval handling to exclusive/inclusive.
  Transaction lookup by ID is now O(n) because transaction IDs are not
  necessarily the same as the offset.

SqlLedger:
  Remove external offset references.
2020-03-18 08:43:41 +01:00
Nemanja
8505135ec2
Wording fixes in GSG (#5018)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-18 08:39:31 +01:00
Nemanja
40ef8b18c7
GSG Application architecture diagram (#5017)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-18 08:39:00 +01:00
Robert Autenrieth
a546269519
Improve ReadService docs (#5043)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-18 07:26:45 +01:00
Remy
eae3844ed9
Engine: Replace SResultMissingDefinition by SResultNeedPackage (#5039)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-17 19:25:57 +01:00
Miklos
9aa61ec3da
Basic documentation for simplified ledger integration API (#5032)
* Switched to ByteString from Array[Byte] on almost all simplified API interfaces.

* Sort output by keys.

* Added comment.
CHANGELOG_BEGIN
CHANGELOG_END

* Removed DamlLogEntryId from LedgerEntry.

* Return a SortedMap ordering output state by its keys' hash in order to have deterministic ordering.

* Code tidying.

* Added implicit conversion for anorm for ByteStrings to make SQL queries cleaner.

* Ooops, missed adding a header.

* Avoid copying bytes by anorm by using ByteString.newInput()

* Added some Scaladoc to simplified API interfaces.

* Added docs to LedgerStateAccess.

* Reverted some changes.

* Added some docs to ValidatingCommitter.

* Corrected some typos.

* Added package-level documentation to kvutils.api.

* Clarified convenience classes for LedgerStateOperations.

* Update ledger/participant-state/kvutils/src/main/scala/com/daml/ledger/participant/state/kvutils/Version.scala

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* Minor rewording.

* Added missing header.

* Fixed problem with merge.

Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-03-17 17:02:27 +00:00
Shayne Fletcher
e71cba698e
daml2ts : Simplify docs, use mapped types for enums. (#5044)
* Simplify docs, use mapped types for enums.

changelog_begin
changelog_end

* remove serializale check

* add 'keys' property to enums

* Simplify docs just a little bit more
2020-03-17 16:27:46 +00:00
Moritz Kiefer
fb86152f8c
Move non-IO initialization out of trigger runner (#5042)
Previously parts of the initialization, in particular, the code for
finding the filter and the heartbeat were part of the Runner. This led
to an akward API and didn’t really make any sense.

Now all of this code is part of a pure `Trigger.fromIdentifier`
method and the runner only takes care of actually running the
ledger. This could also be useful for the trigger service where we
might want to synchronously call `getIdentifier` so users get some
indication if there request even points to a valid trigger
directly. However, this is not tackled by this PR.

changelog_begin
changelog_end
2020-03-17 15:26:58 +01:00
Miklos
c3068b2955
Allow reusing same ValidatingCommitter for multiple participants (#5041)
* Pass submitting participant ID as parameter to ValidatingCommitter.

* Made reference to processSubmission.
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-17 13:06:39 +00:00
Robert Autenrieth
29b4518cff
Add the invalid ledger time rejection reason (#5023)
* Add the invalid ledger time rejection reason

CHANGELOG_BEGIN
CHANGELOG_END

* Move definitions

* Improve wording
2020-03-17 14:06:20 +01:00
Stefano Baghino
54904e201f
Add missing migration for events to H2 (#5040)
Missing from #5029, raised the incorrect behavior of TransactionWriter.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-17 11:34:46 +00:00
Moritz Kiefer
cc2ca20336
Make trigger runner specific to a given trigger (#5038)
Previously the runner class was in a weird state where it was specific
to a DAR but not to an individual trigger. This meant that you had to
pass around a fair bit of state which got a bit awkward. This PR
addresses this by making the trigger class specific to the trigger.

It also now accepts `CompiledPackages` instead of a DAR which should
make it easier in the trigger service to support dynamic package
uploads.

changelog_begin
changelog_end
2020-03-17 11:51:06 +01:00
Remy
6e535e6c42
DAM-LF minor fixes relative to GenMap (#5028)
Here we fix some minor issue relatives to GenMap.

In particular:
    * validity of keys are checked when GenMap is created.
    * interface test
    * outdated comments
2020-03-17 11:00:33 +01:00
Stefano Baghino
13f44ff3d9
Write transactions ready to be served by the Ledger API (#5029)
* Write transactions ready to be served by the Ledger API

CHANGELOG_BEGIN
CHANGELOG_END

* Address https://github.com/digital-asset/daml/pull/5029#discussion_r393192752

* Address https://github.com/digital-asset/daml/pull/5029#discussion_r393191059

* Address https://github.com/digital-asset/daml/pull/5029#discussion_r393201832

* Address https://github.com/digital-asset/daml/pull/5029#discussion_r393492987
2020-03-17 10:36:44 +01:00
Robert Autenrieth
970fa3dbdb
Remove monotonic record time assumption (#5026)
* Remove monotonic record time assumption

CHANGELOG_BEGIN
CHANGELOG_END

* Improve wording
2020-03-17 09:30:36 +01:00
fabiotudone-da
8d8d5dbb27
Add a --version cmdline option to test-tool. (#4996)
* Add a `--version` cmdline option to test-tool.

CHANGELOG_BEGIN
[TestTool] Add a --version command line option to test tool.
CHANGELOG_END

* 🐛 Fix compilation and runtime errors

* Add copyright header

* Fix formatting

* Rebase upon (merged) https://github.com/digital-asset/daml/pull/5004

* Remove unneeded `//:MVN_VERSION` resource

* Remove unneeded maven build config
2020-03-16 22:51:42 +01:00
Moritz Kiefer
3da5e4fae7
Give repl tests a long timeout (#5034)
Apparently adding the TLS and auth tests has made this necessary now.

changelog_begin
changelog_end
2020-03-16 20:43:30 +00:00
Leonid Shlyapnikov
acb41c5829
Fetch by parties endpoint to use new getParties gRPC call (#5027)
* fetch by parties endpoint to use new getParties gRPC call

changelog_begin
changelog_end

* addressing code review comments, thanks @S11001001 and @SamirTalwar-DA

* Update ledger/ledger-api-client/src/main/scala/com/digitalasset/ledger/client/services/admin/PartyManagementClient.scala

thanks @S11001001

Co-Authored-By: Stephen Compall <stephen.compall@daml.com>

* addressing code review comments, thanks @S11001001 and @SamirTalwar-DA

* addressing code review comments, thanks @S11001001 and @SamirTalwar-DA

Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2020-03-16 16:42:01 -04:00
Samir Talwar
b9acad2c6d
kvutils: Don't supply execution contexts to ResourceOwners. (#5010)
* kvutils: Remove the unnecessary execution context from the test base.

* kvutils: Remove the unnecessary execution context from the writer.

* ledger-on-sql: Make a proper owner so it has a proper execution context.

This means the parallelization now needs to come from the test, so I've
augmented ParticipantStateIntegrationSpecBase to take a proper execution
context instead of the serial one that ScalaTest provides, with a
default of `ExecutionContext.global`.

* ledger-on-memory: Make a proper owner with a proper execution context.

* kvutils/app: Remove `executionContext` from LedgerFactory.

Shouldn't need it in `ResourceOwner`. I was bad.

CHANGELOG_BEGIN
CHANGELOG_END

* ledger-on-memory: Make ResourceOwners real classes.

* ledger-on-sql: Make the ResourceOwner a real class.

* ledger-on-sql: Cause side effects on resource acquisition.

Not on owner construction.
2020-03-16 19:05:23 +00:00
Samir Talwar
ad3c3b326b
sandbox: Make sure we can get the details of an empty list of parties. (#5030)
This would fail only on PostgreSQL because `IN ()` is invalid. H2 seems
to be fine with it.

CHANGELOG_BEGIN
- [Ledger API Server] Support a call to `GetParties` with an empty list
  of parties.
CHANGELOG_END
2020-03-16 18:39:41 +00:00
Remy
04196597a7
forbid non-generic comparison in 1.dev (#5020)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-16 17:25:43 +00:00
Miklos
04a0723c81
Switch from Array[Byte] to ByteString on simplified ledger integration API (#4984)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-16 18:02:50 +01:00