Commit Graph

99 Commits

Author SHA1 Message Date
Hubert Slojewski
cb422e2f67
KVL-810 Report the correct gRPC error code on failures due to concurrent transactions (#9218)
* Fix gRPC status codes for inconsistency rejections and DamlLf errors
Also, add unit tests and exclude failing compatibility and conformance tests

CHANGELOG_BEGIN

- [Integration Kit] Fix gRPC status codes for inconsistency rejections and DamlLf errors (ContractNotFound, ReplayMismatch) by changing them from INVALID_ARGUMENT to ABORTED

CHANGELOG_END
2021-03-26 15:26:33 +01:00
nickchapman-da
9b2158508b
Add new variant to Value.scala for builtin-exceptions. (#9084)
* Add new variant to Value.scala for builtin-exceptions.

final case class ValueBuiltinException[+Cid](tag: String, value: Value[Cid]) extends Value[Cid]

And push through the code consequences.
Most places fixed up.
A couple more things to do in this PR (marked NICK)
A couple of things which can be left for later (marked 8020)

fix build

fix another scala match

changelog_begin
changelog_end

* fix any match

* add marker of code which needs attending to in the PR

* extend ledger-api value.proto & fix LfEngineToApi

* undo/comment-out the change to value.proto

* add tests in HashSpec for BuiltinException

* code but dont yet enable value-gen for builtin exceptions

* address comments which suggest we crash in various places

* support BuiltinException in scenario_service.proto

* one more TODO 8020 tag
2021-03-16 13:15:33 +00:00
Moritz Kiefer
5d91a25306
Disable deriving-defaults per file (#8684)
Disabling it per target works nicely for compilation but it gets
annoying in ghci since the warnings are still triggered. We could
disable it everywhere but I think the warning is generally useful. I
tried patching proto3-suite to use DerivingStrategies but that doesn’t
work because haskell-src is dead and doesn’t support it. So for now
adding it to the per-file list seems like the best option.

changelog_begin
changelog_end
2021-01-29 11:01:55 +00:00
Moritz Kiefer
c89e00342d
Clean broken entries from the Bazel cache (#8668)
* Clean broken entries from the Bazel cache

This is hopefully a somewhat reasonable workaround for the "output not
created" errors that keep annoying us.

For now, this is just part of the hourly cronjob but we could move it
somewhere else if desired.

changelog_begin
changelog_end

* Fix GCS credentials

changelog_begin
changelog_end
2021-01-28 17:57:09 +00:00
Stefano Baghino
793fd8fdfc
Correctly document and report malformed party names when allocating (#8642)
* Correctly document and report malformed party names when allocating

changelog_begin
[Ledger API] Documented the hard-coded limit of 255 characters for Daml-LF party names.
[Ledger API] Malformed party names are now correctly reported back with an INVALID_ARGUMENT error
changelog_end

* Make sure that what is given is a suggestion

Co-authored-by: Robert Autenrieth <31539813+rautenrieth-da@users.noreply.github.com>

* Use Future.successful where possible

Co-authored-by: Robert Autenrieth <31539813+rautenrieth-da@users.noreply.github.com>

* Address https://github.com/digital-asset/daml/pull/8642#discussion_r565116571

* Address https://github.com/digital-asset/daml/pull/8642#discussion_r565122362

* Keep fields private where possible

* Whitelist new tests on compatibility suite

* Don't run the new tests on Canton

Co-authored-by: Robert Autenrieth <31539813+rautenrieth-da@users.noreply.github.com>
2021-01-27 18:50:19 +00:00
Richard Kapolnai
03a1fd960e
change comment of listknownparties: known instead of hosted (#8632)
* change comment of listknownparties: known instead of hosted

* CHANGELOG_BEGIN
CHANGELOG_END
2021-01-26 14:30:35 +00:00
Moritz Kiefer
e0c5abd647
Switch to GHC 8.10.3 (#8394)
* Switch to GHC 8.10.3

changelog_begin
changelog_end

* Update bazel-haskell-deps.bzl

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>

* Comment on rules_haskell patch

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
2021-01-25 11:53:53 +00:00
Bernhard Elsner
cda93db944
Daml case and logo (#8433)
* Replace many occurrences of DAML with Daml

* Update docs logo

* A few more CLI occurrences

CHANGELOG_BEGIN
- Change DAML capitalization and docs logo
CHANGELOG_END

* Fix some over-eager replacements

* A few mor occurrences in md files

* Address comments in *.proto files

* Change case in comments and strings in .ts files

* Revert changes to frozen proto files

* Also revert LF 1.11

* Update get-daml.sh

* Update windows installer

* Include .py files

* Include comments in .daml files

* More instances in the assistant CLI

* some more help texts
2021-01-08 12:50:15 +00:00
Gary Verhaegen
a925f0174c
update copyright notices for 2021 (#8257)
* update copyright notices for 2021

To be merged on 2021-01-01.

CHANGELOG_BEGIN
CHANGELOG_END

* patch-bazel-windows & da-ghc-lib
2021-01-01 19:49:51 +01:00
Moritz Kiefer
a8776b0ba1
Fix documentation for InclusiveFilters (#8322)
As discussed in
https://discuss.daml.com/t/registeredtemplates-behaves-as-allindar/1789/5?u=cocreature
the current behavior doesn’t match the docs. There doesn’t seem a good
usecase to change the existing behavior so adapting the docs seems
more sensible.

changelog_begin
changelog_end
2020-12-17 08:33:35 +01:00
Ognjen Maric
b535c33e4d
Document the tx service behavior for filters and transient contracts (#8016)
Document the tx service behavior for filters and transient contracts

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-09 14:48:39 +01:00
Robert Autenrieth
56f5cf107b
Prepare the ledger API for multi-party submissions [KVL-710] (#8125)
* Add multi-party submissions to the ledger API

CHANGELOG_BEGIN
* [Ledger API] Command submission requests now contain new optional
  fields used for multi-party submissions. Such submissions currently
  return UNIMPLEMENTED errors, they will be enabled in the future.
CHANGELOG_END

* Adapt Haskell bindings

* Handle the new ledger API fields

* Fix SubmitAndWaitDummyCommand

* Remove unused methods

* Redesign multi-party auth tests

* Remove direct access of request party

* Apply review comments

* Improve protobuf comments

* Multi-party tracker map

* Fix validation logic

* Consistent metric naming
2020-12-07 16:17:05 +01:00
Stefano Baghino
d3899368ff
Document offset inclusivity and exclusivity in Protobuf (#8145)
* Document offset inclusivity and exclusivity in Protobuf

As brought up in https://discuss.daml.com/t/using-offsets/1731, the
inclusivity and exclusivity of start and end offsets as exposed by
the Ledger API is not well documented.

This commit clearly states which offset is inclusive and which is
exclusive on all relevant endpoints (command completion and
transaction services).

changelog_begin
changelog_end

* Address https://github.com/digital-asset/daml/pull/8145#pullrequestreview-543758867
2020-12-03 16:00:50 +01:00
Samir Talwar
93acf1e6cd
Bazel: Reduce visibility of Protobuf JARs and accoutrements. (#8121)
* Bazel: Reduce the visibility of some Protobuf targets.

* Bazel: Make Protobuf JAR visibility parameterizable.

Default to private, and explicitly make it public where it's needed.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-01 14:00:44 +00:00
Samir Talwar
4d1cf41adf
Bazel: Simplify the implementation of proto_gen. (#8120)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-01 10:36:10 +00:00
Samir Talwar
bdc9679cf4
ledger-api: Remove unused imports from the Proto files. (#8095)
CHANGELOG_BEGIN
CHANGELOG_END
2020-11-27 17:17:21 +00:00
Samir Talwar
052f69cde9
ledger-api: Use proto_jars, and publish Protobuf sources separately from the Scala classes. [KVL-714] (#8091)
* ledger-api: Use `proto_jars`.

CHANGELOG_BEGIN
- [Ledger API] The Scala JARs containing the gRPC definitions no longer
  contain the *.proto files used to generate the ScalaPB-based classes.
CHANGELOG_END

* Create a source JAR for *.proto files in `proto_jars`.

* ledger-api: Publish the protobuf sources as "ledger-api-proto".

CHANGELOG_BEGIN
- [Ledger API] The *.proto files containing the gRPC definitions are now
  provided by a new Maven Central artifact, with the group "com.daml"
  and the artifact name "ledger-api-proto".
CHANGELOG_END

* release: We don't need the "main-jar" option.

* Bazel: Proto JARs will always have a Maven artifact suffix.

* Bazel: Simplify Protobuf source file TAR and JAR targets.

* Bazel: Extract out Protobuf functions.
2020-11-27 17:14:48 +00:00
Oliver Seeliger
83f11aa784
Participant pruning ledger api server support ported from Canton (#7988)
* Participant pruning ledger api server support ported from Canton

CHANGELOG_BEGIN
- [Ledger API]: The preview of `ParticipantPruningService` enables ledger participants to prune the "front" of ledger state at the participant including the ledger api server index.
CHANGELOG_END

* Review feedback from Stefano

* Add pruning tests plus missed command completions change

* Review feedback from Robert

* Improved test readability by having populate helper return offsets

* Review feedback

* Ledger api changes to pruning api and disable canton pruning test

- Change return result to PruneResponse
- Change type of PruneRequest.prune_up_to to string

* Review feedback: Use ApiOffsetSConverter for logged offsets
2020-11-23 15:28:04 +01:00
Kamil Bożek
efa263a717
Documented gRPC error codes returned by the API server (#7844)
CHANGELOG_BEGIN
- gRPC error codes documentation in .proto definitions
CHANGELOG_END
2020-11-05 16:12:43 +01:00
Iggy
0ad6e9d379
Include .proto files in ScalaPB source jar (#7831)
* Include proto files in ScalaPB source jar

* Include proto files in ScalaPB jar

* remove empty line

* use ledger_api_proto_source_root variable

* Fix formatting
CHANGELOG_BEGIN
- Added protobuf source files to ledger-api-scalapb artifact
CHANGELOG_END
2020-10-29 09:14:31 +00:00
Kamil Bożek
9a2317cbdb
API version service [KVL-541] (#7792)
This change exposes a new endpoint with information about the ledger API version.
The current approach is to read contents of ledger-api/VERSION and provide it as it is to the user.

CHANGELOG_BEGIN
- ledger API version endpoint
CHANGELOG_END
2020-10-27 17:47:44 +01:00
Andreas Herrmann
8e706a9bde
Remove vendored pkg_tar (#6934)
* Use Bazel builtin pkg_tar rule

* Use @rules_pkg//:pkg.bzl%pkg_tar

The pkg_tar rule builtin to Bazel has been deprecated.
See https://docs.bazel.build/versions/master/be/pkg.html

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-07-30 15:53:16 +00:00
Moritz Kiefer
674c37cf5a
Stop publishing a tarball with the protobuf files to Maven (#6742)
This was never intentional, nobody even knew that this was possible
and we have an alternative, documented way of getting this via github
releases.

To avoid introducing this issue again, I’ve removed non-jar artifact
types from the Maven upload script.

fixes #448

changelog_begin
changelog_end
2020-07-15 18:48:49 +02:00
Gary Verhaegen
8176fb0c8d
fix the the typo (#6723)
```
s/the the /the /
```

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-14 16:55:33 +00:00
Stefano Baghino
6bab178195
Improve explanation of offsets on the protobuf docs (#6465)
* Improve explanation of offsets on the protobuf docs

changelog_begin
[Documentation] More detailed explanation of how to use (and not use) offsets in the Ledger API reference docs
changelog_end

* Address https://github.com/digital-asset/daml/pull/6465#discussion_r444200217
2020-06-23 14:11:07 +00:00
Stefano Baghino
234319db2e
Document required protobuf definitions for bindings (#5664)
* Document required protobuf definitions for bindings

changelog_begin
[Documentation] Documented required protobuf definitions to build custom bindings
changelog_end

* Address review comments

- remove out-of-date document on errors
- add README.md to tarball, address https://github.com/digital-asset/daml/pull/5664#discussion_r412803050
- remove unnecessary reference to reflection service, address https://github.com/digital-asset/daml/pull/5664#discussion_r412810629
2020-04-22 10:27:23 +00:00
Gerolf Seitz
a2d785e3ee
Use com.daml as root package (#5343)
Packages com.digitalasset.daml and com.daml have been unified under com.daml

Ledger API and DAML-LF DEV protos have also been moved from `com/digitalasset`
to `com/daml` on the file system.
Protos for already released DAML LF versions (1.6, 1.7, 1.8) stay in the
package `com.digitalasset`.

CHANGELOG_BEGIN
[SDK] All Java and Scala packages starting with
``com.digitalasset.daml`` and ``com.digitalasset`` are now consolidated
under ``com.daml``. Simply changing imports should be enough to
migrate your code.
CHANGELOG_END
2020-04-05 19:49:57 +02:00
Gerolf Seitz
329320bad9
Organize maven coordinates (#5272)
* Use com.daml as groupId for all artifacts

CHANGELOG_BEGIN
[SDK] Changed the groupId for Maven artifacts to ``com.daml``.
CHANGELOG_END

* Add 2 additional maven related checks to the release binary

1. Check that all maven upload artifacts use com.daml as the groupId
2. Check that all maven upload artifacts have a unique artifactId

* Address @cocreature's comments in https://github.com/digital-asset/daml/pull/5272#pullrequestreview-385026181
2020-04-01 11:41:18 +02:00
Robert Autenrieth
f8cd628c2f
Remove old time model from ledger config (#5216)
* Remove old time model from ledger config

CHANGELOG_BEGIN
- [Ledger API] Fields related to the old ledger time
  model have been removed from the configuration
  management service and the ledger configuration service.
CHANGELOG_END

* Update ledger/ledger-api-test-tool/src/main/scala/com/daml/ledger/api/testtool/tests/LedgerConfigurationService.scala

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-03-30 08:46:22 +02:00
Gary Verhaegen
1872c668a5
replace DAML Authors with DA in copyright headers (#5228)
Change requested by Manoj.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-27 01:26:10 +01:00
Robert Autenrieth
a3ddde3a9d
Remove MRT and LET (#5172)
Contributes to #4194.
Closes #4231.
Closes #5022.
CHANGELOG_BEGIN
- [Ledger API] The protobuf fields ledger_effective_time and maximum_record_time have been removed from
  command submission. These fields were previously deprecated following the introduction
  of a new ledger time model. See issue `#4194 <https://github.com/digital-asset/daml/issues/4194>`__.
[Java Bindings] removed the usage of ledgerEffectiveTime and
maximumRecordTime, and instead added minLedgerTimeAbsolute and
minLedgerTimeRelative in CommandSubmissionClient and CommandClient
CHANGELOG_END
2020-03-25 16:50:27 +01:00
Robert Autenrieth
3f597aae16
New ledger time (#5100)
* Tighten result type

Command execution can't result in a sequencer error

* New helper method for extracting used contracts

* New error clause

* Add a DAO query for the maximum time of contracts

* Implement algorithm for finding ledger time

CHANGELOG_BEGIN
CHANGELOG_END

* fixup ledgerTimeHelper

* Use new ledger time algorithm

* Mark LET/MRT as deprecated

CHANGELOG_BEGIN
- [Ledger API] DAML ledgers have switched to a new ledger time model.
  The ledger_effective_time and maximum_record_time fields of command submission are deprecated,
  the ledger time of transactions is instead set automatically by the ledger API server.
  Ledger time is no longer strictly monotonically increasing, but only follows causal monotonicity:
  ledger time of transactions is greater than or equal to the ledger time of any used contract.
  See `#4345 <https://github.com/digital-asset/daml/issues/4345>`__.
CHANGELOG_END

* Add ledger time skew check

* Remove command updater

LET/MRT are now deprecated, this class is now useless

* Remove old time model validator

* Switch to new time model check: kvutils

* Switch to new time model check: in-memory ledger

* Switch to new time model check: SqlLedger

* Use initial ledger config

* Ignore user provided LET

* Use TimeProvider in submission services

* Use deduplication_time in daml-script runner

- Also remove unnecessary command completion output of CommandTracker.
- Remove usage of maximum record time in CommandTracker.

* Use arbitrary default value for deduplication time

* Use built-in Instant ordering

* Remove obsolete test

* Remove obsolete test: CommandStaticTimeIT

* Refactor test: TransactionMRTCompliance

* Disable test: CommandTrackerFlow timeout

* thread maxDeduplicationTime through to CommandTracker

* Improve test

* Refactor command client configuration

* Deduplication time should always use UTC

* Add missing method in TimedIndexService after rebase

* Put more details into the deduplication error response.

* Use system time for command dedup submittedAt.

* Use explicit UTC time source in command validator

* Revert CommandTracker[Flow] to previous completion-recovering-behavior

* Adapt scala client command config to new config params

Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-03-25 09:28:56 +01:00
Robert Autenrieth
60a845ec72
Add new protobuf fields for new ledger time model (#5036)
* Add new protobuf fields for new ledger time model

Fixes #4344.

CHANGELOG_BEGIN
CHANGELOG_END

* Adapt haskell bindings
2020-03-19 13:53:49 +01:00
Gerolf Seitz
cf032a02cf
Introduce Stable Offsets (#4953)
A "stable offset" in the context of the Participant Server is the offset
that was provided by the ledger backend (be it kvutils, corda, daml on sql).

The Participant Server does not keep a participant-local offset anymore.
In a single domain/kvutil setup, this makes offsets stable across participants,
since all participants will see the same offset for the same transaction.

The following changes were needed to achieve this:
- The participant server always uses the offset provided by the backend
  AS IS (no more +1 magic).
- Offsets provided to the Ledger API in requests must be treated as
  startExclusive and endInclusive (previously beginInclusive and
  endExclusive).

CHANGELOG_BEGIN
[Ledger API]: Offsets have been redefined. Instead of being represented
by a number or a structured string, an offset is now an opaque string
that can be compared lexicographically.
[DAML Integration Kit]: The bounds for ``Dispatcher`` are now
startExclusive and endInclusive.
CHANGELOG_END

---------
ledger api:
ledger_offset.proto
  Changed definition of offsets, since they can now be compared
  lexicographically.

---------
participant-state-api:
Offset:
  Changed from Array[Long] to ByteString. Ledgers need to make sure that the
  offsets produced are strictly monotonically increasing according to
  lexicographical order.

---------
akka-streams:
Dispatcher, DispatcherImpl, SubSource:
  Changed interval handling to exclusive/inclusive.

---------
ledger-on-memory:
InMemoryLedgerReaderWriter, InMemoryState:
  Changed interval handling to exclusive/inclusive.

---------
ledger-on-sql:
CommonQueries, SqlLedgerReaderWriter:
  Change interval in query and boundary handling.

---------
kvutils:
KeyValueParticipantStateReader, KVOffset:
  Convenience functions for kvutils to add or remove sub-indexes for
  offsets.
  KV ledger implementations can use KVOffset to construct a structured offset.

---------
Participant Server:
JdbcLedgerDao:
  Use Offset instead of Long.
  Fetch offsets directly as Offset from the database with proper anorm
  integration.
  Change interval handling to exclusive/inclusive.

CommandCompletionsReader, CommandCompletionsTable:
  Change interval handling to exclusive/inclusive.

BaseLedger:
  Use Offset instead of Long.
  Change interval handling to exclusive/inclusive.

Conversions:
  Anorm integration for using Offset in queries and result parsers.

JdbcIndexer:
  Remove references to "extenalLedgerEnd" and participant-local Long
  offset (headRef).

---------
sandbox:
In general:
  Use the Offset type everywhere instead of Long.

SQL migrations:
  Change all offset columns to bytea or BINARY.

LedgerBackedIndexService:
  Proper bounds checking has been pushed down to Dispatcher, which
  allowed simplifying the acceptedTransactions implementation.

InMemoryLedger, LedgerEntries:
  Change interval handling to exclusive/inclusive.
  Transaction lookup by ID is now O(n) because transaction IDs are not
  necessarily the same as the offset.

SqlLedger:
  Remove external offset references.
2020-03-18 08:43:41 +01:00
Stefano Baghino
5298ac8690
Fix CreatedEvent Protobuf documentation (#4916)
CHANGELOG_BEGIN
[Documentation] Ledger API documentation clarifies how witness_parties
are determined depending on whether CreatedEvents are served as part of
the flat or full transaction stream.
CHANGELOG_END
2020-03-10 11:19:59 +00:00
Samir Talwar
0f4fb9a8f5
Ledger API Server: Add the GetParties endpoint. (#4888)
* sandbox: Add a database test for storing and retrieving parties.

* sandbox: Add database queries for selecting one or many parties.

* ledger-api-test-tool: Add a test for `ListKnownParties`.

* sandbox: Add an endpoint to retrieve a single party's details.

CHANGELOG_BEGIN
- [Ledger API] Added an endpoint to retrieve a single party's details at
  ``com.digitalasset.ledger.api.v1.admin.PartyManagementService.GetParty``.
  Please consult the ledger API reference documentation for more
  information.
CHANGELOG_END

* sandbox: Add an endpoint to retrieve a multiple parties' details.

CHANGELOG_BEGIN
- [Ledger API] Added an endpoint to retrieve multiple parties's details at
  ``com.digitalasset.ledger.api.v1.admin.PartyManagementService.GetParties``.
  Please consult the ledger API reference documentation for more
  information.
CHANGELOG_END

* sandbox: Getting a single party is a special case of multiple parties.

So let's use that code path and stop duplicating work.

* sandbox: Remove `GetParty`, as it's subsumed by `GetParties`.

"Subsumed" is a great word.
2020-03-09 12:09:56 +00:00
Robert Autenrieth
98095cf80d
Clean up command deduplication (#4801)
* Always return error on duplicate submissions

* Remove unnecessary submission information

Now that duplicate submissions always return an error,
we don't need to store the original submission result.

CHANGELOG_BEGIN
CHANGELOG_END

* Rename ttl to deduplicationTime/deduplicateUntil

* Store absolute deduplicateUntil in domain commands

* Fix my own initials

* Remove CommandDeduplicationEntry

Instead, use CommandDeduplicationResult everywhere,
removing the extra layer.
2020-03-04 14:08:08 +01:00
Robert Autenrieth
79e7ca0627
Implement new command deduplication (#4467)
* Add TTL field to protobuf

* Add command deduplication to index service

* Wire command deduplication to DAO

* Implement in-memory command deduplication

* Remove Deduplicator

* Implement JDBC command deduplication

* Add TTL field to domain commands

* Deduplicate commands in the submission service

CHANGELOG_BEGIN
- [Sandbox] Implement a new command submission deduplication mechanism
  based on a time-to-live (TTL) for commands.
  See https://github.com/digital-asset/daml/issues/4193
CHANGELOG_END

* Remove unused command service parameter

* fixup protobuf

* Add configuration for TTL

* Fix Haskell bindings

* Rename SQL table

* Add command deduplication test

* Redesign command deduplication queries

* Address review comment

* Address review comment

* Address review comments

* Make command deduplication test optional

* Disable more tests

* Address review comments

* Address review comments

* Refine test

* Address review comments

* scalafmt

* Truncate new table on reset

* Store original command result

* Rename table columns

... to be consistent with other upcoming tables

* Rename migrations to solve conflicts

Fixes #4193.
2020-02-26 12:00:02 +01:00
Moritz Kiefer
d68d197948
Disable scaladoc on the MacOS CI (#4524)
* Disable scaladoc on the MacOS CI

It is still built by default locally.

fixes #4441

changelog_begin
changelog_end

* Change tag name
2020-02-14 14:08:24 +00:00
Gerolf Seitz
0b718b2657
Remove empy lines in proto docs to fix rendering on docs.daml.com (#4498)
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-14 10:19:02 +01:00
Andreas Herrmann
f33e79c787
Remove unused dependencies to da_scala_library (#3938)
* Inline all scala_library dependencies

* Run //:buildifier-fix

* TMP scala_library_suite --> scala_library

* da_scala_library: Enable unused dependency checker

* scala_library: Enable unused dependency checker

* //daml-lf/data:data

* //daml-lf/engine:engine

* //ledger-api/rs-grpc-akka:rs-grpc-akka

* //ledger/participant-state:participant-state

* //ledger/ledger-api-client:ledger-api-client

* //scala-protoc-plugins/scala-logging:scala-logging-lib

* //ledger/ledger-api-scala-logging:ledger-api-scala-logging

* //ledger/ledger-api-common:ledger-api-common

* //ledger-service/utils:utils

* //ledger-service/jwt:jwt

* //ledger/ledger-api-auth:ledger-api-auth

* //extractor:extractor

* //daml-assistant/scala-daml-project-config:scala-daml-project-config

* //language-support/codegen-common:codegen-common

* //language-support/scala/codegen:codegen

* //language-support/codegen-main:codegen-main-lib

* //ledger-service/db-backend:db-backend

* //ledger-service/http-json:http-json

* //daml-lf/scenario-interpreter:scenario-interpreter

* //ledger/sandbox:sandbox

* //navigator/backend:navigator-library

* //daml-assistant/daml-sdk:sdk-lib

* //daml-lf/data-scalacheck:data-scalacheck

* //daml-script/test:test-lib

* //ledger/ledger-api-common:ledger-api-common-scala-tests-lib

* //ledger/test-common:test-common

* //ledger/sandbox:sandbox-scala-tests-lib

* //extractor:extractor-scala-tests-lib

* //language-support/java/bindings:bindings-java-tests-lib

* //language-support/java/bindings-rxjava:bindings-java-tests-lib

* //language-support/scala/bindings-akka-testing:bindings-akka-testing

* //language-support/scala/codegen-testing:codegen-testing

* //language-support/scala/codegen-sample-app:daml-lf-codegen-sample-app

* //language-support/scala/codegen-sample-app:daml-lf-codegen-sample-app-testing

* //language-support/scala/codegen-testing:codegen-testing-testing

* //ledger-api/sample-service:sample-service

* //ledger-api/rs-grpc-akka:rs-grpc-akka-tests-lib

* //ledger/ledger-api-test-tool:ledger-api-test-tool-lib

* //ledger/ledger-api-test-tool:ledger-api-test-tool-tests

* //ledger/participant-state/kvutils:kvutils

* //ledger/sandbox:ledger-api-server

* //ledger/sandbox-perf:sandbox-perf-lib

* //navigator/backend:navigator-tests-library

* UNDO scala_library_suite --> scala_library

This reverts commit ab3eb1ae23139e2ec431ab4551fbb0371e0354e1.

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-01-06 18:14:21 +01:00
Gary Verhaegen
878429e3bf
update copyright notices to 2020 (#3939)
copyright update 2020

* update template
* run script: `dade-copyright-headers update .`
* update script
* manual adjustments
* exclude frozen proto files from further header checks (by adding NO_AUTO_COPYRIGHT files)
2020-01-02 21:21:13 +01:00
Jussi Mäki
33c7a1aace Implementation for the configuration management service (#3756)
* Implementation for the configuration management service

- Add configuration generation to the response of SetTimeModel
- Implement the ConfigManagementService
- Implement integration test into test tool

This is still a draft as it has unsolved FIXMEs and
it conflicts with #3744 which should go first.

The main conflict is with changes to PartyAllocationResponse which cannot
no longer reference "domain.PartyDetail" as we would have a cyclic dependency
(participant-state contains Configuration which we point to from domain.ConfigurationEntry).

The still open issues are:
- Revisit PartyDetail
- Naming: LedgerConfigurationService and ConfigManagementService are not talking
  about the same configuration and it feels confusing.
- Remove duplication of ConfigurationEntry? Do we need both domain.ConfigurationEntry
  and ledger.store.ConfigurationEntry? Only difference is in the types of participantId
  and submissionId.

* Address review part 1

* Fix up tests after rebase and address PR review

* Post-merge fixes

* Add missing config MRT checks and fixes to tests

- Check config MRT in InMemoryLedger and SqlLedger
- Use proper source of time in ConfigManagement
- Separate out ConfigManagementServiceIT in sandbox conformance tsets

* Reformat
2019-12-13 17:58:44 +00:00
Jussi Mäki
ce70ad4a2f
Async package management (#3806)
* Add package_entries table

* Change PublicPackageUpload event to cover list of packages.

Add PublicPackageUploadRejected.

* Produce new package update events in KeyValueConsumption

* Update signature of uploadPackages

* Cleanup InMemoryKVParticipantState. Add submissionId to uploadPackages.

* Fix up InMemoryKVParticipantStateIT

* Initial ledger dao changes for package entries

Drop the participant_id as we never expect to see
entries of other participants. This should be done
for party_entries as well.

* Drop UploadPackagesResult

* Implement getPackageEntries and refactor callers

* Add maxRecordTime to uploadPackages

* First cut at updating ApiPackageManagementService

* Update tests, wire through the packageEntries

* Don't extend IndexPackagesService in InMemoryPackageStore

It does not implement the full interface and it isn't used
directly as one anyway.

* Drop maximum_record_time from package_management_service

Adding maximum record times touches the whole stack. Leaving
this change to another PR.

* Wire through the removal of maximum_record_time.

And remove dead code from InMemoryKVParticipantState

* Remove checking for duplicate package uploads

This aligns with the behaviour of WriteService.

* Reformat

* Fix PackageManagementService after adding of submission_id to the service
2019-12-13 09:56:47 +01:00
Jussi Mäki
f998887ad7
Draft of config management service (#3656)
* Add draft of config management service

* Add submission id to SetTimeModelRequest
2019-12-05 10:25:58 +01:00
Andreas Herrmann
f4d0eb636e Replace Hazel by stack_snapshot (#2743)
* Unmangled libz.so and libbz2.so

* Use stack_snapshot instead of Hazel

* Remove Hazel

* Define stack_snapshot

* Update rules_haskell

* Document stack_snapshot

* Clean stack's lock file from aborted builds
2019-11-22 14:24:08 +00:00
Samir Talwar
0ff716df2a Ledger API: Add healthcheck endpoints. (#3573)
* grpc-definitions: Delete health_service.proto

We can use the version in io.grpc:grpc-services instead.

* ledger: Delete ledger/API.md.

* sandbox: Fix warnings in ApiServices flagged by IntelliJ.

* sandbox: Implement a dummy grpc.health.v1.Health.Check endpoint.

* sandbox: Implement a dummy grpc.health.v1.Health.Watch endpoint.

* sandbox: Drop repeated elements from grpc.health.v1.Health.Watch.

* sandbox: Wrap the HealthService in basic tests.

* sandbox: Stop streaming the server health too.

* ledger-api-test-tool: Health check tests.

* Add a changelog entry for the health check endpoints.

CHANGELOG_BEGIN

- [Ledger API] Add healthcheck endpoints, conforming to the
  `GRPC Health Checking Protocol <https://github.com/grpc/grpc/blob/master/doc/health-checking.md>`_.
  It is always ``SERVING`` for now.

- [DAML Ledger Integration Kit] Add conformance test coverage for the
  ``grpc.health.v1.Health`` service.

CHANGELOG_END

* ledger-api-integration-tests: Increment the number of services.

* Apply suggestions from code review

Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* sandbox: Use `AkkaBeforeAndAfterAll` in the HealthServiceSpec.

In an attempt to get it working on CI.

* sandbox: Change `dropRepeated` to `DropRepeated()`.

Keep it in one file.

* test-common: Use `Delayed.by` in `TimeBoundObserver`.

* test-common: Close the source when `TimeBoundObserver` completes.

* ./fmt.sh

That'll teach me not to `--no-verify` just because it's a merge commit.

* sandbox: Inline `HealthService.suppress`.

At some point it was being used twice.

* sandbox: Increase the timeout for HealthServiceSpec.

* sandbox: Reimplement HealthService using the Scala protobuf types.

* sandbox: Generate an Akka-compatible trait for the health service.

And refactor a lot of test code to make it easy to test.

* ledger-api-common: Move the HealthService here.

* rs-grpc-testing-utils: Publish to Maven.

* rs-grpc-testing-utils: Add Maven coordinates.
2019-11-22 14:02:05 +00:00
Remy
fedc49dd12 leger-api: add GenMap support (#3506)
* leger-api: add GenMap support

* cosmetic

* Address Gerolf's  comment

* add a point in the release notes

* Address Fran's comments
2019-11-18 18:13:25 +00:00
Andreas Herrmann
33e47828e3
Bazel 1.1 (#3249)
* bazel: 0.28.1 --> 1.1.0

* bazel-watcher sha256

* Fix missing line in patch

* proto_source_root --> strip_import_prefix

See https://github.com/bazelbuild/bazel/issues/7153 for details.

* Update rules_nixpkgs

Required to avoid errors of the form
```
ERROR: An error occurred during the fetch of repository 'node_nix':
   parameter 'sep' may not be specified by name, for call to method split(sep, maxsplit = None) of 'string'
```

and
```
ERROR: An error occurred during the fetch of repository 'node_nix':
   Traceback (most recent call last):
	File "/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 149
		_execute_or_fail(repository_ctx, <3 more arguments>)
	File "/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 318, in _execute_or_fail
		fail(<1 more arguments>)

Cannot build Nix attribute 'nodejs'.
Command: [/Users/runner/.nix-profile/bin/nix-build, /private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/node_nix/nix/bazel.nix, "-A", "nodejs", "--out-link", "bazel-support/nix-out-link", "-I", "nixpkgs=/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/nixpkgs/nixpkgs"]
Return code: 1
Error output:
src/main/tools/process-tools.cc:173: "setitimer": Invalid argument
```

* Update rules_scala

* .proto has been removed, use [ProtoInfo] instead

See
https://docs.bazel.build/versions/1.1.0/be/protocol-buffer.html#proto_library

* python3_nix add nix_file attribute

To avoid the following error

```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/BUILD:66:1: //:nix_python3_runtime depends on @python3_nix//:bin/python in repository @python3_nix which failed to fetch. no such package '@python3_nix//': Traceback (most recent call last):
        File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 149
                _execute_or_fail(repository_ctx, <3 more arguments>)
        File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 318, in _execute_or_fail
                fail(<1 more arguments>)

Cannot build Nix attribute 'python3'.
Command: [/home/aj/.nix-profile/bin/nix-build, "-E", "import <nixpkgs> { config = {}; overlays = []; }", "-A", "python3", "--out-link", "bazel-support/nix-out-link", "-I", "nixpkgs=/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/nixpkgs/nixpkgs"]
Return code: 1
Error output:
error: anonymous function at /home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/nixpkgs/nixpkgs.nix:3:1 called with unexpected argument 'config', at (string):1:1
```

* rules_haskell unnamed string.split(_, maxsplit = _)

The keyword argument may no longer be named.

* string.replace(_, _, maxsplit = _) may not be named

* Move proto sources from deps to data

Fixes

```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/daml-lf/archive/BUILD.bazel:150:1: in deps attribute of scala_test rule //daml-lf/archive:daml_lf_archive_reader_tests_test_suite_src_test_scala_com_digitalasset_daml_lf_archive_DecodeV1Spec.scala: '//daml-lf/archive:daml_lf_1.6_archive_proto_srcs' does not have mandatory providers: 'JavaInfo'. Since this rule was created by the macro 'da_scala_test_suite', the error might have been caused by the macro implementation
```

* Define sha256 for haskell_ghc__paths

Bazel 1.1.0 fails on missing hashes.

* Disable --incompatible_windows_native_test_wrapper

* //compiler/daml-extension don't modify sources

Modifying sources in-place can cause issues on Windows, where build
actions are not sandboxed and changes on sources can affect other build
steps.

* bazel-genfiles --> bazel-bin

The bazel-genfiles symlink has been removed since Bazel 1.0.
See https://github.com/bazelbuild/bazel/issues/8651

* Mark dev_env_tool repository rule as configure

See
https://docs.bazel.build/versions/1.1.0/skylark/lib/globals.html#repository_rule

* Move data deps into data attribute

* Mark dev_env_tool as local = True

* Manually fetch @makensis_dev_env
2019-11-11 10:06:03 +01:00
Andreas Herrmann
2bd1db490a
Replace bazel-deps by rules_jvm_external (#3253)
* Update bazel-common to fix javadoc issues

Specifically, to fix the following error

```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel:7:1: in javadoc_library rule //ledger-api/rs-grpc-bridge:rs-grpc-bridge_javadoc:
Traceback (most recent call last):
        File "/home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel", line 7
                javadoc_library(name = 'rs-grpc-bridge_javadoc')
        File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/com_github_google_bazel_common/tools/javadoc/javadoc.bzl", line 27, in _javadoc_library
                dep.java.transitive_deps
object of type 'JavaSkylarkApiProvider' has no field 'transitive_deps'
```

* Define Maven deps using rules_jvm_external

* Pin artifacts

* Remove bazel-deps generated targets

* Remove bazel-deps

* Switch to rules_jvm_external targets

* update bazel documentation

* pom_file: There are no more bazel-deps targets

* BAZEL-JVM.md `maven_install` typo
2019-10-28 13:53:14 +01:00