Commit Graph

128 Commits

Author SHA1 Message Date
Stefano Baghino
c51ecd3c58
Extractor with authentication (#3514)
* Make Extractor work against Ledger API servers protected by authentication

* Update changelog

* Address https://github.com/digital-asset/daml/pull/3514#pullrequestreview-318811500

- https://github.com/digital-asset/daml/pull/3514#discussion_r347753205
- https://github.com/digital-asset/daml/pull/3514#discussion_r347753629
- https://github.com/digital-asset/daml/pull/3514#discussion_r347754462

* Address https://github.com/digital-asset/daml/pull/3514#discussion_r347755484

* Address https://github.com/digital-asset/daml/pull/3514#discussion_r347752424

* Address https://github.com/digital-asset/daml/pull/3514#discussion_r347756328

* Fix docs

* Fix artifacts.yaml
2019-11-19 17:12:25 +01:00
Moritz Kiefer
e4632f5fb2 Include "daml trigger" in "daml --help" (#3528) 2019-11-19 16:19:17 +01:00
Oliver Seeliger
f50901347f
Release lf scalacheck test jars to maven central (#3519)
as they are dependencies used downstream
2019-11-19 13:10:28 +01:00
Gerolf Seitz
5f8bf411bd
Upload proto and deploy jars to maven (#3507)
* Add helper to produce an empty zip files.

This is used to generate empty sources and javadoc jars for
deploy jars later on.

* Create empty auxilliary jars.

da_java_binary:
  - empty javadoc jar
  - empty sources jar

da_java_proto_library:
  - empty javadoc jar
  - the sources jar is automatically generated by java_proto_library as a side effect

da_scala_binary:
  - empty javadoc jar
  - empty sources jar

* Support maven upload for jar-deploy and jar-proto

For jar-deploy targets we don't check for internal dependencies,
because these should already be contained in the (fat-)jar itself.

Additionally, the release program now uploads javadocs and sources
for jar-proto and jar-deploy as well to comply with maven central.

* Upload ledger-api-test-tool and kvutils + dependencies to maven central.

This is the diff running the output of the release without and with these changes.
A few artifacts now also get their javadoc and sources uploaded (mostly to bintray,
but now they are ready for a maven central upload).

ledger-api-test-tool has the scala version removed from the artifact as it is a
deploy jar and nobody should care which specific scala version is used.

Only in release/com/daml/ledger/participant-state-kvutils-java-proto/100.13.35: participant-state-kvutils-java-proto-100.13.35-javadoc.jar
Only in release/com/daml/ledger/participant-state-kvutils-java-proto/100.13.35: participant-state-kvutils-java-proto-100.13.35-sources.jar
Only in release/com/daml/ledger/testtool: ledger-api-test-tool
Only in release-before/com/daml/ledger/testtool: ledger-api-test-tool_2.12
Only in release/com/digitalasset/daml/lf/engine/trigger/runner_2.12/100.13.35: runner_2.12-100.13.35-javadoc.jar
Only in release/com/digitalasset/daml/lf/engine/trigger/runner_2.12/100.13.35: runner_2.12-100.13.35-sources.jar
Only in release/com/digitalasset/daml-lf-blindinginfo-java-proto/100.13.35: daml-lf-blindinginfo-java-proto-100.13.35-javadoc.jar
Only in release/com/digitalasset/daml-lf-blindinginfo-java-proto/100.13.35: daml-lf-blindinginfo-java-proto-100.13.35-sources.jar
Only in release/com/digitalasset/daml-lf-transaction-java-proto/100.13.35: daml-lf-transaction-java-proto-100.13.35-javadoc.jar
Only in release/com/digitalasset/daml-lf-transaction-java-proto/100.13.35: daml-lf-transaction-java-proto-100.13.35-sources.jar
Only in release/com/digitalasset/daml-lf-value-java-proto/100.13.35: daml-lf-value-java-proto-100.13.35-javadoc.jar
Only in release/com/digitalasset/daml-lf-value-java-proto/100.13.35: daml-lf-value-java-proto-100.13.35-sources.jar
Only in release/com/digitalasset/damlc/100.13.35: damlc-100.13.35-javadoc.jar
Only in release/com/digitalasset/damlc/100.13.35: damlc-100.13.35-sources.jar
Only in release/com/digitalasset/extractor/100.13.35: extractor-100.13.35-javadoc.jar
Only in release/com/digitalasset/extractor/100.13.35: extractor-100.13.35-sources.jar
Only in release/com/digitalasset/ledger-service/http-json-deploy/100.13.35: http-json-deploy-100.13.35-javadoc.jar
Only in release/com/digitalasset/ledger-service/http-json-deploy/100.13.35: http-json-deploy-100.13.35-sources.jar
Only in release/com/digitalasset/navigator/100.13.35: navigator-100.13.35-javadoc.jar
Only in release/com/digitalasset/navigator/100.13.35: navigator-100.13.35-sources.jar
2019-11-18 14:40:15 +01:00
Gary Verhaegen
8a024c549f release instructions: add label to PR (#3489) 2019-11-15 15:55:36 +00:00
Gerolf Seitz
172ce90597
Upload codegen fat-jars to maven central (#3476) 2019-11-14 18:40:54 +01:00
Gerolf Seitz
e760d4749f Check internal dependencies of maven artifacts (#3454)
* Check internal dependencies of maven artifacts

All artifacts that are uploaded to maven need to also have
their dependencies uploaded to maven.

* inline tuple matching

* remove unnecessary parenthesis

* mark missing artifacts for maven upload
2019-11-14 14:18:54 +01:00
Andreas Herrmann
33e47828e3
Bazel 1.1 (#3249)
* bazel: 0.28.1 --> 1.1.0

* bazel-watcher sha256

* Fix missing line in patch

* proto_source_root --> strip_import_prefix

See https://github.com/bazelbuild/bazel/issues/7153 for details.

* Update rules_nixpkgs

Required to avoid errors of the form
```
ERROR: An error occurred during the fetch of repository 'node_nix':
   parameter 'sep' may not be specified by name, for call to method split(sep, maxsplit = None) of 'string'
```

and
```
ERROR: An error occurred during the fetch of repository 'node_nix':
   Traceback (most recent call last):
	File "/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 149
		_execute_or_fail(repository_ctx, <3 more arguments>)
	File "/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 318, in _execute_or_fail
		fail(<1 more arguments>)

Cannot build Nix attribute 'nodejs'.
Command: [/Users/runner/.nix-profile/bin/nix-build, /private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/node_nix/nix/bazel.nix, "-A", "nodejs", "--out-link", "bazel-support/nix-out-link", "-I", "nixpkgs=/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/nixpkgs/nixpkgs"]
Return code: 1
Error output:
src/main/tools/process-tools.cc:173: "setitimer": Invalid argument
```

* Update rules_scala

* .proto has been removed, use [ProtoInfo] instead

See
https://docs.bazel.build/versions/1.1.0/be/protocol-buffer.html#proto_library

* python3_nix add nix_file attribute

To avoid the following error

```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/BUILD:66:1: //:nix_python3_runtime depends on @python3_nix//:bin/python in repository @python3_nix which failed to fetch. no such package '@python3_nix//': Traceback (most recent call last):
        File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 149
                _execute_or_fail(repository_ctx, <3 more arguments>)
        File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 318, in _execute_or_fail
                fail(<1 more arguments>)

Cannot build Nix attribute 'python3'.
Command: [/home/aj/.nix-profile/bin/nix-build, "-E", "import <nixpkgs> { config = {}; overlays = []; }", "-A", "python3", "--out-link", "bazel-support/nix-out-link", "-I", "nixpkgs=/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/nixpkgs/nixpkgs"]
Return code: 1
Error output:
error: anonymous function at /home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/nixpkgs/nixpkgs.nix:3:1 called with unexpected argument 'config', at (string):1:1
```

* rules_haskell unnamed string.split(_, maxsplit = _)

The keyword argument may no longer be named.

* string.replace(_, _, maxsplit = _) may not be named

* Move proto sources from deps to data

Fixes

```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/daml-lf/archive/BUILD.bazel:150:1: in deps attribute of scala_test rule //daml-lf/archive:daml_lf_archive_reader_tests_test_suite_src_test_scala_com_digitalasset_daml_lf_archive_DecodeV1Spec.scala: '//daml-lf/archive:daml_lf_1.6_archive_proto_srcs' does not have mandatory providers: 'JavaInfo'. Since this rule was created by the macro 'da_scala_test_suite', the error might have been caused by the macro implementation
```

* Define sha256 for haskell_ghc__paths

Bazel 1.1.0 fails on missing hashes.

* Disable --incompatible_windows_native_test_wrapper

* //compiler/daml-extension don't modify sources

Modifying sources in-place can cause issues on Windows, where build
actions are not sandboxed and changes on sources can affect other build
steps.

* bazel-genfiles --> bazel-bin

The bazel-genfiles symlink has been removed since Bazel 1.0.
See https://github.com/bazelbuild/bazel/issues/8651

* Mark dev_env_tool repository rule as configure

See
https://docs.bazel.build/versions/1.1.0/skylark/lib/globals.html#repository_rule

* Move data deps into data attribute

* Mark dev_env_tool as local = True

* Manually fetch @makensis_dev_env
2019-11-11 10:06:03 +01:00
Remy
ce36674ea0 daml-lf: clone dev protobuf for LF 1.7 (#3371) 2019-11-07 12:32:38 +00:00
Stefano Baghino
a2725415eb Add artifacts publishing for //ledger/ledger-api-grpc-utils (#3370)
* Add artifacts publishing for //ledger/ledger-api-grpc-utils

* Address https://github.com/digital-asset/daml/pull/3370#discussion_r343509550
2019-11-07 08:13:18 +00:00
Stephen Compall
d0ec381925 json-api: store contracts similar to Extractor's single-table (#3192)
* new library ledger-service/db-backend

* borrow contracts table schema from extractor

* borrow contract insertion, removing some data to be unused

* match contract schema with insert function

* factor insertContract arguments

* offset table declarations

* CLI argument for query store

* surrogate template IDs

* compute surrogate template IDs on-the-fly

* database init action

* incoherent typeclasses, eh

* newtype SurrogateTpId

* offset fetch/update functions

* bad sql

* bulk insert contracts, function for selecting contracts

* expose contract column name for query's usage

* Initializing DB on startup if configured

* dropping existing tables as part of initialization

* fix some query syntax errors

* createSchema flag

* function for streaming transactions with jwt party selected

* formatting

* usage

* collect acs contracts and the ledger offset at the end

* lastOffset

* fixing merge conflicts, updating the way 3rd party deps are specified

* Moving ContractDao into http-json module

so it can take domain AST as an input

* cleanup

* injecting new dependencies

* split transaction batches into inserts and deletes

* generate sql for deleting contracts

* `fetch_sources = True` for java_deps

* make the delete-constructed fragment more efficient; handle empty list here

* pass logHandler for insertContracts

* ContractDao returns ConnectionIO, it's up to the caller to wrap query into a transaction

* fixing typo

* minor cleanup, moving fromLedgerApi factory function into corresponding companion objects

* don't need it any more

* GetActiveContractsResponse => domain.Contract factory

* make concatFragment private

* add partition graph; move other contract-fetching experiments to ContractsFetch

* experimenting with akka sources

* introducing domain.Offset to work around API's empty/null offset cases

* minor cleanup

* decompose fetchActiveContractsFromOffset

* missed via

* ACS splitting graph

* finish doc for ACS splitting graph

* remove unneeded stages

* WIP

* lazily read a stream of ConnectionIO into a single ConnectionIO

* cancel on IO error

* figuring out how to put all the pieces together

* graph WIP

* Removing workflowId from the JSON API

* simplify acsAndBoundary; describe other flow pieces

* WIP

* use Vector in InsertDeleteStep; add variant for ACS (no deletes)

* `org.wartremover.warts.NonUnitStatements` enforced in `http-json` module

* evaluate InsertDeleteStep to a ConnectionIO

* database variant of LfValueCodec, using numbers for numbers

* convert input to JSON, combine insert plans, connect rest of contractsToOffsetIo

* remove strict contractsToOffset sink

* moving dao methods into an object

* putting pieces together

* contractsFromOffset WIP

* should be it

* cleanup

* cleanup

* contractsIo that takes List[domain.TemplateId.RequiredPkg])

* contractsIo that takes List[domain.TemplateId.RequiredPkg])

* cleanup

* put all pieces together, testing

something does not work yet

* diff is not required to return anything

that is why Sink.lastOption that gives Option[domain.Offset]

* factor out tuple split

* use traverse syntax in contractsIo2

* factor explicit flow steps out of graph DSL; remove aggregate

* locally model the Absolute/Begin distinction for offset bookmarking in DB

* Adding test cases to run HTTP Service with Postgres backend

the same set of test cases, run with and without DB backend

* make better use of domain.Offset in OffsetBoundary

* monomorphize InsertDeleteStep#append

* Disabling a test that fails with DB backend

* add release note

* add release note about workflowId

* a test case that checks the number or stored contracts

* trying to figure out why Postgres test fails on Windows with NPE
2019-11-06 13:53:16 -05:00
Jussi Mäki
0ca8580914 Fix the import paths of {value,transaction,blindinginfo}.proto (#3312)
* Fix the import paths of {value,transaction,blindinginfo}.proto

This cleans the imports in daml_kvutils.proto:

  -import "daml-lf/transaction/src/main/protobuf/transaction.proto";
  +import "com/digitalasset/daml/lf/transaction.proto";

This shouldn't be a breaking change to the outside as the proto files
themselves are not published, only the java libraries generated from them.

* Update missed deps
2019-11-01 14:54:55 +00:00
Robert Autenrieth
e887318f4f Move AuthService (#3272)
* Move AuthService and Claims

* Move interceptor and authorization wrappers

* Add artifact

* Address review comments
2019-10-29 15:46:43 +00:00
Moritz Kiefer
6c36777a8e
Release DAML trigger runner to Bintray (#3243)
This should make it a bit easier to use in a managed environment and
has been requested by some users.
2019-10-21 18:14:46 -04:00
Martin Huschenbett
9b71bf03ea Remove horribly outdated docker image (#3221)
We don't even have the `da` tool anymore referenced in there.
2019-10-18 12:17:22 +00:00
Remy
a3518d671e daml-lf proto: archive proto for daml-lf 1.6 (#3142)
* daml-lf proto: archive proto for daml-lf 1.6

* try to fix archive ProtoTest for windows

* more windows fix for  archive ProtoTest
2019-10-15 18:35:20 +00:00
Remy
e45b585307 daml-lf: repackage daml-lf archive proto (daml_lf -> daml_lf_dev) (#3169) 2019-10-15 09:36:11 +00:00
Remy
d64df4b5f7 daml-lf: clean achive bazel build (#3175)
* clean the mess in daml-lf achive bazel build

* add release note

* Address Moritz's comments

* more release notes
2019-10-15 08:36:46 +00:00
Remy
fc1a000090 daml-lf: rename daml_lf_archive_scala target to daml_lf_archive_reader (#3168)
* daml-lf: rename daml_lf_archive_scala target to daml_lf_archive_reader

* fix release
2019-10-14 14:53:50 +00:00
Gerolf Seitz
5592092cc7
Remove DAML-on-X IndexServer and Reference Server (#3108)
It is not needed anymore and can therefore be deleted.
2019-10-08 11:47:21 +02:00
associahedron
3e86680f3b Mark daml json-api as experimental (#3123) 2019-10-08 09:07:25 +00:00
Moritz Kiefer
78c2ec2f25
Bundle trigger runner in SDK (#3069)
There is deliberately no description and release notes entry to hide
it from `daml --help` and to make it a bit more accessible but
shipping it should make it a bit easier to experiment with.
2019-09-30 11:52:20 +02:00
Andreas Herrmann
98bb580aa2 Bundle packages with SDK releases (#3056)
* Enable SDK daml-lib package dependencies

* integration-test: SDK daml-libs dependency

* trigger --> daml-trigger

Rename the trigger library package to match other builtin packages.

* Bundle daml-trigger.dar in SDK release tarball

* Describe SDK bundled packages in changelog

* Push case match inside expand

Addressing review comment
8208277747 (r329428240)
2019-09-30 09:15:11 +00:00
Gary Verhaegen
f9f118a84d
release instructions: update number format (#3042) 2019-09-26 11:42:34 +01:00
Gary Verhaegen
26a6d77a7d
release instructions: mvn -U (#2934) 2019-09-18 13:58:29 +01:00
Leonid Shlyapnikov
4a9fb1ccff
DAML assistant codegen command (#2800)
* DAML assistant codegen command, codegen config reader and docs
2019-09-12 14:23:49 -04:00
Leonid Shlyapnikov
d237040822 DAML Assistant json-api logback configuration (#2797)
* filter out exception stack traces logged at debug

* Adding logback.xml for DAML Assistant release

* Fixing readme

* Fixing readme

* Use the config file in the assistant
2019-09-09 11:13:50 +02:00
Moritz Kiefer
2e29d3c0d9
Integrate the HTTP JSON API with the assistant (#2792) 2019-09-06 15:48:15 +02:00
Andreas Herrmann
ed39800a9f Rename hazel_deps to hackage_deps (#2789)
* hazel_deps --> hackage_deps

Mechanical change:

```
sed -i 's/hazel_deps/hackage_deps/g' $(ag -l hazel_deps)
```

* Hazel dependencies --> Hackage dependencies
2019-09-06 09:01:09 +00:00
Moritz Kiefer
1bd2a34877
Update instructions for publishing documentation (#2757) 2019-09-05 11:57:52 +02:00
Leonid Shlyapnikov
99e765a1e0
Releasing http-json-binary (#2751)
com/digitalasset/ledger-service/http-json-deploy/100.13.21
2019-09-04 15:47:24 -04:00
Stefano Baghino
e452aa2cea
Port semantic tests to the new test tool (#2718) 2019-09-02 14:40:10 +02:00
Oliver Seeliger
ed73c850cf
Release missing sources jars (#2683)
* Release missing sources jars

* Review feedback by Gerolf
2019-08-29 08:21:57 +02:00
Martin Huschenbett
901165092e Extend release readme with intructions for manual testing comment (#2691)
Add a step to the release process readme such that people leave a comment
like "All manual tests have passed" on the release PR.
2019-08-28 18:52:27 +00:00
Gary Verhaegen
126ac64bbf
daml-sdk-head also installs HEAD jars (#2652) 2019-08-27 18:08:52 +02:00
associahedron
d9e42d9798 Robuster logic for --install-assistant=auto (#2656)
* Robuster logic for --install-assistant=auto

* Handle Windows correctly
2019-08-26 10:02:19 +00:00
Robin Krom
f209950f0b
language: append the version to the output dar name by default. (#2559)
* language: append the version to the output dar name by default.

We now by default output foo-1.0.0.dar instead just foo.dar. Also the
maven coordinate default naming got removed.

* fixing integration tests and quickstart.dar occurences
2019-08-15 18:07:52 +02:00
Leonid Shlyapnikov
d9121af530 Releasing //ledger-service/http-json module and its dependencies (#2543) 2019-08-14 17:19:31 +00:00
Gary Verhaegen
99ea93168d
update copyright notices (#2499) 2019-08-13 17:23:03 +01:00
associahedron
72a7bc63a6
Organizing SDK ledger commands, adding ledger upload-dar and ledger navigator commands. (#2416)
* Add deploy-navigator command to daml helper.

* Rebase

* Organize ledger commands

* Sandbox /= Remote Ledger, should not use that config.
2019-08-06 12:41:43 +01:00
associahedron
aa4340e499 Hide daml deploy interface temporarily (#2396) 2019-08-05 12:57:59 +00:00
nickchapman-da
299d0c1b98
daml deploy, now with party management (#2384)
* daml deploy, now with party management

* address some review comments
2019-08-05 13:07:37 +01:00
Stephen Compall
52efc0966c
daml-lf: new data-scalacheck library for ImmArray, FrontStack, et al (#2342)
* new data-scalacheck library for ImmArray, FrontStack, et al

* add checkLaws functions as library to data-scalacheck

* make checkLaws more configurable at use points
2019-08-01 11:10:56 -04:00
nickchapman-da
b30228ab95
{-# LANGUAGE OverloadedStrings #-} is now on by default (#2270) 2019-07-24 08:09:26 +01:00
mziolekda
ac06bef329
eliminate v2 version of the participant-state interface (#2255)
* eliminate v2 version of the participant-state interface

* remove participant-state-v1 from release artifacts
2019-07-23 14:59:15 +02:00
nickchapman-da
49a6607160
daml deploy: first version (#2176)
* daml deploy: first version

* fix build: replace callCommand with runProcess_

* abstract/share: doBuild, getDarPath

* hide deploy command for now

* make sandbox port configurable for both deploy & start daml-helper commands
2019-07-17 14:09:55 +01:00
Stefano Baghino
467dcd9767 Mention need to adjust release notes early in the release instructions (#2189) 2019-07-17 12:01:08 +00:00
Stefano Baghino
df9872eafd Be more specific about SDK removal in Windows installer (#2184)
Fixes #2183
2019-07-17 10:28:15 +00:00
Gary Verhaegen
e2d9541c8c bundle vsix in sdk tarball (#2113) 2019-07-12 21:35:05 +00:00
Moritz Kiefer
e374a9adaa Include NOTICES file in release tarballs (#2110) 2019-07-11 15:02:29 +00:00