Commit Graph

6702 Commits

Author SHA1 Message Date
Moritz Kiefer
7890381cac
Publish EE JSON API to artifactory (#9392)
changelog_begin
changelog_end
2021-04-13 16:12:18 +02:00
Moritz Kiefer
6ab965577e
Fix scala version in quickstart-scala (#9401)
Setting this manually has resulted in #9397 where we use a scala
version that doesn’t have a silencer version. This PR changes the
quickstart template to use the scala version we also use in our builds
which should avoid them getting out of sync in the future.

changelog_begin
changelog_end
2021-04-13 15:54:40 +02:00
Gary Verhaegen
87d3f898ba
offboarding @hurryabit (#9398)
:sadpanda:

CHANGELOG_BEGIN
CHANGELOG_END
2021-04-13 11:59:24 +00:00
Miklos
29fcebe224
Fix recording of cache metrics [KVL-888] (#9382)
* Fix recording of cache metrics.
CHANGELOG_BEGIN
CHANGELOG_END

* Removed unused imports.

* Code tidying.
2021-04-13 10:47:08 +00:00
Moritz Kiefer
67b0e2be93
Drop hurryabit from release rotation 😭 (#9393)
changelog_begin
changelog_end
2021-04-13 12:35:32 +02:00
mziolekda
24c64ceff3
Clean up participant-integration-api command line help page (#9385)
CHANGELOG_BEGIN
CHANGELOG_END
2021-04-13 10:57:40 +02:00
azure-pipelines[bot]
2f4b32fbfe
update NOTICES file (#9391)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-04-13 10:16:18 +02:00
Hubert Slojewski
cda294028a
KVL-861 Add support for dumping expected and actual updates in the integrity checker (#9379)
* Add support for dumping expected and actual updates in the integrity checker to make debugging easier

CHANGELOG_BEGIN
CHANGELOG_END

* Fix dependencies

changelog_begin
changelog_end

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2021-04-13 09:00:23 +02:00
tudor-da
7fc86b7311
Fix flaky StateCacheSpec test (#9389)
* Use AsyncFlatSpec and wait for putAsync Future results

CHANGELOG_BEGIN
CHANGELOG_END
2021-04-13 06:50:57 +00:00
Robert Autenrieth
301dcd9e40
DPP-316: Enable the use of the append only index database (#9368)
* Enable append-only schema in index

changelog_begin
changelog_end

* Enable append-only schema in indexer

* Add CLI flags to configure append-only indexer

* Fix CLI flag name

* Remove unused parameter

* Change CLI flag description

... it's independent of Postgres

* Refactor how indexer config is specified in CLI

* Upper case constants
2021-04-12 21:11:03 +00:00
Stephen Compall
9ac74e697d
Oracle json-api: websockets and testing (#9278)
* comparison queries

* name the contract primary key constraint

* use ignore_row_on_dupkey_index instead of merge

- suggested by @cocreature in #9286 f7b2f14294fa33d6804251ce841529a1e2bd298d; thanks

* retrySqlStates for oracle

* enable all non-websocket tests

* name the template_id primary key constraint

* clean up concatFragment calls

* add Websocket tests for oracle

* move iouCreateCommand to be usable by oracle integration tests

* work around Scala 2.12 NPE in Source

* multiquery support for Oracle

* matchedQueries, therefore query-stream support for Oracle

* enable websocket tests

* test '& bar' and 5kb strings

- 5kb string fails on Oracle with
  ORA-01704: string literal too long

* refine the long data cases; gets too long at 4000 bytes as expected

- however, the predicate fails for unknown reason before then; possibly a missed
  escape character case

* handle long data with a fallback

- now the predicate fails in all cases instead of a SQL error, which is...better

* only interpolate true, false, null into JSON predicate conditions

- the problem was with JSON-formatted data; it must be SQL-typed instead

* adapt equal's large-data solution for comparison as well

- only works for numbers and strings, but that's all we need to compare

* move Implicits to Queries

* remove stray spaces in output

* test Oracle query expressions alongside Postgresql

* test that bools aren't compared like numbers and strings

* test @> conjunctions and special {}-query handling

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* note on PASSING ... AS X

- suggested by @cocreature; thanks

* remove printlns; these functions don't really need scaffolding anymore

- suggested by @stefanobaghino-da; thanks
2021-04-12 17:49:19 +00:00
Kamil Bożek
104ad067ef
Streaming contract state events (on the append-only schema) [DPP-304] (#9365)
* ContractStateEventsReader draft based on the POC

* Code formatting

* Moved the contract state event reader as a method to the TransactionsReader.
(required to ease parallel stories development)

* Basic unit tests for reading contract state events

* Removed the dependence on not yet present  column.

* Workaround for the lack of 'event_sequential_id' column in the 'parameters' table.
The workaround is intended to be replaced with a proper solution when the append-only schema comes in

* Fixed the unit test for contract state events

* Re-enabled all ledger dao suites

* Included 'JdbcLedgerDaoContractEventsStreamSpec' in the H2 db suites

* Intermediary DTO for contract state events

* Added a comment explaining introduction of the RawContractStateEvent

* Simplified contract state event reading query

* Easier transition for the append-only schema

* Minor refactor

* Minor change

* Ingest contract key for consuming exercise nodes

* CHANGELOG_BEGIN
- [Integration Kit] new streaming query for contract state events
- [Integration Kit] indexing contract keys for consuming exercise events
CHANGELOG_END

* Minor change

* Fixed formatting

* Ingesting key values for consuming exercised events - H2 database

* Addressed review comments

* Moving changes from the 'dao' to the 'appendonlydao' package - ContractStateEventsReader

* Moving changes from the 'dao' to the 'appendonlydao' package - indexing create_contract_key for exercise nodes

* Reduced code duplication - minor

* Reverted changes to the JdbcLedgerDaoSuite
Tests for the ContractStateEventsReader will come in the future

* Restored original naming for the indexed create key

* Improved comments + throwing UnsupportedOperationException

* Added ledger_effective_time to the ContractStateEvent.Created

* Update a metric name

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>

* Simplified GlobalKey imports

* Added a comment on events stream parallelism level

* Inline event_kind types in the SQL query for contract state events

* A constant for the event sequential id edge case

* Removed unused imports

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>
2021-04-12 18:14:40 +02:00
mziolekda
da6a0d612d
Extend prometheus metrics with min/mean/max values (#9380)
* Extend prometheus metrics with min/mean/max values

CHANGELOG_BEGIN
CHANGELOG_END

* address review comments
2021-04-12 15:02:29 +00:00
mziolekda
10edc66070
Implement Prometheus metrics back-end (#9373)
* implement prometheus metrics backend

CHANGELOG_BEGIN
- Add prometheus metrics as a cli option in the participant integration api
CHANGELOG_END

* repin scala 2.13 lockfile

* format scala
2021-04-12 12:35:37 +00:00
tudor-da
89b5dbb26e
LedgerDao and ContractsReader interface updates (#9349)
* LedgerDao and ContractsReader interface updates for mutable state cache implementation
* Factored the LfValueSerialization cache out of the ContractStore
* Implemented state lookup methods (at valid_at)

This PR does not contain tests for the new lookup methods.
The tests have been extracted in a separate branch and will be merged
when the DAO integration testing suite will be adapted
for the append-only schema.

CHANGELOG_BEGIN
CHANGELOG_END

* Addressed review comments
2021-04-12 10:18:37 +00:00
Robin Krom
8480032d85
daml package: bump timeout for tests (#9377)
CHANGELOG_BEGIN
CHANGELOG_END
2021-04-12 09:45:32 +00:00
Hubert Slojewski
bca24a9eb1
Make LogEntryId computation strategy injectable to submission validators (#9302)
CHANGELOG_BEGIN
CHANGELOG_END
2021-04-12 10:08:41 +02:00
tudor-da
78dc238ae3
StateCache implementation for mutable contract state (#9299)
* StateCache implementation for mutable contract state
* Wrapper around CaffeineCache
* Ensures order-deterministic update conflict resolution
* Concurrency and correctness tests

CHANGELOG_BEGIN
CHANGELOG_END

* Addressed review comments
2021-04-09 13:11:06 +00:00
Moritz Kiefer
91b65e8004
Patch hoogle binary to include bugfix (#9366)
This includes https://github.com/ndmitchell/hoogle/pull/367.

As usual, I unfortunately cannot test this myself so please review
carefully.

Note that this will slightly increase compile times since we will now
build hoogle. However, we still only build hoogle rather than
everything which takes less than 2min on my very weak personal
laptop. We could integrate this with our nix cache but for now, I’m
not that worried about this.

changelog_begin
changelog_end

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2021-04-09 14:17:03 +02:00
Moritz Kiefer
dc4b9e5968
Publish trigger service EE fat JARs (#9363)
* Publish trigger service EE fat JARs

changelog_begin
changelog_end

* Include ee in artifact name

changelog_begin
changelog_end
2021-04-09 13:24:18 +02:00
anthonylusardi-da
11e5dd37e7
link to postgres docs (#9353)
* link to postgres docs

* CHANGELOG_BEGIN
CHANGELOG_END
2021-04-09 09:28:58 +02:00
Robin Krom
f84b6abc05
daml build: add a --access-token-file for remote dependencies (#9358)
* daml build: add a --access-token-file for remote dependencies

This adds a `ledger.access-token-file` field in the `daml.yaml` project
file and a `--access-token-file` flag to `daml build` to authorize
querying/fetching of remote dependencies.

CHANGELOG_BEGIN
[daml build] A new flag `--access-token-file` is added for the `daml
build` command. It allows the specify the path to an access token to
authenticate against the ledger API. This is needed if the project
depends on a remote Daml package hosted on such a ledger. Alternatively,
the path to the token can also be specified in the `daml.yaml` project
file under the `ledger.access-token-file` field.
CHANGELOG_END
2021-04-08 23:20:52 +02:00
Gary Verhaegen
dfe26b93b0
fix hoogle (#9364)
After #9362, hoogle stopped returning any result. It was up and running,
but with an empty database.

Problem was two-fold:
1. In the switch to `nix`, I lost track of the fact that we were
   previously doing all the work in `~/hoogle` rather than `~/`, which
   broke the periodic refresh.
2. The initial setup has been broken for months; machines have been
   initializing with an empty DB and only getting data on the first
   refresh since #7370.

CHANGELOG_BEGIN
CHANGELOG_END
2021-04-08 18:45:47 +02:00
Gary Verhaegen
c220e05380
infra/hoogle: use nix (#9362)
This is a demonstration, commit-by-commit, of how to use the blue/green
deployment for Hoogle servers.

The actual change (using nix) is stolen from #9352.

CHANGELOG_BEGIN
CHANGELOG_END
2021-04-08 16:59:16 +02:00
Robert Autenrieth
5a983e3c4c
Deduplicate LfValueTranslation cache (#9354)
* Deduplicate LfValueTranslation cache

changelog_begin
changelog_end

* Fix remaining usages
2021-04-08 16:13:12 +02:00
Moritz Kiefer
e84c954973
Expose Oracle support in the EE trigger service (#9342)
* Expose tho Oracle support in the EE trigger service

This PR builds on the previous PR that did all the actual work on
Oracle support and exposes it in the enterprise edition. This PR only
releases the enterprise edition via the SDK tarball. I’ll add
artifactory publishing separately.

changelog_begin
changelog_end

* Update daml-assistant/daml-sdk/validate.sh

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2021-04-08 14:50:47 +02:00
Moritz Kiefer
0303017343
Delete "testing with scenarios" section (#9360)
This is superseded by
https://docs.daml.com/daml/intro/2_DamlScript.html which seems much
nicer and I don’t see what we gain by having both.

fixes #8839

changelog_begin
changelog_end
2021-04-08 14:50:33 +02:00
Richard Kapolnai
587bff2b9d
Fix comment formatting in state (#9359)
* fix Daml comments in DA.Action.State.Class.ActionState
changelog_begin
changelog_end

* fix typo

* add backticks
2021-04-08 12:14:50 +00:00
Oliver Seeliger
4c231dc25c
Update canton tests to pre 0.23.0 release (#9356)
- switch lf_version to stable and latest from legacy
- enable non-contract-key race condition tests in regular canton deployment (on non-unique contract key domain)
- reenable temporarily disabled semantic tests (see KVL-876 - fyi @hubert-da)

changelog_begin
changelog_end
2021-04-08 13:53:49 +02:00
Moritz Kiefer
bc4e00bada
Run Canton tests with JDK11 (#9355)
* Run Canton tests with JDK11

changelog_begin
changelog_end

* review comments

changelog_begin
changelog_end
2021-04-08 10:42:40 +02:00
Gary Verhaegen
948d4dd964
infra: hoogle blue/green tf (#9351)
Making it a bit easier to manage rollouts.

CHANGELOG_BEGIN
CHANGELOG_END
2021-04-08 10:07:11 +02:00
Nemanja
69ecf577b8
Removing Git as a prerequisit from the GSG (#9202)
* Removing Git as a prerequisit from GSG

* CHANGELOG_BEGIN
CHANGELOG_END
2021-04-08 06:30:34 +00:00
Gary Verhaegen
2745bc03a5
macos: move cache setup to step 2 (#9350)
The caches really need to be set up before we warm them up.

CHANGELOG_BEGIN
CHANGELOG_END
2021-04-07 21:42:15 +02:00
Gary Verhaegen
38c417e981
bump hoogle ubuntu (#9344)
16.04 is approaching EOL.

CHANGELOG_BEGIN
CHANGELOG_END
2021-04-07 20:11:36 +02:00
Remy
a2ccf1b2a0
LF: release LF 1.13 archive snapshot (#9348)
CHANGELOG_BEGIN
CHANGELOG_END
2021-04-07 19:12:42 +02:00
Moritz Kiefer
867e62520d
Add exception handling to Daml Script (#9324)
* Add exception handling to Daml Script

changelog_begin
changelog_end

* Update daml-script/daml/Daml/Script.daml

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>

* Add more test cases

changelog_begin
changelog_end

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
2021-04-07 18:48:24 +02:00
Gary Verhaegen
c97db24295
fix macOS cache cleaning (#9343)
The script needs to run once before the first build, otherwise the cache
folders get created on the main partition.

CHANGELOG_BEGIN
CHANGELOG_END
2021-04-07 18:46:44 +02:00
Moritz Kiefer
bac3521be9
Add sqrt to DA.Math (#9346)
* Add sqrt to DA.Math

fixes #9149

Not a huge fan of DA.Math but this seems like a nicer fix to #9149
than removing the lint.

changelog_begin
changelog_end

* revert change to lints

changelog_begin
changelog_end
2021-04-07 17:25:45 +02:00
Sofia Faro
06701f7d5e
Expose rounding modes as constructors + add BigNumeric docs. (#9336)
* Expose rounding modes as constructors.

This PR exposes the rounding modes as constructors for RoundingMode. Pattern matching for RoundingMode is not implemented and not critical (will open a separate issue).

This PR also adds documentation for BigNumeric.

changelog_begin
changelog_end

* Fix rounding mode literals

* Update data-types.rst

* expose constructors

* expose constructors (part 2)

* Update compiler/damlc/daml-prim-src/GHC/Types.daml

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Try to improve `Numeric n` description.

* Add message for RoundingMode match and a test.

* Restrict RoundingMode test to 1.dev

* Update version numbers to 1.13

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2021-04-07 14:58:31 +00:00
Moritz Kiefer
b90a9c3511
Remove exception message from AnyException (#9328)
This doesn’t really make that much sense in there and it makes some
things like Daml Script harder because I need to duplicate the logic
to get the exception message.

changelog_begin
changelog_end
2021-04-07 16:23:35 +02:00
Remy
35759fc83a
LF: Freeze archive proto for LF 1.13 (#9345)
CHANGELOG_BEGIN
CHANGELOG_END
2021-04-07 15:25:17 +02:00
nickchapman-da
0251e930a2
Improve TX normalization. (#9341)
* Improve TX normalization.

Extend `foreachInExecutionOrder` / `foldInExecutionOrder` to handle rollback nodes. Two new args control behaviour when entering/leaving rollback nodes. Caller can control (with a bool) if the traversal continues under exercise/rollback nodes. Update existing callers.

Use `foldInExecutionOrder` to define `reachableNodeIds`. And test.

Use `foldInExecutionOrder` to improve the behaviour of `NormalizeTransaction` so that all nodes within a rollback node are dropped. Also completely reworking the spec/test for the new behaviour.

changelog_begin
changelog_end

* fix build
2021-04-07 12:49:54 +01:00
Robin Krom
2b1f882a9d
daml-ledger: new list-packages command (#9325)
Fixes #9196.

This adds a new command `daml package list` which fetches all
available packages from the ledger and prints package name/versions to
stdout.

CHANGELOG_BEGIN
[daml packages] A new `daml packages list` command has been added to
list packages deployed on a remote Daml ledger.
CHANGELOG_END
2021-04-07 13:23:56 +02:00
Remy
568a8528a8
LF: release preview of LF 1.13 (#9329)
CHANGELOG_BEGIN
- Daml: (Early access) add support for BigNumeric
CHANGELOG_END
2021-04-07 12:42:54 +02:00
azure-pipelines[bot]
8b8f7369aa
update compat versions for 1.12.0-snapshot.20210406.6646.0.631db446 (#9332)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-04-07 08:50:55 +02:00
anthonylusardi-da
960134dd54
Fix typo in profiler docs, file name starts with profile (not profiler) (#9338)
* fix typo, file name starts with profile (not profiler)

* CHANGELOG_BEGIN
CHANGELOG_END
2021-04-07 08:50:21 +02:00
Moritz Kiefer
0eac00f343
Avoid deprecated forHostWithLedgerIdDiscovery in quickstart-java (#9337)
fixes #9319

changelog_begin
changelog_end
2021-04-06 22:38:35 +02:00
Marton Nagy
5b115717de
Introduce parallel indexer (#9304)
* Introduce parallel indexer

* Adds parallel indexer in PoC quality
* Adds relevant metrics, and wiring to the parallel indexer code
* Minor fixes to tracing in TransactionsReader

changelog_begin
changelog_end

* Tag all todos

... with 'append-only', so that they are easier to find.

* Refactor metrics

* Remove AverageCounter

Co-authored-by: Robert Autenrieth <robert.autenrieth@digitalasset.com>
2021-04-06 20:07:22 +02:00
Sofia Faro
e84a94e8c1
Update issue number for BigNumeric Divisible instance. (#9334)
changelog_begin
changelog_end
2021-04-06 16:37:16 +00:00
Moritz Kiefer
8f7ef05765
Avoid mention of scenarios in Sandbox docs (#9335)
fixes #9331 
changelog_begin
changelog_end
2021-04-06 18:21:36 +02:00