Commit Graph

6068 Commits

Author SHA1 Message Date
Stephen Compall
c0c2a59275
hybrid database-then-gRPC websocket query (#8226)
* group database queries while still providing the matchedQueries part of the event

* SQL query for multiquery websocket request all at once

* fetchAndPersist responds with a single bookmark

* unify connection imports

* convert DB results to domain, reassociate with proper query indices

* reassociate multiple matches with proper query indices

* make overlap more likely in testing overlap

* simpler matchedQueries merging for multi-query case

* integrate DB query with metadata-ful StreamQuery's

* expose daoAndFetch; better insertDeleteStepSource doc

* more efficient query path for contract key streams

* missed LogHandler

* persist resolved template IDs; glue the prefiltered set into the stream

* ticks and phantom removal need the state from the ACS

* compile SQL queries for query language predicates on WS

- wrong matchedQueries order

* harmonize order of ACS-vector and Positives

* misc compilation conversions

* WebsocketServicePostgresInt mixin

* update (C) date

* test websocket queries under postgres

* looking for new way to compile queries with proper matchedQueries offsets

* model that querying without matchedQueries requires only one SQL query

* SQL path for contract key streams

* nondeterminism

* fix 3 fetch tests with SQL syntax

* nondeterminism mk 2

* fix multi-party query tests by dealing with nondeterminism properly

* temp logs to track down the contract duplication

* match new scalafmt from #8437

* remove completed TODOs

* add changelog

CHANGELOG_BEGIN
- [JSON API] If the JDBC query store is enabled, it will be used to optimize
  Websocket queries as well as the previously-supported synchronous queries.
  See `issue #8226 <https://github.com/digital-asset/daml/pull/8226>`__.
CHANGELOG_END

* fix up matchedQueries indices

* complete the fast path for by-id queries

* remove AS c

- suggested by @cocreature; thanks

* remove temporary debugging logs

- suggested by @cocreature; thanks
2021-01-14 10:01:17 -05:00
azure-pipelines[bot]
65b365c2c6
update NOTICES file (#8499)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-01-14 08:45:53 +01:00
Remy
f872f473cb
LF: move package lookup utilities directly inside GenPackage (#8496)
There is no reason why they have are in a special object inside the
engine package.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-13 23:25:39 +01:00
Moritz Kiefer
cdb2c2d0ba
Port the rest of //ledger/... to Scala 2.13 (#8497)
* Port the rest of //ledger/... to Scala 2.13

draw the rest of the fcking owl

Omitted for now are the ledger API test tool which has a dependency
only compatible with 2.12 and the generated code of the Scala
codegen (the codegen compiles and runs with 2.13, the generated code
does not).

changelog_begin
changelog_end

* Less symbols

changelog_begin
changelog_end
2021-01-13 22:23:31 +01:00
Remy
77bec01db3
LF: provide methods to enrich LF values missing type and field names (#8493)
CHANGELOG_BEGIN
CHANGELOG_END
2021-01-13 19:02:51 +01:00
Stephen Compall
89bc670445
fix race condition in DB update when ACS has later contracts than the ledger-end (#8480)
* fix race condition in DB update when ACS has later contracts than the ledger-end

CHANGELOG_BEGIN
- [JSON API] Under rare conditions, a multi-template query backed by Postgres
  could have mismatched snapshots of the ACS for different templates.  These
  conditions are now checked and accounted for.
  See `issue #8226 <https://github.com/digital-asset/daml/pull/8226#issuecomment-756446537>`__.
CHANGELOG_END

* contractsFromOffsetIo already saves the offset to DB

* notes on why we rerun DB update to fix the race condition

* if the ACS last-offset exceeds the tx stream offset, save it in the DB instead
2021-01-13 11:27:33 -05:00
Moritz Kiefer
d636505d61
Release yet another 1.9.0 snapshot (#8492)
* Release yet another 1.9.0 snapshot

This includes #8490 which fixes triggers with LF 1.11. Since LF 1.11
is mentioned very prominently in our release notes, we don’t want to
break triggers there.

changelog_begin
changelog_end

* Release: Update to commit 9ed787cb3d.

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2021-01-13 15:58:10 +00:00
Moritz Kiefer
84a9488077
Port more of //ledger/... to Scala 2.13 (#8488)
* Port more of //ledger/... to Scala 2.13

changelog_begin
changelog_end

* Remove unusued dependency

changelog_begin
changelog_end

* Rename bf to factory to reflect the fact that it’s now a Factory

changelog_begin
changelog_end

* Use regex match instead of sliding string equalityt

changelog_begin
changelog_end

* regex matches are bad

changelog_begin
changelog_end
2021-01-13 16:30:43 +01:00
Stefano Baghino
9ed787cb3d
Add grpc-reverse-proxy with optional server interceptors (#8481)
* Factor 'withServices' out of ServerReflectionClientSpec

* Add grpc-reverse-proxy with optional server interceptors

changelog_begin
changelog_end

* Switch from Protobuf byte strings to native byte arrays

* Fix year in copyright headers

* Address https://github.com/digital-asset/daml/pull/8481#discussion_r555915935

* Add missing maven_coordinates tag

* Address https://github.com/digital-asset/daml/pull/8481#discussion_r555916684
2021-01-13 15:41:30 +01:00
Samir Talwar
544b0a2caa
kvutils: Make migration to the new key and value types easier. (#8483)
* kvutils: Make migration to the new key and value types easier.

CHANGELOG_BEGIN
CHANGELOG_END

* kvutils: Extract out common implicit conversions into a trait.
2021-01-13 14:38:33 +00:00
Gary Verhaegen
e6d1f1399b
fix out-of-rotation release notifs (#8491)
Only ping people on rotation for automated releases.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-13 14:44:23 +01:00
Samir Talwar
78f4b3ff7e
kvutils: Use mockito-scala's ArgCaptor. (#8486)
This results in terser code that I find easier to read.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-13 13:34:45 +00:00
Moritz Kiefer
427ea0fb7d
Remove redundant list of LF versions (#8490)
* Remove redundant list of LF versions

After #8472, I realized that there must be a list used for daml-stdlib
and daml-prim already and it turns out there is. I’ve removed that one
in favor of the one added in #8472 since I like having all in one
place and the one from #8472 is created by filtering an existing list
instead of creating a completely separate list like we do here.

changelog_begin
changelog_end

* Introduce SCRIPT_LF_VERSIONS

changelog_begin
changelog_end
2021-01-13 13:21:39 +00:00
tudor-da
ef342f75a2
Remove redundant transaction decoding in TransactionCommitter (#8457)
* Remove redundant transaction decoding in TransactionCommitter

CHANGELOG_BEGIN
CHANGELOG_END

* Revert to using lazy value for decoded transaction in DamlTransactionEntrySummary
2021-01-13 14:30:43 +02:00
azure-pipelines[bot]
75d8c0c623
rotate release duty after 1.9.0-snapshot.20210112.6048.0.e6312fa5 (#8485)
@remyhaemmerle-da is taking care of 1.9.0-snapshot.20210112.6048.0.e6312fa5 (#8484), so they get pushed back to the end of the line.

Please do not merge this before #8484.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-01-13 12:30:37 +01:00
Gary Verhaegen
742feb58be
trigger PRs job on generated PRs (#8489)
CHANGELOG_BEGIN
CHANGELOG_END
2021-01-13 11:30:17 +00:00
Moritz Kiefer
290584a378
Port //ledger/ledger-api-client/... to Scala 2.13 (#8478)
* Port //ledger/ledger-api-client/... to Scala 2.13

This pulls in Sandbox next and kvutils as a dependency so those now
build on 2.13 as well.

changelog_begin
changelog_end

* Upgrade scala-colllection-compat

changelog_begin
changelog_end

* Use toVector.sortBy instead of to(LazyList).sortBy

changelog_begin
changelog_end

* Use a view for passing things to varargs

changelog_begin
changelog_end

* avoid symbol literal in CommandClientIT

changelog_begin
changelog_end
2021-01-13 11:35:46 +01:00
Sofia Faro
51c014b367
Add the missing UTryCatch primitive. (#8462)
* Add the missing UTryCatch primitive.

I tested that the types are correct by uncommenting the corresponding
line in DA.Internal.Exception (which must remain commented, because
the engine does not yet support exceptions).

changelog_begin
changelog_end

* indent/associate

* typo :-(
2021-01-13 10:33:55 +00:00
Kamil Bożek
c720326e10
Changed index type for some of participant_events table indices to hash (#8432)
CHANGELOG_BEGIN
- Changed index method type from B-Tree to Hash for: participant_events_template_id_idx, participant_events_transaction_id_idx, participant_events_contract_id_idx indices of participant_events table
CHANGELOG_END
2021-01-13 09:52:29 +01:00
tudor-da
13a5715213
Enable asynchronous commit per connection level (#8419)
* Enable asynchronous commit per connection level

CHANGELOG_BEGIN
CHANGELOG_END

* Added HikariConnectionSpec test for asserting async commits in connection
2021-01-13 06:28:53 +00:00
Remy
e6312fa52c
LF: get ride of user-defined patterns in ValueTranslation. (#8465)
We prune the code of ValueTransalation from the user-defined patterns
in the code of Speedy's ValueTranslation. In particular we remove the usage of
com.daml.lf.language.Util.TTyConApp which:
- induces bad complexity,
- make difficult optimization.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-12 20:09:02 +01:00
Gary Verhaegen
9ec54628a2
do not support Eq, Ord, Show for large tuples (#8475)
CHANGELOG_BEGIN
CHANGELOG_END
2021-01-12 17:43:35 +01:00
Gary Verhaegen
58e8b9c40f
remove dade-test-sh (#8479)
Looks like an unfinished attempt at automating the running of inexistent
bats/osht tests.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-12 17:09:36 +01:00
Gary Verhaegen
28d9cea05e
remove jfrog-cli (#8477)
As far as I can figure out, this has been unused since #5422.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-12 15:01:01 +00:00
Moritz Kiefer
4ee21f588e
Release 1.9. snapshot (#8474)
Includes the fix for DAML Script against LF 1.11 and a minor docs fix
in the ledger model.

changelog_begin
changelog_end
2021-01-12 14:44:12 +01:00
mziolekda
0ff919f498
Update offset when party/config rejected on a duplicate (#8463)
CHANGELOG_BEGIN
Advance ledger end when the indexer encounters an duplicate party or configuration update. This allows indexer to skip-over the unexpected message and continue processing subsequent updates.
CHANGELOG_END
2021-01-12 13:43:15 +00:00
arne-da
b398a55780
Fixing typos in documentation (#8163)
CHANGELOG_BEGIN
Fixed minor typos in documentation
CHANGELOG_END
2021-01-12 13:11:12 +01:00
Gary Verhaegen
ce96802a6d
only update NOTICES from main daily compats (#8473)
See #8468 for failure mode of current config.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-12 12:10:27 +00:00
Ognjen Maric
7171cb38fa
Allow multiple requesters for a commit in the ledger model (#8471)
CHANGELOG_BEGIN
- Adjust the ledger model to account for multi-party submissions
CHANGELOG_END
2021-01-12 11:49:52 +00:00
Moritz Kiefer
b32177cb0b
Build DAML Script library for all LF versions (#8472)
The hardcoded list is obviously garbage and didn’t include 1.11 so
this PR changes it to use the one we already have so they are always
in sync.

changelog_begin
changelog_end
2021-01-12 11:23:41 +00:00
Moritz Kiefer
6b2ac0e0c6
Port parts of //ledger/... to Scala 2.13 (#8470)
* Port parts of //ledger/... to Scala 2.13

Fairly random choice of directories, I just went through them in
alphabetical order. The one thing that I had to disable for now are
the conformance tests since the ledger API test tool has a dependency
not compatible with Scala 2.13.

changelog_begin
changelog_end

* Remove accidentally included //ledger/ledger-api-client/...

doesn’t actually work yet

changelog_begin
changelog_end
2021-01-12 10:55:00 +01:00
azure-pipelines[bot]
9141ea22ae
update NOTICES file (#8468)
* update compat versions for 1.9.0-snapshot.20210111.6034.0.7855b023

CHANGELOG_BEGIN
CHANGELOG_END

* update NOTICES file

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-01-12 09:16:59 +01:00
Moritz Kiefer
cd9aeaf78e
Release 1.9.0 snapshot (#8464)
changelog_begin
changelog_end
2021-01-11 21:51:12 +01:00
Moritz Kiefer
aebde84842
Upgrade scopt to 4.0.0 (#8466)
* Upgrade scopt to 4.0.0

Scopt 3.x has some issues with Scala 2.13 because it expects an
immutable Seq on 2.13 meaning you cannot just pass in an Array. Rather
than fixing our callsites to convert to an immutable Seq everywhere,
this PR bumps to Scopt 4.0 which goes back to collection.Seq.

and leaving that aside, I’m a fan of upgrading dependencies anyway :)

changelog_begin
changelog_end

* Use val instead of def

changelog_begin
changelog_end
2021-01-11 21:23:02 +01:00
Andreas Herrmann
7855b023ac
Oauth template (#8453)
* Use ActorSystem and ExecutionContext from RequestContext

* Factor out middleware server class

To avoid passing around config and state manually.

changelog_begin
changelog_end

* Depend on databricks/sjsonnet

changelog_begin
changelog_end

* Generate request params from jsonnet template

changelog_begin
changelog_end

* Split middleware test suite sources

* Add test suite for request templates

* fmt

* TriggerServiceFixture template arguments

* Use null to indicate missing applicationId claim

Addressing
https://github.com/digital-asset/daml/pull/8453/files#r555025173

* Fix invalid path on Windows

* Close request template source

* Avoid repeated re-reading of Jsonnet files

https://github.com/digital-asset/daml/pull/8453/files#r555044262

* Factor out template argument mappings

* factor out template error handling

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-01-11 17:27:46 +00:00
Gerolf Seitz
9d120f5121
Bump Ledger API version to 1.8.0 (#8455)
CHANGELOG_BEGIN
[Ledger API] With the introduction of multi-party submissions, the
Ledger API is now at version 1.8.0
CHANGELOG_END
2021-01-11 16:02:01 +00:00
Moritz Kiefer
1abde753c9
Build and test //ledger-api/... against Scala 2.13 (#8452)
changelog_begin
changelog_end
2021-01-11 16:53:54 +01:00
Robert Autenrieth
57217f33e8
Remove expensive test (#8450)
The test was useful for finding the bug in groupContiguous,
but is not useful in the long term.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-11 16:16:47 +01:00
Bernhard Elsner
5cfa6d71f8
Add table correlating Connect and Ledger API version (#8456)
* Add table correlating Connect and Ledger API version

CHANGELOG_BEGIN
CHANGELOG_END

* Use a legal character for header underlining

* Flip columns

* Remove empty line
2021-01-11 15:09:53 +00:00
Richard Kapolnai
592e794adf
Doc minor issues (#8451)
* fix typos in docs
CHANGELOG_BEGIN
CHANGELOG_END

* minor corrections

* revert a change

* update structure doc based on comments
2021-01-11 15:52:43 +01:00
Sofia Faro
e604ba1fad
Add LFConversion support for exception primitives. (#8427)
* Add conversion for exception types

* Start adding exception primitives.

changelog_begin
changelog_end

* add more error message primitives

* Comment out the exception primitives for now.

* Dont use runtimeUnsupported

* Organize DA.Internal.Exception a bit.

* Add ActionThrow and ActionCatch.

* import all of DA.Internal.Prleude

* haskell brain
2021-01-11 14:05:24 +00:00
Gary Verhaegen
7ca49bacc7
publish docs bundle as pipeline artifact (#8454)
This make the docs bundle available as a download from any build on
Azure. I mostly thought of this as a workaround for @bame-da because of
the Big Sur thing, but I figure that may occasionally useful to other
people too.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-11 13:30:51 +00:00
Remy
fef1c70b9b
Sandbox: add early access flag to the sandboxes (#8441)
The flag allows the sandboxes (both classic and next) to use preview
version of the upcoming Daml-LF version (currently LF 1.11).

CHANGELOG_BEGIN
* [Sandbox-classic] add a flag to allow early access to the next
  Daml-LF to be released.
* [Sandbox] add a flag to allow early access to the next Daml-LF
  to be released.
CHANGELOG_END
2021-01-11 14:03:57 +01:00
azure-pipelines[bot]
287d98346d
update compat versions for 1.9.0-snapshot.20210106.5986.0.c6995a9c (#8413)
* update compat versions for 1.9.0-snapshot.20210106.5986.0.c6995a9c

CHANGELOG_BEGIN
CHANGELOG_END

* update generated copyright header

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2021-01-11 13:46:10 +01:00
Shaul Kfir
7b409b9e87
Typo fix in fmap documentation (#8350)
changelog_begin
changelog_end
2021-01-11 11:01:44 +00:00
Stefano Baghino
8d30cb5f9a
Pin scalafmt version in configuration file (#8447)
* Pin scalafmt version in configuration file

Looks like there is no other way for IntelliJ to pick up an arbitrary version.

See: https://www.jetbrains.com/help/idea/work-with-scala-formatter.html#change_scalafmt

changelog_begin
changelog_end

* Address https://github.com/digital-asset/daml/pull/8447#discussion_r554928850
2021-01-11 10:05:58 +00:00
Bernhard Elsner
64f85ae8f6
Fix docs logo (#8446)
CHANGELOG_BEGIN
CHANGELOG_END
2021-01-11 08:44:02 +00:00
Moritz Kiefer
35a866fdc2
Add Moritz as a codeowner for daml-lf stuff (#8445)
changelog_begin
changelog_end
2021-01-11 09:43:24 +01:00
Moritz Kiefer
b83c4e945d
Bump perf test for scalafmt update (#8444)
changelog_begin
changelog_end
2021-01-09 13:12:54 +01:00
Moritz Kiefer
22ce940954
Upgrade scalafmt and enable trailing commas (#8437)
This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.

As announced, this will be merged on Saturday to avoid too many conflicts.

changelog_begin
changelog_end
2021-01-09 11:37:37 +01:00