Commit Graph

7230 Commits

Author SHA1 Message Date
Remy
a6b536f3c4
Compiler: Make LF 1.13 the default output (#9907)
* Compiler: Roll LF Version

Fix #8591

CHANGELOG_BEGIN
- [Compiler] Default ouput LF version is now 1.13
CHANGELOG_END
2021-06-24 11:58:50 +02:00
mziolekda
29ddc88dcb
remove redundant spaces in the log format (#10093)
CHANGELOG_BEGIN
CHANGELOG_END
2021-06-24 07:36:03 +00:00
Robert Autenrieth
01e329f2de
DPP-432 Add exception tests to the JdbcLedgerDao suite (#10040)
* Add exception tests

changelog_begin
changelog_end

* Add another test

This one tests a transaction that can only be
produced by a privacy-aware ledger.

* Address review comments
2021-06-23 23:20:18 +02:00
Moritz Kiefer
3d79cbf2b1
Bump compat versions (#10096)
We need to do the same override for the gRPC version we do on main to
resolve conflicts.

changelog_begin
changelog_end
2021-06-23 22:12:04 +02:00
Moritz Kiefer
5bf7d8f570
Fix speedy perf reporting (#10100)
jq does not like dashes see https://github.com/stedolan/jq/issues/38

changelog_begin
changelog_end
2021-06-23 20:16:47 +02:00
Moritz Kiefer
0970821ce7
Bump cpu alloc for build-and-lint (#10099)
changelog_begin
changelog_end
2021-06-23 17:57:41 +00:00
tudor-da
cbca779657
[In-memory fan-out] Ensure getTransactionLogUpdates with max fetch size (#10064)
CHANGELOG_BEGIN
CHANGELOG_END
2021-06-23 16:32:58 +00:00
Kamil Bożek
c756153951
Release SDK 1.11.2 (#10094)
CHANGELOG_BEGIN
CHANGELOG_END
2021-06-23 18:26:47 +02:00
azure-pipelines[bot]
3a4235400d
rotate release duty after 1.15.0-snapshot.20210622.7213.0.d867d904 (#10086)
@S11001001 is taking care of 1.15.0-snapshot.20210622.7213.0.d867d904 (#10085), so they get pushed back to the end of the line.

Please do not merge this before #10085.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-06-23 10:29:16 -04:00
Stephen Compall
ecc2d11536
recommend some daml compiler warnings to enable (#10082)
- based upon @cocreature's
  https://discuss.daml.com/t/making-the-most-out-of-daml-compiler-warnings/739

CHANGELOG_BEGIN
CHANGELOG_END
2021-06-23 14:20:59 +00:00
Moritz Kiefer
b2bb45e4d9
Bump daml repl timeout (#10089)
changelog_begin
changelog_end
2021-06-23 14:19:26 +00:00
mziolekda
db60d15b5e
Log ledger api validation failures at info level (#10080)
* CHANGELOG_BEGIN
Log ledger api validation failures at info level
CHANGELOG_END

* log validation failures in Api*Service family of classes

* address review comments
2021-06-23 13:31:48 +00:00
Moritz Kiefer
f0dc025ac9
Release 1.15 snapshot (#10090)
changelog_begin
changelog_end
2021-06-23 14:31:29 +02:00
nickchapman-da
5b73813dfd
Support reinterpretation for failing commands (#9993)
* WIP: reinterpret must fail

changelog_begin
changelog_end

EXPLORE: add x/y prefix to reinterpret

MiniEngineTest.scala

MiniTests.daml

setup reinterpret testing one step down, to be able to see rollback nodes

MiniEngineTest.scala - refactor yreinterpret

first minimal reinterpret test

setup testcase for reinterpret on a command which throws

catch exception during reinterpret; and test

introduce type for Validating; avoid boolean blindness

remove exploratory MiniTests

remove exploratory x/y prefix on reinterpret

question

fix build

* small cleanup

* add `unsafeCompileForReinterpretation`; preserve interface of existing `unsafeCompile` (command list)

* remove the Validating type (not needed for this work)

* undo format change

* more private

* update comment for Engine.reinterpret

* test ensure failure works correctly; no rollback expected

* refactor to share duplicated code in reinterpret tests

* remove unhelpful create/ensure tests

* test that reinterpretation of an uncatchable error remains a failure

* test that rollback-version-check-error remains a failure for reinterpretation

* remove dev comment left in by accident
2021-06-23 10:39:10 +00:00
Samir Talwar
e32856ecf4
dev-env: Use nix-shell for scripts instead of sourcing dade-common. (#10088)
* Introduce shell.nix to expose the Nix tooling, if required.

* dev-env: Rewrite the ghcide script to use nix-shell.

* language-support/hs: Use `nix-shell` for export-package.sh.

* Nix: Add a shebang to the Bazel wrapper script.

CHANGELOG_BEGIN
CHANGELOG_END

* dev-env: Standardize the `set` header in `ghcide`.
2021-06-23 09:39:00 +00:00
Samir Talwar
682c096d4f
dev-env: Remove some unused stuff. (#10087)
* dev-env: Replace `gpg` and `make` with symlinks.

* dev-env: Remove sphinx-autobuild. It's brought in by pipenv.

* dev-env: Remove Python 3.6 references. They seem to be unused.

CHANGELOG_BEGIN
CHANGELOG_END
2021-06-23 08:13:38 +00:00
Moritz Kiefer
85e383ce0b
Fix --max-inbound-message-size flag in Daml Repl (#10078)
fixes #10072

changelog_begin
changelog_end
2021-06-23 10:10:40 +02:00
Stephen Compall
d867d90400
define query offset semantics solely in terms of #9847 per-query offsets (#10071)
Since per-request offset can be described strictly as a special case of the new
per-query offset semantics, go ahead and describe it that way, so that really
only one model needs to be understood to fully comprehend the query request
semantics.

CHANGELOG_BEGIN
CHANGELOG_END
2021-06-22 12:32:48 -04:00
Victor Peter Rouven Müller
1f5aa4491c
[JSON-API] Add metrics for ledger command submission timing (#10076)
changelog_begin

- [JSON-API] Timing metrics which measure how long the processing of a command submission request takes on the ledger are now available

changelog_end
2021-06-22 15:59:23 +00:00
azure-pipelines[bot]
7eb2ce72a9
update compat versions for 1.15.0-snapshot.20210615.7169.0.adeba206 (#10028)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-06-22 17:44:18 +02:00
Moritz Kiefer
c44f33736e
Simplify speedy benchmark (#10079)
This PR drops two things:

1. The check that the benchmark hasn’t been modified. This hasn’t ever
been useful and it keeps being annoying.
2. It stops the comparison against the old version and instead just
benchmarks the current version. We really only care about the day to
day changes. Comparing against an arbitrary year old version has lost
all meaning at this point.

changelog_begin
changelog_end
2021-06-22 15:11:47 +00:00
Remy
58b1c4e771
Speedy: Refactor contract Id/Key callback (#10033)
CHANGELOG_BEGIN
CHANGELOG_END
2021-06-22 16:34:28 +02:00
Oliver Seeliger
8558c73f27
Upgrade canton to 0.25.0 based on daml 1.14.0 (#10074)
changelog_begin
changelog_end
2021-06-22 12:18:28 +00:00
Moritz Kiefer
e4c1c58317
Run scenarios in off-ledger machine (#10070)
* Run scenarios in off-ledger machine

This PR builds on the previous PR that split scenario execution in two
different speedy machines and now actually makes the machine that runs
scenarios run in off-ledger mode just like we handle Daml Script.

This required a bunch of refactoring to make it nice so apologies for
the slightly large PR. Hopefully it’s still relatively easy to follow
and luckily it deletes more code than it adds.

changelog_begin
changelog_end

* Apply suggestions from code review

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* review comments

changelog_begin
changelog_end

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2021-06-22 12:11:00 +02:00
Gary Verhaegen
31a76a4a2a
allow CI pools to use any zone (#10069)
This morning we started with very restricted CI pools (2/6 for Windows
and 7/20 for Linux), apparently because the region we run in (us-east1)
has three zones, two of them were unable to allocate new nodes, and the
default policy is to distribute nodes evenly between zones.

I've manually changed the distribution policy. Unfortunately this option
is not yet available in our version of the GCP Terraform plugin.

CHANGELOG_BEGIN
CHANGELOG_END
2021-06-22 10:43:08 +02:00
Victor Peter Rouven Müller
0391f3529f
[JSON-API] Add db metrics & response construction metrics (#10068)
changelog_begin

- [JSON-API] The database operations (regardless of in-memory or postgres) contain now metrics for fetching contracts by id or key (seperate metrics foreach)
- [JSON-API] The time for a repsonse payload construction of a request is now also tracked in a metric

changelog_end
2021-06-22 10:34:29 +02:00
Gary Verhaegen
ea089ad14f
clean up local VSCode after release (#10065)
TIL that our release process leaves us with a non-upgrading VSCode
extension.

CHANGELOG_BEGIN
CHANGELOG_END
2021-06-21 19:11:35 +02:00
Gary Verhaegen
39a5890b50
fix 9_functional redirect (#10067)
CHANGELOG_BEGIN
CHANGELOG_END
2021-06-21 19:11:23 +02:00
Moritz Kiefer
fb757d8ed9
Handle visibility outside of speedy (#10056)
It makes no sense to pass NotVisible to Speedy especially since that
is not how visibility is handled for the Engine. Also it lets us
delete code and I like deleted code.

changelog_begin
changelog_end
2021-06-21 17:31:37 +02:00
Stefano Baghino
b796695768
Add per-query offsets to HTTP-JSON API (#9847)
* Add per-query offsets to HTTP-JSON API

changelog_begin
[HTTP/JSON API] The streaming query endpoint now accepts to override the
offset for each query using the `offset` field. You can read more
about it on the documentation.
changelog_end

* Replace mapValues and filterKeys with something that compiles with both 2.12 and 2.13

* Refactor offset comparison

- Addresses https://github.com/digital-asset/daml/pull/9847/files#r642363229
- Addresses https://github.com/digital-asset/daml/pull/9847/files#r642363848

* Add tests (failing), fix definition of starting offset

* Fix failing test

* More tests

* Add tests for verify older behavior still works

* Fix regression

* Remove debug prints

- Addresses https://github.com/digital-asset/daml/pull/9847#discussion_r642362683
- Addresses https://github.com/digital-asset/daml/pull/9847#discussion_r642362774

* Failed attempt at ACS call integration

* Revert "Failed attempt at ACS call integration"

This reverts commit 879e0cfbdb579362674120f93a9c93388ed81bc1.

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r647019447

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r647019965

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r647020825

* Integrate with ACS calls

* Fix three broken tests

* Fix one regression

* Fix last regression

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r651586811

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r651668995

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r651669134

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r651669691

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r651670134

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r651676902

* Fix Scala 2.12 compilation error

* Fix compilation issues on Scala 2.13 and 2.12

* Remove unnecessary dependency

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r651824658

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r651831880

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r651833646

* Address two comments:

- https://github.com/digital-asset/daml/pull/9847#discussion_r651841615
- https://github.com/digital-asset/daml/pull/9847#discussion_r651842951

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r651838831

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r651949821

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r651943962

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r652492756

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r652537652

* query_offset -> queryOffset

* Document per-query offsets

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r655117581

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r655119961

* Apply suggestions from code review

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r655233517

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r655266806

* Address https://github.com/digital-asset/daml/pull/9847#discussion_r655265717

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2021-06-21 16:38:07 +02:00
tudor-da
ab8bac53ea
[In-memory fan-out] BuffersUpdater subscribes from the ledger end (#10037)
CHANGELOG_BEGIN
CHANGELOG_END
2021-06-21 14:37:55 +00:00
Victor Peter Rouven Müller
e4585295c6
[JSON-API] Add moar timing metrics (#10045)
* [JSON-API] Add more timing metrics

changelog_begin

- [JSON-API] Timing metrics are now available for party management, package management, command submission and query endpoints.
- [JSON-API] Also added a timing metric for parsing and decoding of incoming json payloads

changelog_end

* Add comments to new metrics

* Split metrics up more & remove obsolete metric

* Split up timers for query endpoints
2021-06-21 14:37:36 +00:00
Gary Verhaegen
e12a449c81
s/DABL/Daml Hub/ (#10062)
It's been long enough.

CHANGELOG_BEGIN
CHANGELOG_END
2021-06-21 16:37:17 +02:00
tudor-da
a205d0e446
Enriched conformance tests for transactions (trees) streams (#9977)
* Check transactions from streams in TXHideCommandIdToNonSubmittingStakeholders
* Symetrically implement TXTreeHideCommandIdToNonSubmittingStakeholders
* Check witness non-leakage in TXRequestingPartiesWitnessVisibility
* Check witness non-leakage in TXTreesRequestingPartiesWitnessVisibility

CHANGELOG_BEGIN
CHANGELOG_END
2021-06-21 11:23:45 +02:00
Moritz Kiefer
b98ad910fa
Bump test sha (#10055)
This needs bumping due to the off-ledger on-ledger speedy split.

changelog_begin
changelog_end
2021-06-18 10:03:55 +02:00
danielporterda
f3106c3e34
I9996 value limits it fails oracle (#10027)
* Creating OR predicate to reduce length of string literals
CHANGELOG_BEGIN
CHANGELOG_END

* removing unnecessary thrown exception
CHANGELOG_BEGIN
CHANGELOG_END

* switching to camel case for constants
CHANGELOG_BEGIN
CHANGELOG_END

* run format
CHANGELOG_BEGIN
CHANGELOG_END

* remove tolist conversion
CHANGELOG_BEGIN
CHANGELOG_END

* scalafmt

Co-authored-by: Brian Healey <brian.healey@digitalasset.com>
2021-06-17 23:48:31 +00:00
Moritz Kiefer
febca5d62d
Use ScenarioRunner.submit in Daml Script (#10053)
* Use ScenarioRunner.submit in Daml Script

changelog_begin
changelog_end

* privatize ledger variable

changelog_begin
changelog_end

* drop space

changelog_begin
changelog_end
2021-06-17 17:12:45 +00:00
Stephen Compall
3113702dc0
index json-api signatories and observers on Oracle (#9895)
* nvarchar2 keys are text-incompatible, but varchar2 keys are fine

* commit the ACS update before query

* add changelog

CHANGELOG_BEGIN
- [JSON API] The Oracle database schema has changed; if using
  ``--query-store-jdbc-config``, you must rebuild the database by adding
  ``,createSchema=true``.  See #9895.
CHANGELOG_END

* expand the InitDdl set to include materialized views

* replace search index with a materialized view that expands the stakeholders

* allow materialized views to be created in Oracle testing

* join and query the contract_stakeholders table for party-set membership

- restoring a few elements removed by 3e6661128d (#9484)
2021-06-17 11:52:37 -04:00
azure-pipelines[bot]
cb3a42a9bd
update NOTICES file (#10047)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-06-17 17:32:55 +02:00
Sofia Faro
2f5186938e
Allow constraints in any position in data-deps. (#10049)
This fixes #8411. It turns out the issue was some missing parentheses.

changelog_begin
changelog_end
2021-06-17 15:45:10 +01:00
Stephen Compall
7dfa36fccd
pass supportedJdbcDriverNames implicitly, and only to what actually needs it (#10036)
This solves two warts in the code:

- the validate/createUnsafe double-parse, because scopt doesn't let you flatMap;
- the non-JdbcConfig sub-configs appeared to need to know the JDBC drivers when
  they really don't, because of a quirk in the inherited implementation

That coherence of scopt.Read instances calls for all its dependencies to be
coherent leads us to treat supportedJdbcDriverNames as a nullary typeclass
instance.  This is a nullary typeclass by the same justification as
SupportedJdbcDriver; see scaladoc there for more.

And we solve the latter problem by...adding a type parameter, how else.

CHANGELOG_BEGIN
CHANGELOG_END
2021-06-17 10:40:49 -04:00
Marton Nagy
0fbc1ee67d
H2 support for append only schema (part2) [DPP-394] (#10030)
* Preparation: pull up initialisation
* Preparation: introduce partyArrayContext for more flexibility
* Preparation: add append-only-from-scratch feature to FlywayMigrations
* Add squashed Flyway migration for append-only H2
* Add support for H2 StorageBackend
* Extend conformance test suites for H2
* Extend JDBCLedgerDao unit test suites for H2
* Extend sandbox-classic unit test suites for H2
* Fixing typo, some renames based on review

changelog_begin
changelog_end
2021-06-17 15:34:25 +02:00
Moritz Kiefer
cd2fda6f3e
Fix broken package.json config (#10044)
Duplicate field names are not good

changelog_begin
changelog_end
2021-06-17 14:34:49 +02:00
Moritz Kiefer
10ab05f36f
Fix vsce publishing (#10043)
changelog_begin
changelog_end
2021-06-17 14:34:36 +02:00
azure-pipelines[bot]
090dd9590b
rotate release duty after 1.14.0-snapshot.20210615.7169.0.adeba206 (#10019)
@stefanobaghino-da is taking care of 1.14.0-snapshot.20210615.7169.0.adeba206 (#10018), so they get pushed back to the end of the line.

Please do not merge this before #10018.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-06-17 14:25:20 +02:00
tudor-da
eee484b996
Implemented in-memory buffers prunning in BaseLedger (#9936)
CHANGELOG_BEGIN
CHANGELOG_END
2021-06-17 13:56:12 +02:00
Kamil Bożek
b96639ebc3
Do not terminate streams when no end-offset given in ledger-api-bench-tool [DPP-422] (#10035)
* Do not terminate transactions/transaction trees streams when no end-offset provided in the ledger-api-bench-tool

CHANGELOG_BEGIN
CHANGELOG_END

* Minor change
2021-06-17 13:30:16 +02:00
Moritz Kiefer
2b915e93da
Split scenario & ledger execution (#10039)
* Split scenario & ledger execution

This PR by no means aims to solve everything we can do here. It is
rather the minimal change that I could get to work that provides us
with one Speedy machine for scenario execution (which is still an
on-ledger machine to avoid having to change the callsites to much in
this PR) and one speedy machine per submission.

There is tons of cleanup we can do afterwards but this should
hopefully set the right foundations.

changelog_begin
changelog_end
2021-06-17 12:45:35 +02:00
Marton Nagy
591176ccb9
Remove SqlSequence, cleanup code (#10042)
changelog_begin
changelog_end
2021-06-17 08:14:58 +00:00
Moritz Kiefer
8ea240f72a
Release SDK 1.14.0 (#10041)
changelog_begin
changelog_end
2021-06-17 10:11:09 +02:00