Commit Graph

6642 Commits

Author SHA1 Message Date
Moritz Kiefer
f7b2f14294
Add Oracle support in the trigger service (#9286)
* Add Oracle support in the trigger service

This PR migrates the ddl & queries and adds tests for this. It does
not yet expose this to users. I’ll handle that in a separate PR.

changelog_begin
changelog_end

* use getOrElse

changelog_begin
changelog_end
2021-03-31 18:39:08 +02:00
Samir Talwar
26259535a0
ledger-api-test-tool: Show a pleasant error message if DAR upload fails. (#9311)
Previously, we would show a big scary "UNCAUGHT EXCEPTION", which was
unnecessary. This is normal on connection failure.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-31 16:13:38 +00:00
Stefano Baghino
4b807899ca
Do not publish non-repudiation client JARs to Artifactory (#9312)
So far, every attempt failed with a 409 Conflict.
We'll try again next week.

changelog_begin
changelog_end
2021-03-31 16:02:21 +00:00
Samir Talwar
d727630009
ledger-api-test-tool: Use Akka Streams for repeated tests. (#9309)
Rather than sequencing test runs with a `foldLeft`, use the existing
`mapAsync` call.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-31 14:36:56 +00:00
Remy
451181e42f
Speedy: Cleanup Map litereals (#9242)
* Rename SGenMap to Map
* Fix complexity of SMapInsert and SMapDelete

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-31 16:08:29 +02:00
Sofia Faro
f2e0464227
Add BigNumeric literals in Daml. (#9289)
* Add BigNumeric literals in Daml.

Adds support for BigNumeric literals (when available). Only literals that will fit in a single Numeric are supported for now.

I introduced the `IsNumeric t` typeclass because this is an easy way to restrict `fromRational` without moving it into its own typeclass. (Moving it into a typeclass causes some gnarly problems with the specializer -- it starts creating references to GHC.Real in an attempt to optimize the invocation, and that's a problem because  GHC.Real doesn't exist in LF.)

I then added the "fromNumeric" and "fromBigNumeric" conversion functions in that class, since they seem really convenient, and it also means we could in the future make it so that any type that implements `IsNumeric` gains literals (via Numeric or BigNumeric literals). This would improve a lot of Numeric code, since it eliminates the need to annotate types so often! But for now only Numeric and BigNumeric literals are supported.

changelog_begin
changelog_end

* lint

* Add test for too large bigNumeric
2021-03-31 13:39:39 +00:00
Robin Krom
b4bda47e7d
remote pkgs: resolve package names in data dependencies via ledger (#9270)
* remote pkgs: resolve package names in data dependencies via ledger

This implements part 3) of #8976.

This adds the ability to specify package names/versions in daml.yaml in
the data-dependencies stanza. They are being resolved via the project
ledger and a daml.lock lock file.

CHANGELOG_BEGIN
  [remote dependencies] Package names and versions, as well as package
  ID's are allowed in the `data-dependencies` list of `daml.yaml`. These
  packages are fetched from the project ledger. The auto-generated
  `daml.lock` file keeps track of the package name/version to package
  ID's resolution and should be checked in to version control of the
  project.
CHANGELOG_END

* added docs

* Update compiler/damlc/lib/DA/Cli/Damlc/DependencyDb.hs

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Update docs/source/daml/reference/packages.rst

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* suggestions

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2021-03-31 15:37:36 +02:00
Stefano Baghino
4fa78fa8c8
1.12.0-snapshot.20210331.6634.0.b98a3881 (#9307)
changelog_begin
changelog_end

![never-gonna-give-you-up](https://media1.giphy.com/media/Vuw9m5wXviFIQ/giphy.gif?cid=ecf05e47gesfxg3d3rqy8noh0knjbo88hu31ptpppjcczi48&rid=giphy.gif)
2021-03-31 13:25:20 +00:00
Stefano Baghino
b98a3881e5
Non-repudiation: documentation (#9288)
* Non-repudiation: documentation

changelog_begin
[Runtime Components] New feature: the non-repudiation middleware is now available
as an early access feature, please have a look at its page in the early access
section of the documentation to learn more
changelog_end

Closes #8659

* Remove exceprt from Wikipedia

* Fix typo

* Fix naming

* Address https://github.com/digital-asset/daml/pull/9288#discussion_r604647664

* Address https://github.com/digital-asset/daml/pull/9288#discussion_r604651195

* Address https://github.com/digital-asset/daml/pull/9288#discussion_r604792871
2021-03-31 11:30:54 +00:00
azure-pipelines[bot]
7c3d082ca0
update NOTICES file (#9300)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-03-31 13:24:54 +02:00
Stefano Baghino
cbd76c3465
Publish non-repudiation-core to Artifactory as well (#9305)
changelog_begin
changelog_end
2021-03-31 11:23:24 +00:00
Moritz Kiefer
48e273dc58
Bump y18n to address security vulnerability (#9301)
changelog_begin
changelog_end
2021-03-31 12:40:21 +02:00
Moritz Kiefer
e3ef08cc10
Avoid base64 encoding of cache suffixes (#9303)
base64 includes / which is at the very least pretty confusing and also
broke our cache cleanup which assumes that the cache suffix takes up
one directory. Afaict, we are not length-restricted in gcp paths so we
can just use the hex digits we get from md5

changelog_begin
changelog_end
2021-03-31 11:30:46 +02:00
Andreas Herrmann
4edefbc2d9
Generate DALFs for data-dependencies in Daml script dump (#9265)
* Don't mark dalfs in data-dependencies as main dalfs

This causes the LF version consistency check to fail on DALF data
dependencies that were generated with a different LF version and also
occur in dalfsFromDependencies. E.g. a dalf like `daml-prim-DA-Types`
triggers this issue.

changelog_begin
- [DAML Compiler] DALFs in data-dependencies that are imported directly
  now require corresponding `--package` flags to make them visible. The
  reason for this is that DALFs that are data-dependencies are no longer
  treated as main DALFs.
changelog_end

* Daml script dump write DALFs instead of DARs

Produces DALFs for dependencies and adds them to the data-dependencies.
Package flags for main DALFs are added to the build options.

* Update test-cases

* address review

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-03-31 10:14:56 +02:00
Stefano Baghino
1743672793
1.12.0-snapshot.20210331.6626.0.29a68134 (#9295)
changelog_begin
changelog_end
2021-03-31 07:38:08 +00:00
Stefano Baghino
a35400f461
Fix copy-pasta in HTTP JSON API EE options (#9296)
changelog_begin
changelog_end
2021-03-31 08:57:51 +02:00
Moritz Kiefer
29a6813465
Fix location of non-repudiation artifact (#9294)
This should end up in artifactory not github.

changelog_begin
changelog_end
2021-03-31 07:45:33 +02:00
Marton Nagy
bbfbe04f73
Fork com.daml.platform.store.dao (#9281)
Forking
- below LedgerDao interface
- into com.daml.platform.store.appendonlydao
- based on PoC code
This commit duplicates JdbcLedgerDao and dependencies as part of
the parallel ingestion rollout work. The fork will be removed as
the complete rollout is finished.

- adds appendonlydao package
- adapts LedgerDao: marks LedgerWriteDao methods for cleanup
- fixes error message for lookupMaximumLedgerTime

changelog_begin
changelog_end
2021-03-30 21:26:01 +00:00
Stefano Baghino
35fb66fccf
1.12.0-snapshot.20210330.6622.0.eef158d6 (#9292)
changelog_begin
changelog_end
2021-03-30 20:18:12 +00:00
Stefano Baghino
48c932b9e3
Allow to clean up more than one process listening on a port (#9293)
* Allow to clean up more than one process listening on a port

changelog_begin
changelog_end

* Address https://github.com/digital-asset/daml/pull/9293#pullrequestreview-624602747
2021-03-30 19:21:42 +00:00
Moritz Kiefer
eef158d6e8
Fix build command for non-repudiation-client (#9290)
The first attempt failed because the src jar wasn’t there. Afaict, the
path is fine but we didn’t build it.

changelog_begin
changelog_end
2021-03-30 18:36:34 +00:00
Moritz Kiefer
eb91dcf853
Only walk list once in splitAt (#9275)
* Only walk list once in splitAt

No idea if that makes a significant difference anywhere but walking
the list twice is definitely not faster.

changelog_begin
changelog_end

* Avoid hardcoded locations in tests

changelog_begin
changelog_end
2021-03-30 19:26:59 +02:00
Stefano Baghino
46c6701d8e
Prepare release 1.12.0-snapshot.20210330.6619.0.007b8674 (#9285)
changelog_begin
changelog_end
2021-03-30 17:05:03 +00:00
Robert Autenrieth
007b867489
DPP-313 migration safety check (#9279)
* Add test for the safety check

changelog_begin
changelog_end

* improve test assertion

* Remove unnecessary test
2021-03-30 12:28:40 +00:00
Remy
6c1f404c7d
LF: Add TO_TEXT_BIGNUMERIC (#9276)
This is part of #8719.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-30 13:31:42 +02:00
Samir Talwar
6035840d8a
Upgrade Scala 2.12 to v2.12.13. (#9280)
* Upgrade Scala 2.12 to v2.12.13.

This is being pulled in anyway because of Maven/Gradle/etc's fun
"favor the most recent" resolution mechanism. The version of Akka we
depend upon transitively depends on Scala 2.12.13, so any downstream
consumers will see that as the Scala version required.

Bringing the Daml repo in line means no more confusion.

CHANGELOG_BEGIN
- [Scala Bindings] If you are using Daml on Scala 2.12, it now depends
  on Scala v2.12.13 (from v2.12.12).
CHANGELOG_END

* Scala 2.12.13 is the default version in our pinned version of nixpkgs.

* Upgrade Scala 2.13's Wartremover version.

* Rename `scala_version_rule` to `scala_version_configure`.

* Add a test case to ensure the Scala versions are the same everywhere.

* Add tests for the Scala JAR versions in maven_install_*.json

* gatling-utils: Change the sort order of the expected CSV in tests.

I don't know why this changed, but it seems to be stable.

* compatibility: `scala_version` -> `scala_version_configure`.

* Bazel: Disable the Scala version tests on Windows.

* compatibility: Upgrade Wartremover to Scala 2.12.13.
2021-03-30 11:02:18 +00:00
Stefano Baghino
4f0c72b0d8
Publish non-repudiation client libraries to Artifactory (#9282)
* Publish non-repudiation client libraries to Artifactory

changelog_begin
changelog_end

Closes #9097

* Address https://github.com/digital-asset/daml/pull/9282#discussion_r603875118
2021-03-30 12:18:07 +02:00
Gary Verhaegen
45c4ba2230
macos cache cleaning (#9245)
This is adapting the same approach as #9137 to the macOS machines. The
setup is very similar, except macOS apparently doesn't require any kind
of `sudo` access in the process.

The main reason for the change here is that while `~/.bazel-cache` is
reasonably fast to clean, cleaning just that has finally caught up to us
with a recent cleanup step that proudly claimed:

```
before: 638Mi free
after: 1.2Gi free
```

So we do need to start cleaning the other one after all.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-30 02:46:05 +02:00
Edward Newman
a98e03981f
Increase nix partition to max of 60Gb (#9259)
Increase nix partition to max of 60Gb

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-30 01:06:58 +02:00
Robert Autenrieth
4a602d25cf
DPP-313 Append only schema migration (#9246)
* Add hidden option to enable experimental schema

* First version of the append-only schema

The schema is experimental,
the migration is locked behind a hidden feature flag.

changelog_begin
changelog_end

* Rename flag and add todos

* Improve description
2021-03-30 00:33:04 +02:00
Stephen Compall
480216e50b
Oracle json-api: more queries (#9159)
* support scalaz.Foldable1 in Fragments.in

* incorporating signatories and observers in Oracle contract query

* join syntax; allowed aggregation

* aggregate the signatories and observers independently before join

- prior: ERROR at line 8 (the GROUP BY line):
         ORA-00932: inconsistent datatypes: expected - got CLOB

* make toSqlWhereClause portable, mostly

* name the constraints for debugging

* import cleanup

* skip inserting contract on conflict (for read committed)

* support lookup by contract ID

* remove ::jsonb from fetch-by-key for Oracle

* proper key comparison and retrieval

* on conflict ignore for signatories and observers

* contract ID, party, offset, package ID column types

* template module and entity name types

- nvarchar2 for name type because

        ,template_module_name CLOB NOT NULL
        ,template_entity_name CLOB NOT NULL
        ,UNIQUE (package_id, template_module_name, template_entity_name)
        )
    , Error Msg = ORA-02329: column of datatype LOB cannot be unique or a primary key

* type-aware == and @> output for Oracle

* pick arbitrary maximum module/entity name size

      Cause: Error : 1450, Position : 0, Sql =
      CREATE TABLE
        template_id
        (tpid NUMBER(19,0) GENERATED ALWAYS AS IDENTITY NOT NULL PRIMARY KEY
        ,package_id NVARCHAR2(64) NOT NULL
        ,template_module_name NVARCHAR2(1594) NOT NULL
        ,template_entity_name NVARCHAR2(1594) NOT NULL
        ,UNIQUE (package_id, template_module_name, template_entity_name)
        )
    , Error Msg = ORA-01450: maximum key length (6398) exceeded

* happy path for query-less queries

* done todo

CHANGELOG_BEGIN
CHANGELOG_END

* handle 2.13 deprecation

* factor NVARCHAR2(255)s

- suggested by @cocreature; thanks

* deal with where only a signatory OR observer matches

- suggested by @cocreature; thanks
2021-03-29 15:29:12 -04:00
Moritz Kiefer
aa3508815e
Bump timeout in StaticTimeIT (#9273)
This times out occasionally on CI on Windows, e.g., https://dev.azure.com/digitalasset/adadc18a-d7df-446a-aacb-86042c1619c6/_apis/build/builds/73358/logs/201.

changelog_begin
changelog_end
2021-03-29 15:32:42 +00:00
Gary Verhaegen
cbba75a2c0
oracle: switch to entreprise (#9254)
* oracle: switch to entreprise

CHANGELOG_BEGIN
CHANGELOG_END

* --network host

changelog_begin
changelog_end

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2021-03-29 16:56:30 +02:00
Gerolf Seitz
8948e961db
Get codeowners for /ledger and /ledger-api in shape (#9266)
CHANGELOG_BEGIN
CHANGELOG_END
2021-03-29 15:45:34 +02:00
Sofia Faro
27dbf0bef9
BigNumeric in the stdlib cleanup. (#9253)
* Cleaning up BigNumeric somewhat.

Done in this PR:
- Moved BigNumeric and RoundingMode to GHC.Types to be alongside Numeric and Decimal
- Additive, Multiplicative, Number instances for BigNumeric
- Eq, Ord instances for BigNumeric and RoundingMode

Todo:
- BigNumeric literals in Daml
- Constructors and pattern matching for rounding mode
- Better documentation

changelog_begin
changelog_end

* remove ^ for now

* define (^) because it's required

* Error in (^) for negative powers.

* better error message

* import fromString and error

* Add -Wno-unused-imports because of compiling to different versions.

* add (fromString) and (error)

* Update bignumeric test.
2021-03-29 13:33:36 +00:00
Samir Talwar
45b4d63d16
kvutils: Fix a flaky test in BatchingQueueSpec with eventually. (#9268)
This is an async test, and therefore we can't rely on the queue
processing information immediately. We need to wait at least a little.

This change brought to you by failures seen in CI.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-29 12:55:53 +00:00
Samir Talwar
7fefeb2513
kvutils: Remove the conversions between Raw types. (#9267)
They have been deprecated since v1.11 and are due to be removed before
the next major release.

CHANGELOG_BEGIN
- [Integration Kit] The implicit conversions between `Raw` types (and
  the conversions to and from `ByteString`) have been removed. You will
  need to explicitly convert if necessary. This should not be necessary
  for most use cases.
CHANGELOG_END
2021-03-29 11:45:53 +00:00
Moritz Kiefer
a591accf04
Fix ledger API test tool exclusions (#9264)
This fixes the exclusions added in #9218. There are two issues:

1. Old ledger API test tool versions only support excluding full test
suites.
2. Apparently SDK 1.6.0 renamend SemanticDoubleSpend to SemanticDoubleSpendBasic.

changelog_begin
changelog_end
2021-03-29 10:29:42 +02:00
azure-pipelines[bot]
b4096cc967
update NOTICES file (#9263)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-03-29 08:56:48 +02:00
Moritz Kiefer
077d8da58f
Upgrade nixpkgs (#9248)
* Upgrade nixpkgs

changelog_begin
changelog_end

* ghc bump

changelog_begin
changelog_end

* more native buildinputs

changelog_begin
changelog_end

* Switch assignment

changelog_begin
changelog_end

* update lockfile

changelog_begin
changelog_end

* Revert "Switch assignment"

This reverts commit af6d25120b.
2021-03-26 21:26:40 +01:00
Moritz Kiefer
767e47fb94
Make occurence occur less often (#9258)
changelog_begin
changelog_end
2021-03-26 21:17:09 +01:00
Gary Verhaegen
205763e11a
correct notifs for new linux jobs (#9255)
Without this we sometimes get success notifications on Slack for failed
jobs.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-26 15:26:24 +00:00
Remy
d3dcfa932e
LF-Repl: run only scenarios marked as test (#9251)
CHANGELOG_BEGIN
CHANGELOG_END
2021-03-26 16:18:01 +01:00
Hubert Slojewski
cb422e2f67
KVL-810 Report the correct gRPC error code on failures due to concurrent transactions (#9218)
* Fix gRPC status codes for inconsistency rejections and DamlLf errors
Also, add unit tests and exclude failing compatibility and conformance tests

CHANGELOG_BEGIN

- [Integration Kit] Fix gRPC status codes for inconsistency rejections and DamlLf errors (ContractNotFound, ReplayMismatch) by changing them from INVALID_ARGUMENT to ABORTED

CHANGELOG_END
2021-03-26 15:26:33 +01:00
Gary Verhaegen
3a91cbccba
get oracle image from Docker Hub (#9252)
CHANGELOG_BEGIN
CHANGELOG_END
2021-03-26 14:43:04 +01:00
Moritz Kiefer
8790bd7d75
Upgrade ssri to address security warning (#9249)
I tried fixing it properly without a resolution but that requires an
upgrade to webpack 5 which runs into issues since the modernizr loader
we use does not support webpack 5 and at that point I gave up. I did
test navigator manually with this change.

changelog_begin
changelog_end
2021-03-26 13:01:23 +01:00
Remy
181afe4607
Daml: Add basic support for BigNumeric in 1.dev (#9243)
CHANGELOG_BEGIN
CHANGELOG_END
2021-03-26 09:00:05 +01:00
azure-pipelines[bot]
f9df848e61
update NOTICES file (#9247)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-03-26 08:24:23 +01:00
Kamil Bożek
c91d9ec3ff
Race condition test suite for the ledger-api-test-tool [DPP-274] (#9138)
* Early draft of the race condition ITs

* Archival vs Successful lookup by key test

* More descriptive failure messages

* Unsuccessful lookup vs non-transient creation test

* Double-archival test

* Fixed a test case name

* Archival vs Creation order test

* Reduced number of test templates

* Improved race test template naming

* Helper object with transaction and template utils

* Simplified transaction util

* Fixed wrong choice name

* Removed redundant println

* Formatted code changes

* Minor change

* CHANGELOG_BEGIN
- Integration Kit - added a test suite for race condition to the ledger-api-test-tool
CHANGELOG_END

* Removed unnecessary sorting of transactions

* Added explanatory comments to test cases

* Mechanism for running ledger-api-test-tool test cases multiple times

* Running each race condition test case 5 times

* Fixed WWArchiveVsNonTransientCreate test case

* Fixed flakiness of RWArchiveVsNonConsumingChoice

* Disabled RaceConditionIT in Canton tests

* Formatted code changes

* Moved RaceConditionIT to conformance tests with unique contract keys mode on for Canton

* Nicer delay mechanism

* Improved WWArchiveVsNonTransientCreate to take contention into account

* Fixed RWTransientCreateVsNonTransientCreate conditions for Canton

* Increased the delay before reading the transaction trees stream to 1 second

* Fixed incorrect conformance tests definition for RaceConditionIT

* Running race condition tests sequentially to avoid timeouts

* Simplified race condition test case definition

* Return sum of durations for repeating test cases in the ledger-api-test-tool

* Reverted previous change with computing sum of durations

* Exclude RaceConditionIT from sandbox-on-x conformance tests

* Print the number of a test run only for cases when the number of repetitions is > 1

* Fixed RWArchiveVsFetch scenario
2021-03-25 23:07:01 +01:00
Remy
6d90985667
Speedy: cleanup dynamic casting in builtins
CHANGELOG_BEGIN
CHANGELOG_END
2021-03-25 18:32:11 +01:00