Commit Graph

8492 Commits

Author SHA1 Message Date
Hubert Slojewski
445031114f
Exclude/fix compatibility tests (#11529)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-04 19:50:49 +01:00
azure-pipelines[bot]
699c28f59e
rotate release duty after 1.18.0-snapshot.20211102.8257.0.7391a3cd (#11522)
@robin-da is taking care of 1.18.0-snapshot.20211102.8257.0.7391a3cd (#11521), so they get pushed back to the end of the line.

Please do not merge this before #11521.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines Daml Build <support@digitalasset.com>
2021-11-04 19:49:41 +01:00
Sofia Faro
4d4869f06e
interfaces: Improve fixed choice lookup (#11551)
* interfaces: Improve fixed choice lookup

This PR adds a lazy map to match fixed choice names to their
interface, to improve the worst case for choice lookup.

Part of #10810

changelog_begin
changelog_end

* scalafmt

* apply improvements from remy
2021-11-04 18:44:48 +00:00
Marton Nagy
ba96bf784c
Add DbDtoToStringsForInterning [DPP-708] (#11557)
* Adds collecting functions and unit tests

changelog_begin
changelog_end
2021-11-04 18:12:08 +00:00
Remy
692dac665d
LF: deprecate remaining type alias in Transaction (#11554)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-04 18:02:16 +00:00
nicu-da
7bc0f82733
[kvutils] - Extract command deduplication steps from the TransactionCommitter [KVL-1174] (#11547)
CHANGELOG_BEGIN

CHANGELOG_END
2021-11-04 16:29:04 +00:00
Marton Nagy
0e95ccb354
Wire LedgerEndCache to relevant StorageBackend factories [DPP-704] (#11549)
* Add ledgerEndCache dependency to relevant StorageBackend factory methods
* Wire ledgerEndCache to appropriate places
* REFACTORING: StorageBackendTests to instantiate TestBackend, wire ledgerEndCache update
* REFACTORING: RecoveringIndexerIntegrationSpec to simplify party existence, wire ledgerEndCache update
* REFACTORING: Remove StorageBackend super-trait, move StorageBackend DTO-s
* REFACTORING: Move common part of StorageBackendFactory to CommonStorageBackendFactory

changelog_begin
changelog_end
2021-11-04 15:39:44 +01:00
Remy
73cb42e102
LF: deprecate ensureNoCid and assertNoCid in CidContainer (#11502)
After #10827 that drops type parameters in Value, `CidContainer`'s
`ensureNoCid` and `assertNoCid` loose their interest.
To check a value does not contain Contract Ids one should use
`foreachCid` or `cids` methods instead.

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-04 14:49:19 +01:00
Oliver Seeliger
30bfee9822
Upgrade canton for conformance tests (#11550)
This also eases the work on:
- error codes - fyi @tudor-da
- open sourcing

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-04 13:10:48 +00:00
Stefano Baghino
b84a9142d1
Extractor: replace sandbox-classic-based fixtures with sandbox-based ones (#11489)
* Replace `sandbox-classic`-based fixtures with `sandbox`-based ones

changelog_begin
changelog_end

* Replace Scenarios with Script for BasicPrimitiveTypes

* Adapt ListsSpec

* Adapt TextMapsSpec

* Adapt OptionalSpec

* Adapt more tests

* More tests passing

- removed workflow_id testing (Daml Script doesn't set one)
- reduced failure threshold for VeryLargeArchiveSpec
- solved issu in TransactionSingleTableSpec (wrong parties were used)

* Port EnumMod and make EnumSpec pass

* Port GenMapMod and make GenMapSpec pass

* Shaking things around to make Windows happy -- maybe
2021-11-04 13:08:51 +00:00
Kamil Bozek
d86fe92950
ledger-api-bench-tool - flexible stream filters [DPP-667] (#11458)
* ACS filters

* Moved filters to a separate object

* Filters for transactions and transaction trees streams

* Updated help

CHANGELOG_BEGIN
- [Integration Kit] - ledger-api-bench-tool - added flexible party and template filters for transactions, transaction trees and active contract streams
CHANGELOG_END

* Changed filter separators to + and @
2021-11-04 12:08:38 +01:00
fabiotudone-da
214c7ab445
Self-service errors: generate groups documentation [KVL-1186] (#11526)
* Add groups information to JSON

CHANGELOG_BEGIN
CHANGELOG_END

* Code cleanup

* Fix docs generation

* Fix KVErrors group explanation string

* Render error group explanations in generated docs

* Fix ErrorCodeDocumentationGeneratorSpec
2021-11-04 12:07:29 +01:00
Moritz Kiefer
60dc2869e7
Avoid unnecessarily decoding DALFs during dependency installation (#11541)
We only need the package id which only requires decoding the header
which is exactly what decodearchivePackageId exists for.

On a large project this seems like a 10-20s improvement which is
pretty significant.

changelog_begin
changelog_end
2021-11-04 11:36:39 +01:00
Moritz Kiefer
afd79b6729
Improve logging for dependency installation (#11540)
Also drops the weird Base module and the always empty tags (yet
another reason why I want to burn our own logging lib with fire).

changelog_begin
changelog_end
2021-11-04 11:36:03 +01:00
Gary Verhaegen
fdde5353f4
fix hoogle by pinning nixpkgs (#11548)
Hoogle has been down for at least 24h accoridng to user reports. What
seems to be happening is that our nixpkgs pinning is not taking effect,
and the nixpkgs version of Hoogle already includes the patch we are
trying to add. This confuses nix, which fails, and thus the boot
sequence is broken.

I've applied the minimal possible patch here (i.e. enforce the pin),
which gets things running again. I've already deployed this change.

We may want to look at bumping the nixpkgs snapshot.

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-04 10:22:30 +00:00
tudor-da
5e93df6b52
Full conformance test with self-service error codes on ledger-on-memory (#11524)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-04 09:36:06 +01:00
azure-pipelines[bot]
efaf20b042
update NOTICES file (#11546)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-11-04 08:40:08 +01:00
akshayshirahatti-da
c7bb94be65
Modify contract key attribute age for createAndExercise to avoid contract key violation on slow running tests (#11530)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-04 05:40:13 +00:00
Marton Nagy
541d07c3a9
Add string-interning data access [DPP-707] (#11484)
* Add new table, and party_id to party_entries
* Add write side basics (DbDto, Schema)
* Add read side basic (StorageBackendStringInterning)
* Add StorageBackend unit test
* Support for all DB backends

changelog_begin
changelog_end
2021-11-04 02:07:32 +00:00
Marton Nagy
6c9416492d
Switching to composition at StorageBackend [DPP-704] (#11520)
* Switch to StorageBackendFactory
* Fixing tests

changelog_begin
changelog_end
2021-11-03 22:53:05 +00:00
Marton Nagy
d006ad0e8b
Add string-interning view primitives [DPP-702] (#11475)
changelog_begin
changelog_end
2021-11-03 20:27:10 +00:00
nicu-da
a917520851
Save the submission time in the command deduplication value [KVL-1173] (#11509)
CHANGELOG_BEGIN

CHANGELOG_END
2021-11-03 18:30:03 +00:00
Marton Nagy
669186f118
Add priority-queue based worker Source [DPP-717] (#11500)
* Add ACSReader trait, and primitives
* Add pullWorkerSource implementation + unit tests

changelog_begin
changelog_end
2021-11-03 17:59:57 +00:00
nicu-da
2091f59aac
Remove cats from kvutils (#11537)
CHANGELOG_BEGIN

CHANGELOG_END
2021-11-03 17:40:04 +00:00
Moritz Kiefer
b83af37077
Fix recache log statement (#11536)
turns out I cannot type

changelog_begin
changelog_end
2021-11-03 17:09:10 +00:00
Sofia Faro
776790e386
Add choice collisions in scala name collision checker (#11528)
* Add choice collisions in scala name collision chk

This brings it in line with the LF spec.
This PR also adds collisions for inherited (interface) choices.
Closes #11137.

changelog_begin
changelog_end

* scalafmt

* Apply suggestions from code review

Co-authored-by: Remy <remy.haemmerle@daml.com>

Co-authored-by: Remy <remy.haemmerle@daml.com>
2021-11-03 17:01:34 +00:00
nicu-da
2efcb21f3e
Extract the setting of the time bounds on the commit context [KVL-1173] (#11516)
CHANGELOG_BEGIN

CHANGELOG_END
2021-11-03 14:39:47 +00:00
nicu-da
9bb1d6443c
[ledger-api] - Return FAILED_PRECONDITION gRPC status code for invalid deduplication duration [KVL-1170] (#11483)
CHANGELOG_BEGIN
[ledger-api] - Return FAILED_PRECONDITION gRPC status code, instead of INVALID_ARGUMENT, for an invalid command deduplication duration
CHANGELOG_END
2021-11-03 13:34:31 +00:00
Moritz Kiefer
c098d75621
Add a --log-level flag to damlc (#11514)
This also allows us to quite down our builds a bit which is nice.

changelog_begin
changelog_end
2021-11-03 14:22:57 +01:00
Moritz Kiefer
afee94858f
Bump rules_nixpkgs (#11517)
We need to make some changes to have any chance of ARM support working
and I’d rather make those on top of the latest version.

Also deletes an unused patch.

changelog_begin
changelog_end
2021-11-03 12:02:46 +01:00
azure-pipelines[bot]
d6fb67c71e
release 1.18.0-snapshot.20211102.8257.0.7391a3cd (#11521)
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.

@robin-da is in charge of this release.

Commit log:
```
7391a3cdcd Migrate language-support codegen tests to sandbox (#11508)
da76e2825b iface: check for interface implementations precond (#11494)
9e045c105b Remove a leftover from `check-protobuf-stability.sh` (#11513)
17c2fa9122 ErrorCodeDocumentationGenerator uses multiple prefixes for scanning classpaths (#11504)
a8ae4d6f9c LF: Rename GenNode to Node (#11469)
73b6f8b613 Fixed ledger-api-bench-tool random distribution in submission phase. (#11507)
92495b0662 Upgrade nixpkgs (#11490)
48afb7b5ea interface: Add inherited choice names in scala AST (#11503)
9f8b039be3 [Trigger-Service] Migrate tests to sandbox (#11501)
6bafeb03e8 DPP-662 Support archivals in bench tool (#11474)
63d855f202 Pass `damlc build` logs through daml logger (#11485)
565dfac2e5 Bump rules_nodejs to 4.4.2 (#11495)
9214956218 Enable experimental nix command (#11498)
f81d880bcc LF: drop com.daml.lf.transaction.Transaction.Transaction type alias. (#11470)
505a7f59f3 Extend debug logging around dependencies/data-dependencies (#11488)
55d35e5521 Drop TLA+ from dev-env (#11492)
3a5cb18e7f [Self-service error codes] Adapt all grpc assertions to additionally take self-service error codes [DPP-680] (#11437)
98e7461fc5 LF: ContractInstance should be a CidContainer (#11487)
9afb3b36bd Add and wire ledger-end-cache [DPP-703] (#11480)
cdd2accc26 Explicitly import Java runtime from Nix/dadew (#11411)
b5540a9c3b Evidence for contract-key testing (#11479)
78735a8d27 Less logging on //docs:generate-error-codes-json (#11486)
d8898b8c80 Check fixed choices in name collision checker (#11481)
7a0198ba52 daml-react: allow for React 17 (#11463)
c415d9d1ee Migrate trigger tests from sandbox-classic to sandbox (#11478)
f9ec12be12 Add cocreature as a codeowner of the docs (#11450)
a79cc08687 DbDispatcher handles SQL exceptions with self-service error codes [DPP-690] (#11435)
164cdbbd7d update compat versions for 1.18.0-snapshot.20211026.8179.0.e474b2d1 (#11426)
741a6e75a7 interfaces: Add list of fixed choices in TemplateImplements structure. (#11364)
79505b5474 oracle debug (#11386)
257d5365c2 Switch from sandbox to sandbox-classic in daml test-script (#11476)
e91ced54f3 KV: generate self-service error docs [KVL-1148] (#11431)
8ffd835f24 Migrate daml script tests to Sandbox next (#11472)
df65d02c4e Exclude dedup tests affected by current changes (#11471)
45beaf7aac interfaces: typecheck for precond, haskell side. (#11468)
e445fe1aee LfEngineToApi conversions do not have to deal with version. (#11414)
d3712745e2 Sandbox-classic: contracts sent to the engine do not need enrichment (#11416)
05312f1ca0 Lift out constraint tuples to synonyms in dfun contexts (#11467)
a492a6cad0 [DPP-636] Update docs on MacOS firewall issue when running tests (#11460)
5a85881cfa rotate release duty after 1.18.0-snapshot.20211026.8179.0.e474b2d1 (#11422)
1096f9144b Adds default keepalive support for postgreSQL [DPP-697] (#11453)
379996348f Fix akka-streams shutdown at LedgerConfigurationSubscriptionFromIndex (#11462)
e89da7c1ed Move non-repudiation proxy conformance tests to their own package (#11464)
b98a3ade29 interfaces: precondition, scala ast, decoder/encoder (#11452)
fd973e6b86 Drop assistant integration tests for relative/absolute DAML_PROJECT (#11461)
05505e3d03 Multiplex multiple virtual query streams through a single web socket (#10221)
3098b70624 pretty print contract IDs properly (#11359)
bb4f4c5a40 DPP-665 Make in-flight commands configurable (#11456)
bf00956143 replace OneAnd party Sets with NonEmpty Set (#11420)
570160b718 Add LoggingValue instance for Time.Timestamp (#11444)
bad50a51e9 Type reexports explicitly use 'type' namespace (#11451)
d3b9c84908 ifaces: preconditions, proto changes (#11448)
73718a22fa Mark script export tests as flaky (#11449)
22e9e6b56e Multi-observer command submission in the ledger-api-bench-tool [DPP-660] (#11413)
3eacbf9d67 Bazel cache retry on connection reset by peer (#11445)
fcc7e581cf Make stable packages dependent on supported LF versions (#11382)
6c45f099bb Remove TODO (#11446)
d8b92b7c99 Drop flaky marker from lsp-tests (#11442)
68f44325a6 Improve evidence of testing (#11428)
6126fc2230 Remove append only prefix from conformance tests [KVL-1152] (#11424)
926655d3c3 Drop daml ledger timeout tests from compat tests (#11441)
15cb84093b Switch back to binary execution logs (#11440)
ac28e615b8 Update ScalaPB to the latest version (0.11.6) (#11409)
a71a8f999e Prevent overflow in DeduplicationPeriod (#11432)
6c88e55509 Pass submission id as correlation id to error codes in Ledger API (#11381)
ba106873b9 Replace 1.14 min version conformance test by custom test (#11433)
a600f75c9f Bump bazel memory limit to 3gb (#11436)
ebe742098d fix es ingest: file too large (#11415)
d86eca0b05 Add documentation annotations to KV self-service errors [KVL-1144] (#11385)
46d31d5d48 [DPP-676][Self-service error codes] Adapt ApiConfigManagementService (#11402)
927b2378c3 Make Oracle tests exclusive (#11408)
68d7f83a5f Switch bazel exec logs to json format (#11430)
12e782b1ea ifaces: update ghc-parser (#11383)
503e39160d [Short] Typo (#11410)
3031190994 [DPP-676][Self-service error codes] Adapt request validators (#11379)
b412f7191d Fix MonotonicRecordTimeIT for pruned participants (#11423)
500e5b068e release 1.18.0-snapshot.20211026.8179.0.e474b2d1 (#11421)
d678a4046d LF: remove type parameter from ContractInstance (#11419)
```
Changelog:
```

- [@daml/react] Our React bindings now request the `react` library as a
  peer dependency, and will accept React 17 in addition to the current
  React 16. For new projects, the `create-daml-app` template now uses
  React 17.

- [Integration Kit] Document KV-specific self-service errors

[TS bindings] When using daml-react and daml-ledger, all streaming request to the query endpoint will be multiplexed through a single web socket.
[daml LF] - Add LoggingValue instance for Time.Timestamp
- [Integration Kit] - ledger-api-bench-tool can generate synthetic contracts with multiple observers
[api-test-tool] - remove AppendOnly prefix from test suites as the append-only schema is the only one left
```

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines Daml Build <support@digitalasset.com>
2021-11-03 11:56:30 +01:00
Moisés Ackerman
ac49d69e13
Ensure that -Werror converts parser warnings into errors (#11511)
closes #11399

changelog_begin
changelog_end
2021-11-03 11:28:52 +01:00
Samir Talwar
bda35ca234
ledger-offset: Add a conversion to ByteString. (#11491)
* ledger-offset: Add a conversion to ByteString.

I seem to be doing this a lot.

CHANGELOG_BEGIN
CHANGELOG_END

* ledger-offset: Add a conversion from ByteString.
2021-11-03 09:54:23 +00:00
pbatko-da
8a9f15b8de
[DPP-686][Self-service error codes] Removing default error factories (#11403)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-03 09:40:19 +01:00
Moritz Kiefer
bb37eef156
Switch script & trigger tests to postgres-backed Sadbonx (#11525)
DPP-695 is gift that keeps on giving and seems to also result in
timeouts here. Just marking all of our tests that depend on Sadbonx as
flaky doesn’t seem all that helpful so switch to a postgres-backed
sandbox instead.

changelog_begin
changelog_end
2021-11-03 08:39:46 +00:00
akshayshirahatti-da
7391a3cdcd
Migrate language-support codegen tests to sandbox (#11508)
* Migrate language-support codegen tests to sandbox

CHANGELOG_BEGIN
CHANGELOG_END

* changes to create unique parties via party admin service
2021-11-02 18:11:12 +00:00
Robin Krom
da76e2825b
iface: check for interface implementations precond (#11494)
* iface: check for interface implementations precond

We change speedy to also check all preconditions defined in interface
implementations upon a create.

CHANGELOG_BEGIN
CHANGELOG_END

* Update daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/speedy/Compiler.scala

Co-authored-by: Remy <remy.haemmerle@daml.com>

* format

* interfaces: lfconversion for iface preconditions

This adds interface preconditions to the interface implementations
during completion phase in the LF conversion.

CHANGELOG_BEGIN
CHANGELOG_END

* format

* rebase

Co-authored-by: Remy <remy.haemmerle@daml.com>
2021-11-02 17:27:49 +00:00
Hubert Slojewski
9e045c105b
Remove a leftover from check-protobuf-stability.sh (#11513)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-02 17:11:17 +00:00
tudor-da
17c2fa9122
ErrorCodeDocumentationGenerator uses multiple prefixes for scanning classpaths (#11504)
* Removed ProtoDeserializationErrors (not needed at the moment)

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-02 17:09:08 +00:00
Remy
a8ae4d6f9c
LF: Rename GenNode to Node (#11469)
Following up #10827 and #10921 that drop type parameters from GenNode,
we rename GenNode into Node.

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-02 16:42:16 +00:00
Kamil Bozek
73b6f8b613
Fixed ledger-api-bench-tool random distribution in submission phase. (#11507)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-02 17:28:12 +01:00
Moritz Kiefer
92495b0662
Upgrade nixpkgs (#11490)
* Upgrade nixpkgs

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end
2021-11-02 15:52:55 +01:00
Sofia Faro
48afb7b5ea
interface: Add inherited choice names in scala AST (#11503)
Also checks that they're right. There's no collision checking yet.

Part of #11137.

changelog_begin
changelog_end
2021-11-02 14:35:37 +00:00
akshayshirahatti-da
9f8b039be3
[Trigger-Service] Migrate tests to sandbox (#11501)
* Migrate trigger service tests to sandbox from sandbox-classic

CHANGELOG_BEGIN
CHANGELOG_END

* remove config override for EngineMode.Dev based on code review
2021-11-02 14:35:30 +00:00
Robert Autenrieth
6bafeb03e8
DPP-662 Support archivals in bench tool (#11474)
* Support archivals in bench tool

changelog_begin
changelog_end

* Rename archive_chance
2021-11-02 15:09:09 +01:00
Moritz Kiefer
63d855f202
Pass damlc build logs through daml logger (#11485)
* Pass `damlc build` logs through daml logger

This gives us some timings which can be useful in some
circumstances (I want timings around data-dependencies next).

I also changed the default log level to INFO. The `damlc build` output
doesn’t get any noisier from that.

Example output:

```
2021-11-01 15:25:17.16 [INFO]  [build]  []
Compiling foobar to a DAR.

2021-11-01 15:25:17.80 [INFO]  [build]  []
Created .daml/dist/foobar-0.0.1.dar
```

changelog_begin
changelog_end

* Adjust scenario service log level

changelog_begin
changelog_end

* Adjust tests

changelog_begin
changelog_end

* delete comment

changelog_begin
changelog_end
2021-11-02 12:10:09 +00:00
Moritz Kiefer
565dfac2e5
Bump rules_nodejs to 4.4.2 (#11495)
Ran into an issue on M1 and there has been some recent work around
that area. Not sure this actually fixes it but why not upgrade anyway.

changelog_begin
changelog_end
2021-11-02 12:02:30 +01:00
Moritz Kiefer
9214956218
Enable experimental nix command (#11498)
* Enable experimental nix command

Nix 2.4 got released which confusingly marked the nix command as
experimental despite it being actively worked on and being more stable
than before.

changelog_begin
changelog_end

* switch to CLI flag

changelog_begin
changelog_end

* Revert "switch to CLI flag"

This reverts commit 58d6a25d83.
2021-11-02 10:16:38 +00:00
Remy
f81d880bcc
LF: drop com.daml.lf.transaction.Transaction.Transaction type alias. (#11470)
After dropping type parameter from GenTransaction we do not
com.daml.lf.transaction.Transaction.Transaction type alias.  We use
instead directly com.daml.lf.transaction.VersionedTransaction, wich is
anyway more informative.

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-02 10:23:17 +01:00
Moritz Kiefer
505a7f59f3
Extend debug logging around dependencies/data-dependencies (#11488)
* Extend debug logging around dependencies/data-dependencies

changelog_begin
changelog_end

* stop stealing by $s hlint

changelog_begin
changelog_end
2021-11-02 09:34:45 +01:00