Commit Graph

8519 Commits

Author SHA1 Message Date
Moisés Ackerman
cfd521ff86
Fix GHC build errors on macOS (#11560)
changelog_begin
changelog_end
2021-11-09 14:42:52 +01:00
Remy
a7559d047e
LF: Rename GenTransaction to Transaction (#11599)
* LF: Rename GenTransaction to Transaction

Following up #10827 and #10921 that drop type parameters from GenNode,
we rename GenTransaction to Transaction.

CHANGELOG_BEGIN
CHANGELOG_END

* fix

* a bit more fixes
2021-11-09 13:30:34 +00:00
Remy
3b61a1b748
LF: allow exercise command by template for inherited choice. (#11584)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-09 13:10:06 +00:00
Robin Krom
af1bee7b82
precondition lf conversion (#11538)
* interfaces: lf conversion of preconditions

This adds the LF conversion of preconditions from the desugared GHC code
to Daml LF for preconditions. This depends on the corresponding patch of
the GHC fork.

CHANGELOG_BEGIN
CHANGELOG_END

* pin linux stackage snapshot

* added a test

CHANGELOG_BEGIN
CHANGELOG_END

* pinned linux stackage snapshot

CHANGELOG_BEGIN
CHANGELOG_END

* additional InterfacePrecondition test

this checks that that conjunction of all preconditions of intererfaces
and templates is checked.

CHANGELOG_BEGIN
CHANGELOG_END

* pin stackage snapshot

* pin windows stackage snapshot

* update compile.yml (again)

* pin linux stack snapshot

* fix InterfacePrecondition test

* pin windows stackage
2021-11-09 11:43:46 +00:00
Moritz Kiefer
f722cf13cd
Bump extractor init timeout to 60s (#11598)
I’ve seen this timeout at 30s on CI under load.

changelog_begin
changelog_end
2021-11-09 11:14:50 +00:00
Andreas Herrmann
1e0a221379
Set symlink_node_modules = False (#11606)
By default `yarn_install` will install `node_modules` in the repository.
The advantage is that editor tooling and the like can use this
`node_modules` tree without needing to duplicate it between Bazel and
tooling. The downside is that Bazel has less strict control over this
tree. In particular, on refetch it can happen that old files remain in
the tree and confuse the build going forward.

For example, we have occasionally observed errors of the form
```
warning Cannot find a suitable global folder. Tried these: "/usr/local, /does-not-exist/.yarn"
 ERROR  The following files have the same case insensitive path, which isn't supported by the VSIX format:
  - extension/node_modules/form-data/Readme.md
  - extension/node_modules/form-data/README.md
```

Setting this flag to `False` means that `node_modules` will instead be
installed under Bazel's output base and Bazel can make sure to clear the
tree before a refetch.

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-11-09 11:55:27 +01:00
Moritz Kiefer
ac19c8cff5
Synchronize in multi-participant script tests (#11594)
* Synchronize in multi-participant script tests

This hopefully fixes the flakiness where the party allocation is not
yet propagated to the other participant.

changelog_begin
changelog_end

* .

changelog_begin
changelog_end
2021-11-09 09:27:54 +00:00
Gerolf Seitz
dd1b0347ec
Restart the submission interpretation in case of a race [DPP-737] (#11579)
Command interpretation happens in two stages:
1. Daml interpretation
2. Determine a suitable ledger effective time

The race can happen in the following situation:
In 1) a contract key K is resolved to contract C1.
Between 1) and 2), a transaction is stored on the participant that
archives C1, but creates C2 with the same contract key K.
In 2) the ledger api server tries to lookup the ledger effective time
for all input contracts to the transaction. If it doesn't find one of
the input contracts at all, it can conclude that the transaction
wouldn't be accepted by the ledger anyway.

The behavior before this patch was to simply abort command
interpretation and return a rather cryptic error to the user.
With this patch, the ledger api server restarts the command
interpretation.
If the "missing contract" was an explicit input to the
command (e.g. as the contract for the exercise or as an argument to an
exercise), then this command will be rejected because the contract is
now archived.
If the contract ID was determined via a contract key lookup, then
restarting the interpretation will result in either a negative lookup or
a different contract ID for the new contract under this contract key.

CHANGELOG_BEGIN
[Ledger API] Retry the interpretation of a command in case of a race
with other transactions. This fix drastically reduces the likelihood of the error
"Could not find a suitable ledger time after 0 retries".
CHANGELOG_END
2021-11-09 10:03:57 +01:00
akshayshirahatti-da
8d2b1b9ffe
[JSON-API] Port of http-perf-test to sandbox(aka next) (#11543)
* Changes to migrate http-perf-test back to sandbox with more parallelization for single user scenarios.
Increased parallelization is due to the architectural changes in sandbox where it uses
a tick every 100 millis to trigger stuff/data to be available on the read side

CHANGELOG_BEGIN
CHANGELOG_END

* Parallelization fixes for scenarios ExerciseCommand and SyncQueryNewAcs scenarios

* refactor sequential scenario run, make query part of SyncQueryVariableAcs run with single user
2021-11-09 08:44:37 +00:00
Simon Meier
56a6db8f73
codeowners: reflect change in onwership of meiersi-da (#11505)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-09 09:02:15 +01:00
Andreas Lochbihler
d4336510af
fix a few typos in Daml-LF spec (#11583)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-09 08:36:28 +01:00
Marton Nagy
5dee88b038
Refactoring store factories [DPP-709] (#11572)
* Pulling up DbDispatcher factory form JdbcLedgerDao
* Adapt usage
* Moving ReadStorageBackend factory
* Minor SQL fixing (parameter join remnants)

changelog_begin
changelog_end
2021-11-08 22:53:32 +01:00
Oliver Seeliger
41fb2890d6
Port LedgerApiServer rate limiter interface from canton (#11577)
* Port LedgerApiServer/ApiSubmissionService rate limiter interface from canton

Also includes ability to specify `AsyncCommitMode` indexer config (to get around
`DbType` being platform-package private).

changelog_begin
changelog_end

* Review feedback

* Review feedback - rely on DbType becoming public in #11572

* Review feedback including unit test
2021-11-08 19:55:57 +00:00
Moritz Kiefer
04cc5d7a76
Mark JSON API integration tests as flaky (#11586)
changelog_begin
changelog_end
2021-11-08 18:47:26 +00:00
Moritz Kiefer
3bed7a9b4f
Drop unused defaultFeatureFlags (#11580)
I believe this is still from Daml 1.1, it’s definitely not the default
right now so this is at best confusing.

changelog_begin
changelog_end
2021-11-08 17:54:48 +01:00
Sofia Faro
cbc3c8c686
Add by_interface field in Create, Fetch, & Exercise transaction nodes. (#11576)
* Add by_interface field in tx nodes.

This PR adds an optional `by_interface` field in the Create, Fetch,
and Exercise nodes, as part of #10915. It also updates TransactionCoder
to support these fields.

Setting these fields (in the LF interpreter) is left to a separate PR.

changelog_begin
changelog_end

* update security-evidence.md

* Add version checks in TransactionCoder
2021-11-08 16:33:02 +00:00
Moisés Ackerman
7d68e05f7f
Remove virtual choices (#11482)
* Remove virtual choices

* Remove choices without a body in 'interface' definition
* Remove choices in 'template ... implements' section

part of #11372

changelog_begin
changelog_end

* Remove virtual choices cont.

Switch uses of virtual choices to fixed choice with method implementation

* update snapshot after pin on windows

* Disable failing interface tests with TODO #10810
2021-11-08 17:05:23 +01:00
nicu-da
9b94fa9769
[ledger-api] - metadata for invalid deduplication period [KVL-1170] (#11534)
CHANGELOG_BEGIN
[ledger-api] - Return max_deduplication_duration as part of the metadata sent with the gRPC status for commands rejected because of invalid deduplication periods
CHANGELOG_END
2021-11-08 14:45:35 +00:00
azure-pipelines[bot]
e7eb60f087
update compat versions for 1.18.0-snapshot.20211102.8257.0.7391a3cd (#11531)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-11-08 13:47:27 +00:00
Moritz Kiefer
a3d04cafaf
Migrate matches-docs tests to sandbox on postgres (#11573)
* Migrate matches-docs tests to sandbox on postgres

We’ve seen timeouts on party allocation which are likely caused by the
h2 issues that keep popping up. Switching to postgres should hopefully
solve that.

We need to wrap postgres to set LOCALE_ARCHIVE for this to work in
builds. We already pass it through in tests which is why it works fine there.

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* Update nix/bazel.nix

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>

* disable on Windows

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
2021-11-08 13:39:02 +00:00
tudor-da
21c97453bd
[Self-service error codes] ValueSwitch parameterized method (#11567)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-08 12:11:56 +00:00
Moritz Kiefer
6a0b84e3d0
Run Script export tests against BR (#11574)
More h2 garbage

changelog_begin
changelog_end
2021-11-08 10:41:09 +00:00
Gary Verhaegen
a3195fd62d
fix link to blog (#11570)
I'm not entirely sure that this is the intended link (v. say
`https://daml.com/blog`), but the current one is dead so presumably this
is better.

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-08 10:38:37 +01:00
fabiotudone-da
08236012af
Self-service errors: generate error deprecation docs [KVL-1187] (#11539)
* Render error deprecation in generated docs

CHANGELOG_BEGIN
CHANGELOG_END

* Build export-error-codes-json-app with Scala 2.13 only

* Generate docs only with Scala 2.13

* Generate daml-on-sql docs only with Scala 2.13

* Split the error codes JSON generator off from the error lib

* Build all docs only with Scala 2.13
2021-11-05 01:23:38 +01:00
Robert Autenrieth
8d48cf669d
DPP-664 Create contracts in batches (#11515)
* Create contracts in batches through a factory contract

changelog_begin
changelog_end

* Use batching on protobuf level

* Revert needless changes

* Address review comments
2021-11-05 00:09:47 +01:00
Marton Nagy
9ef2eaf803
Remove parameter table join from SQL queries [DPP-705] (#11562)
* Add wiring of ledger-end to necessary places
* change SQL queries to use ledger-end-cache instead of parameter table join
* Fixes tests/test suite DB providers

changelog_begin
changelog_end
2021-11-04 21:16:07 +01:00
Moritz Kiefer
af7f7f3f01
Fix LF coedowners (#11563)
GH uses the last matching rule not all matching rules so unfortunately
we have to duplicate entries.

changelog_begin
changelog_end
2021-11-04 20:02:52 +00:00
Hubert Slojewski
445031114f
Exclude/fix compatibility tests (#11529)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-04 19:50:49 +01:00
azure-pipelines[bot]
699c28f59e
rotate release duty after 1.18.0-snapshot.20211102.8257.0.7391a3cd (#11522)
@robin-da is taking care of 1.18.0-snapshot.20211102.8257.0.7391a3cd (#11521), so they get pushed back to the end of the line.

Please do not merge this before #11521.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines Daml Build <support@digitalasset.com>
2021-11-04 19:49:41 +01:00
Sofia Faro
4d4869f06e
interfaces: Improve fixed choice lookup (#11551)
* interfaces: Improve fixed choice lookup

This PR adds a lazy map to match fixed choice names to their
interface, to improve the worst case for choice lookup.

Part of #10810

changelog_begin
changelog_end

* scalafmt

* apply improvements from remy
2021-11-04 18:44:48 +00:00
Marton Nagy
ba96bf784c
Add DbDtoToStringsForInterning [DPP-708] (#11557)
* Adds collecting functions and unit tests

changelog_begin
changelog_end
2021-11-04 18:12:08 +00:00
Remy
692dac665d
LF: deprecate remaining type alias in Transaction (#11554)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-04 18:02:16 +00:00
nicu-da
7bc0f82733
[kvutils] - Extract command deduplication steps from the TransactionCommitter [KVL-1174] (#11547)
CHANGELOG_BEGIN

CHANGELOG_END
2021-11-04 16:29:04 +00:00
Marton Nagy
0e95ccb354
Wire LedgerEndCache to relevant StorageBackend factories [DPP-704] (#11549)
* Add ledgerEndCache dependency to relevant StorageBackend factory methods
* Wire ledgerEndCache to appropriate places
* REFACTORING: StorageBackendTests to instantiate TestBackend, wire ledgerEndCache update
* REFACTORING: RecoveringIndexerIntegrationSpec to simplify party existence, wire ledgerEndCache update
* REFACTORING: Remove StorageBackend super-trait, move StorageBackend DTO-s
* REFACTORING: Move common part of StorageBackendFactory to CommonStorageBackendFactory

changelog_begin
changelog_end
2021-11-04 15:39:44 +01:00
Remy
73cb42e102
LF: deprecate ensureNoCid and assertNoCid in CidContainer (#11502)
After #10827 that drops type parameters in Value, `CidContainer`'s
`ensureNoCid` and `assertNoCid` loose their interest.
To check a value does not contain Contract Ids one should use
`foreachCid` or `cids` methods instead.

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-04 14:49:19 +01:00
Oliver Seeliger
30bfee9822
Upgrade canton for conformance tests (#11550)
This also eases the work on:
- error codes - fyi @tudor-da
- open sourcing

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-04 13:10:48 +00:00
Stefano Baghino
b84a9142d1
Extractor: replace sandbox-classic-based fixtures with sandbox-based ones (#11489)
* Replace `sandbox-classic`-based fixtures with `sandbox`-based ones

changelog_begin
changelog_end

* Replace Scenarios with Script for BasicPrimitiveTypes

* Adapt ListsSpec

* Adapt TextMapsSpec

* Adapt OptionalSpec

* Adapt more tests

* More tests passing

- removed workflow_id testing (Daml Script doesn't set one)
- reduced failure threshold for VeryLargeArchiveSpec
- solved issu in TransactionSingleTableSpec (wrong parties were used)

* Port EnumMod and make EnumSpec pass

* Port GenMapMod and make GenMapSpec pass

* Shaking things around to make Windows happy -- maybe
2021-11-04 13:08:51 +00:00
Kamil Bozek
d86fe92950
ledger-api-bench-tool - flexible stream filters [DPP-667] (#11458)
* ACS filters

* Moved filters to a separate object

* Filters for transactions and transaction trees streams

* Updated help

CHANGELOG_BEGIN
- [Integration Kit] - ledger-api-bench-tool - added flexible party and template filters for transactions, transaction trees and active contract streams
CHANGELOG_END

* Changed filter separators to + and @
2021-11-04 12:08:38 +01:00
fabiotudone-da
214c7ab445
Self-service errors: generate groups documentation [KVL-1186] (#11526)
* Add groups information to JSON

CHANGELOG_BEGIN
CHANGELOG_END

* Code cleanup

* Fix docs generation

* Fix KVErrors group explanation string

* Render error group explanations in generated docs

* Fix ErrorCodeDocumentationGeneratorSpec
2021-11-04 12:07:29 +01:00
Moritz Kiefer
60dc2869e7
Avoid unnecessarily decoding DALFs during dependency installation (#11541)
We only need the package id which only requires decoding the header
which is exactly what decodearchivePackageId exists for.

On a large project this seems like a 10-20s improvement which is
pretty significant.

changelog_begin
changelog_end
2021-11-04 11:36:39 +01:00
Moritz Kiefer
afd79b6729
Improve logging for dependency installation (#11540)
Also drops the weird Base module and the always empty tags (yet
another reason why I want to burn our own logging lib with fire).

changelog_begin
changelog_end
2021-11-04 11:36:03 +01:00
Gary Verhaegen
fdde5353f4
fix hoogle by pinning nixpkgs (#11548)
Hoogle has been down for at least 24h accoridng to user reports. What
seems to be happening is that our nixpkgs pinning is not taking effect,
and the nixpkgs version of Hoogle already includes the patch we are
trying to add. This confuses nix, which fails, and thus the boot
sequence is broken.

I've applied the minimal possible patch here (i.e. enforce the pin),
which gets things running again. I've already deployed this change.

We may want to look at bumping the nixpkgs snapshot.

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-04 10:22:30 +00:00
tudor-da
5e93df6b52
Full conformance test with self-service error codes on ledger-on-memory (#11524)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-04 09:36:06 +01:00
azure-pipelines[bot]
efaf20b042
update NOTICES file (#11546)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-11-04 08:40:08 +01:00
akshayshirahatti-da
c7bb94be65
Modify contract key attribute age for createAndExercise to avoid contract key violation on slow running tests (#11530)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-04 05:40:13 +00:00
Marton Nagy
541d07c3a9
Add string-interning data access [DPP-707] (#11484)
* Add new table, and party_id to party_entries
* Add write side basics (DbDto, Schema)
* Add read side basic (StorageBackendStringInterning)
* Add StorageBackend unit test
* Support for all DB backends

changelog_begin
changelog_end
2021-11-04 02:07:32 +00:00
Marton Nagy
6c9416492d
Switching to composition at StorageBackend [DPP-704] (#11520)
* Switch to StorageBackendFactory
* Fixing tests

changelog_begin
changelog_end
2021-11-03 22:53:05 +00:00
Marton Nagy
d006ad0e8b
Add string-interning view primitives [DPP-702] (#11475)
changelog_begin
changelog_end
2021-11-03 20:27:10 +00:00
nicu-da
a917520851
Save the submission time in the command deduplication value [KVL-1173] (#11509)
CHANGELOG_BEGIN

CHANGELOG_END
2021-11-03 18:30:03 +00:00
Marton Nagy
669186f118
Add priority-queue based worker Source [DPP-717] (#11500)
* Add ACSReader trait, and primitives
* Add pullWorkerSource implementation + unit tests

changelog_begin
changelog_end
2021-11-03 17:59:57 +00:00