Commit Graph

1721 Commits

Author SHA1 Message Date
Marton Nagy
5fa49c3312
Disables flaky test suite of InstrumentedSource.bufferedSource (#10321)
CHANGELOG_BEGIN
CHANGELOG_END
2021-07-20 06:48:49 +00:00
Samir Talwar
9a01065f29
participant-state: Remove the Daml-LF transaction and value aliases. [KVL-1002] (#10317)
* participant-state: Remove the Daml-LF transaction aliases.

They add nothing.

CHANGELOG_BEGIN
CHANGELOG_END

* participant-state: Move the `ContractInst` alias to where it's used.

Either inlining it or extracting out a similar alias elsewhere.

This further removes dependencies on _participant-state_.

* Fix a couple of typos.

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2021-07-19 18:32:28 +00:00
Remy
159728d716
LF: use Either by default in all archive reader API (#10295)
This is a follow up of #10277.
This is part of #9974.

CHANGELOG_BEGIN
CHANGELOG_END
2021-07-19 19:48:37 +02:00
Victor Peter Rouven Müller
80b65f6def
[DOCS] Add documentation for the JSON API metrics (#10312)
* [DOCS] Add documentation for the JSON API metrics

changelog_begin

- [JSON-API] You can now find a section `Metrics` in the http-json api documentation explaining how to enable metrics and which are available

changelog_end

* Fix rst build warnings

* Update docs/source/json-api/metrics.rst

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Adapt metrics doc to state that it IS an exhaustive list and remove wrong copy pasta text & add info about prometheus

* Update the legal values for the metrics reporter cli option

* shorten the description, the change prior was unnecessary ._.

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2021-07-19 17:32:23 +02:00
nickchapman-da
0e09f3ba08
Remove optLocation field from transaction nodes. (#10301)
* Remove Optional[Location] field from transaction nodes.

xoptLocation

working but very messy!

cleanup PartialTransaction code

pass locationInfo in CompleteTransaction

dont pass locationInfo in SumittedTransaction

pass optLocation to CheckAuthorization

temp reinstate xoptLocation in ExerciseContextInfo

cleanup Conversions.scala

remove Optional[Location] field from transaction nodes.

CHANGELOG_BEGIN
CHANGELOG_END

* fix bad merge

* fix build after merge

* rename

* doc comment, and question
2021-07-19 13:03:30 +00:00
fabiotudone-da
e62852fbfd
Make ledger API test tool's 'max-connection-attempts' into an option (#10311)
CHANGELOG_BEGIN
CHANGELOG_END
2021-07-19 14:43:44 +02:00
Samir Talwar
bd12170af2
participant-state: Remove the ParticipantId, PackageId, and Party aliases. [KVL-1002] (#10308)
* participant-state: Remove the `ParticipantId` alias.

This alias adds nothing. By using `Ref.ParticipantId` directly, many
packages can remove their dependency on the _participant-state_ package.

CHANGELOG_BEGIN
CHANGELOG_END

* participant-state: Remove the `PackageId` and `Party` aliases.

They don't add anything. Let's just use `Ref`.

* kvutils: Restore missing compat imports.
2021-07-19 12:31:25 +00:00
Samir Talwar
bd35f80efa
ledger-configuration: Correct the protobuf package. [KVL-1002] (#10302)
* ledger-configuration: Correct the protobuf package.

CHANGELOG_BEGIN
- [Integration Kit] The *ledger_configuration.proto* Protobuf definition
  has been repackaged under ``com.daml.ledger.configuration``, and the
  Java and C# packages have been renamed accordingly. If you are using
  this Protobuf definition, you will need to update your imports. The
  Maven artifact name has not changed.
CHANGELOG_END

* ledger-configuration: Resolve a naming collision in Scala 2.12.
2021-07-19 11:00:11 +00:00
fabiotudone-da
d69328d281
Ledger API Test Tool: add a --max-connection-attempts command line option [KVL-977] (#10297)
* Add a `--max-connection-attempts` command line option to the Ledger API Test Tool

BEGIN_CHANGELOG

- [Integration Kit] Add a `--max-connection-attempts` command line option to the Ledger API Test Tool

END_CHANGELOG

* Remove `.unbounded()` from the cmdline option

* fmt

* Restore `.unbounded()` in `[endpoints...]`

BEGIN_CHANGELOG
- [Integration Kit] Add a `--max-connection-attempts` command line option to the Ledger API Test Tool
END_CHANGELOG

* Remove stale comment

CHANGELOG_BEGIN
- [Integration Kit] Add a `--max-connection-attempts` command line option to the Ledger API Test Tool
CHANGELOG_END

* Update ledger/ledger-api-test-tool/src/main/scala/com/daml/ledger/api/testtool/infrastructure/LedgerTestCasesRunner.scala

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>

* fmt

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2021-07-19 09:52:32 +00:00
Samir Talwar
1ee53c0736
participant-state-index: Use Ref directly. (#10299)
This avoids the dependency on _participant-state_.

CHANGELOG_BEGIN
CHANGELOG_END
2021-07-15 17:21:03 +00:00
Miklos
0eba812109
Remove trace_context field from Ledger API [KVL-1021] (#10256)
CHANGELOG_BEGIN
* [Integration Kit] Removed trace_context field from Ledger API and its bindings as we now have trace context propagation support via gRPC metadata. If you are constructing or consuming Ledger API requests or responses directly, you may need to update your code.
CHANGELOG_END
2021-07-15 19:06:25 +02:00
Samir Talwar
88886beb8d
ledger-configuration: Add tests for the Configuration decoding methods. [KVL-1002] (#10293)
* ledger-configuration: Add tests for Configuration.

CHANGELOG_BEGIN
CHANGELOG_END

* ledger-configuration: Reject a negative maximum deduplication time.

* ledger-configuration: Remove duplication in Configuration.

* ledger-configuration: Make some Configuration methods private.

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>

* ledger-configuration: Remove some extra braces.

* ledger-configuration: Use tables to simplify rejection tests.

* ledger-configuration: Add a test for an unknown version.

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2021-07-15 16:15:08 +00:00
Samir Talwar
cb29f34d4b
ledger-offset: Move Offset to a new package. [KVL-1002] (#10296)
* ledger-offset: Move `Offset` to a new package.

CHANGELOG_BEGIN
- [Integration Kit] The ``Offset`` type has been moved to a new Maven
  package, ``ledger-offset``, from the ``participant-state`` package.
  The Java package has been renamed to ``com.daml.ledger.offset``. If
  you are using this type, you will need to update your dependencies and
  imports.
CHANGELOG_END

* Avoid rewrapping offsets for no reason.

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>

* participant-integration-api: Sort some imports.

* participant-integration-api: Fix dependencies for the Oracle tests.

I didn't add `ledger-offset`.

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2021-07-15 17:53:03 +02:00
Samir Talwar
70b12f04d0
ledger-on-sql: Remove conformance tests for H2 and SQLite files. (#10291)
We don't use these in any kind of production setting, and it reduces the
CI overhead a lot.

CHANGELOG_BEGIN
CHANGELOG_END
2021-07-15 13:31:25 +00:00
Samir Talwar
a9a0b70b85
ledger-configuration: Extract the configuration from participant-state. [KVL-1002] (#10279)
* ledger-configuration: Extract configuration from participant-state.

The configuration is often used without the state, and doesn't need to
be versioned in the same way.

CHANGELOG_BEGIN
- [Integration Kit] The ledger configuration classes, ``Configuration``,
  ``LedgerInitialConditions``, and ``TimeModel``, have been moved from
  *participant-state* to a separate package named
  *ledger-configuration*, in the Java package
  ``com.daml.ledger.configuration``. You will need to update your
  dependencies and imports.
CHANGELOG_END

* participant-state: Remove the `LedgerId` aliases.

* ledger-configuration: Rename `TimeModel` to `LedgerTimeModel`.

This avoids confusion with the protobuf-generated `TimeModel` classes.

CHANGELOG_BEGIN
- [Integration Kit] ``TimeModel`` has been renamed to
  ``LedgerTimeModel``. If you are using the ledger configuration classes
  directly, you may need to update your code.
CHANGELOG_END

* ledger-configuration: Remove colons in LedgerInitialConditions' docs.

* kvutils: Restore a missing compat import.

* participant-integration-api: Add ledger-configuration to Oracle tests.

* sandbox-common: Fix `--max-ledger-time-skew` docs.

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2021-07-15 13:03:19 +00:00
Robert Autenrieth
d565cae913
DPP-457 Add UpdateToDbdto tests (#10116)
* Extract interface for LfValueTranslation

* Add unit tests for UpdateToDbDto

changelog_begin
changelog_end

* Move constants to companion object

* Address review comments

* Address review comments
2021-07-15 12:00:46 +02:00
tudor-da
ba8e829c8d
[In-memory fan-out] Oracle support (#10263)
* [In-memory fan-out] Oracle support

CHANGELOG_BEGIN
CHANGELOG_END

* Correct so Oracle backend works with fan out enabled - add implicit conversion and correct view defaulting for BLOB fields
Add missing tests to oracle append only spec as it lagged behind postgres and h2
Regen sql sha for oracle

CHANGELOG_BEGIN
CHANGELOG_END

* avoid global import of implicit that is not needed

* reduce duplication on conformance test targets

* update sha for oracle append only flyway scripts

Co-authored-by: Brian Healey <brian.healey@digitalasset.com>
2021-07-14 21:10:12 +00:00
Remy
a8f190214c
LF: change type from Try to Either in archive module (#10277)
* LF: change type from Try to Either in archive module

This is the first part of restructuring errors in archive module.
This is part of #9974.

CHANGELOG_BEGIN
CHANGELOG_END

* Apply suggestions from code review

Co-authored-by: Stephen Compall <stephen.compall@daml.com>

* remove type alias

* apply stephen suggestion

* fix after rebase

* fix test

Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2021-07-14 19:24:31 +00:00
Marton Nagy
191d3b82ce
Add unit test to SequentialWriteDao [DPP-456] (#10265)
* also fix an expectation in the ParallelIndexerFactorySpec

changelog_begin
changelog_end
2021-07-14 14:25:06 +00:00
mziolekda
2271f93b69
Fix crash on duplicate submissionId (#10242)
* CHANGELOG_BEGIN
Fix crash in ConfigManagement- and PackageManagement- services on duplicate submissionsIds from different participants.
Extend ledger-api-test-tool to cover the duplicate submissionId cases.
CHANGELOG_END

* Addressing review comments and fixing failing tests

* format

* address more review comments

* adopt after rebase to main
2021-07-14 14:12:31 +00:00
Remy
906184ce1a
LF: Simplify UniversalDarReader (#10271)
CHANGELOG_BEGIN
CHANGELOG_END
2021-07-14 15:38:22 +02:00
Remy
caf85a2270
LF: rationalize archive Parser/Reader/Decoder (#10239)
CHANGELOG_BEGIN
CHANGELOG_END
2021-07-14 11:05:17 +02:00
Miklos
21a9e62f27
Add disclaimer to participant state v2 API [KVL-998] (#10254)
* Added disclaimer to Read/WriteService.

* Removed NeedCompletionOffsetForSubmissionId rejection reason.

* CHANGELOG_BEGIN
CHANGELOG_END

* Added disclaimers to adaptors regarding partial implementation.
2021-07-14 08:42:49 +00:00
Samir Talwar
6c63f96ba9
participant-integration-api: Move SeedService here. (#10261)
I don't see a reason why it's part of the participant state API, and
it definitely doesn't need to change between v1 and v2.

CHANGELOG_BEGIN
- [Integration Kit] The class ``SeedService`` has been moved from the
  *participant-state* Maven package to the *participant-integration-api*
  Maven package, under the Java package name
  ``com.daml.platform.apiserver`` to reflect its usage by the API
  server, not the participant state API. If you use this class directly,
  you will need to change your imports.
CHANGELOG_END
2021-07-14 08:35:29 +00:00
Hubert Slojewski
58c0b4694a
kvutils: Simplify usage of Rejections [KVL-1015] (#10258)
CHANGELOG_BEGIN
CHANGELOG_END
2021-07-14 09:15:39 +02:00
Hubert Slojewski
41aba1ca39
kvutils: Move unrelated test cases to a proper test class [KVL-1015] (#10257)
CHANGELOG_BEGIN
CHANGELOG_END
2021-07-14 09:15:25 +02:00
Samir Talwar
1f35db17c7
ledger-on-sql: Use random log entry ID allocation so we don't depend on SeedService. [KVL-1002] (#10255)
* ledger-on-sql: Abstract out the log entry ID allocator.

This is so we don't need to depend on `SeedService`.

CHANGELOG_BEGIN
CHANGELOG_END

* kvutils: Remove `allocateRandomLogEntryId` from production code.

* kvutils: Remove `SubmissionValidator.create`, as it's only for tests.

And remove default parameters for
`SubmissionValidator.createForTimeMode`.

* ledger-on-sql: Use the random log entry ID allocator.

In Sandbox, we still use the seed service, but this is not necessary
for ledger-on-sql tests.

* kvutils: Make `LogEntryIdAllocator.random` an `object`.

* ledger-on-sql: Move `RandomLogEntryIdAllocator` here.

It's not used anywhere else.

* ledger-on-sql: Add some documentation about the application code.

Just to make it clear it's not used in production.
2021-07-13 17:10:30 +00:00
tudor-da
de7a08fa7b
[In-memory fan-out] Handle null submitters in getTransactionLogUpdates (#10248)
* [In-memory fan-out] Handle `null` submitters in getTransactionLogUpdates

CHANGELOG_BEGIN
CHANGELOG_END

* Address review comments
2021-07-13 12:11:27 +00:00
Miklos
584169a2cc
Participant state v1-to-v2 adaptors [KVL-1002] (#10246)
CHANGELOG_BEGIN
CHANGELOG_END
2021-07-13 13:35:35 +02:00
Kamil Bożek
edaf541063
Adapt ledger-api-bench-tool to work with LR [DPP-424] (#10171)
* Print ledger-api-bench-tool final report when a stream in ended by shutting down API server
CHANGELOG_BEGIN
CHANGELOG_END

* Log ledger-api-bench-tool failure with specific header to make it compatible with the lr-sidecar

* Print final report on graceful shutdown even if streams are not complete

* trigger build
2021-07-13 09:58:51 +02:00
Marton Nagy
05e5218d06
Consolidating EventStorageBackend (#10224)
* Switching to composed anorm queries
* Switching to data-driven prepared statements where possible

changelog_begin
changelog_end
2021-07-12 19:55:52 +02:00
tudor-da
3fd3abfb26
Instrument buffer after contract state events stream (#10209)
CHANGELOG_BEGIN
CHANGELOG_END
2021-07-12 19:37:32 +02:00
Miklos
643a2de006
v2 participant state API draft [KVL-998] (#10210)
CHANGELOG_BEGIN
CHANGELOG_END
2021-07-12 18:11:52 +02:00
Marton Nagy
5abcd0482e
Adds Unit tests for parallel-ingestion [DPP-455] (#10238)
changelog_begin
changelog_end
2021-07-12 14:46:12 +00:00
Hubert Slojewski
3cdedcf885
kvutils: Extract validators from TransactionCommitter [KVL-1015] (#10235)
CHANGELOG_BEGIN
CHANGELOG_END
2021-07-12 15:04:18 +02:00
Remy
4eba4b00e8
Ledger: decouple BD value serialization library from archive library. (#10218)
The DB serialization library should no use damlLfCodedInputStream from
(archive decoding) to decode values.

CHANGELOG_BEGIN
CHANGELOG_END
2021-07-09 15:42:13 +02:00
Marton Nagy
48393444df
TODO Cleanup: QueryNonPruned [DPP-459] (#10225)
changelog_begin
changelog_end
2021-07-09 09:20:56 +00:00
tudor-da
0bea5e3ef7
Allow retriable lookupMaximumLedgerTime on contention (#10211)
CHANGELOG_BEGIN
CHANGELOG_END
2021-07-09 09:30:05 +02:00
Remy
f19f5b0821
LF: Simplify DarReader (#10217)
This PR simplifies a bit DarReader and UniversalDarReader

This is a follow up of #10208.

CHANGELOG_BEGIN
CHANGELOG_END
2021-07-08 17:03:45 +02:00
Remy
ebb76dca4c
LF: reorganize errors in com.daml.lf.archive (#10213)
* LF: reorganize errors in com.daml.lf.archive

part of #9974

CHANGELOG_BEGIN
CHANGELOG_END

* fix
2021-07-08 13:03:42 +00:00
Remy
41b8448b17
LF: Simplify archive reader. (#10208)
* LF: Simplify archive reader.

- decouple Reader and Decoder
- introduce case class to handle hash, proto payload, and version

CHANGELOG_BEGIN
CHANGELOG_END

* Address Moritz' review

* cosmetic
2021-07-07 19:56:40 +00:00
Moritz Kiefer
4fe626a055
Drop logging on CompletionSource (#10190)
The debug logs don’t seem all that useful and we don’t have them for
other sources, e.g., TransactionSource. They also come with error logs
for failed sources which again is inconsistent with other sources,
noisy and confusing.

changelog_begin
changelog_end
2021-07-06 16:12:05 +00:00
Gary Verhaegen
8750c0c47c
reduce noise in Scala bindings (#10187)
It's unlikely most clients will want all the details of the stack trace.

CHANGELOG_BEGIN
CHANGELOG_END
2021-07-06 15:02:40 +02:00
tudor-da
7b93923c16
[In-memory fan-out] Performance optimizations [DPP-470] (#10127)
* [In-memory fan-out] Fix execution contexts in ReadOnlySqlLedgerWithMutableCache
* servicesExecutionContext is used in all non-resource components
* Remove widespread use of resourceContext

CHANGELOG_BEGIN
CHANGELOG_END

* [In-memory fan-out] TransactionLogUpdatesConversions optimization
* Lazily convert to transaction trees in TransactionLogUpdatesConversions.ToTransactionTree

* Addressed review comments
2021-07-06 09:11:46 +02:00
Samir Talwar
e7e8a5705b
daml-lf/data: Optionally truncate party names in structured logs. [KVL-996] (#10163)
* daml-lf/data: Truncate party names in log output, on request.

The party name can grow quite long, so we offer ledger implementors the
opportunity to truncate it in structured log output.

Unfortunately, because we use Logback through the global
`LoggerFactory`, there is no place to inject logging configuration. This
means we also need to use global, mutable state to configure logging
output. I have added a `LoggingConfiguration` class+object in Daml-LF
Data, which may not be the best place, but I can't think of a better
one right now. I suggest we leave it there until it has reason to grow,
at which point we may want to move it.

CHANGELOG_BEGIN
CHANGELOG_END

* logging-entries: Make `ToLoggingValue` mixin-able.

* participant-integration-api: Truncate parties in filters when logging.

* participant-integration-api: Cast to `Party` for logging.

Invalid input should not break the request at this point. No assertions.

* daml-lf/data: Move `Party to LoggingValue` to a new package.

This avoids the transitive dependency issue most of the time.

* daml-lf-data: Move the `Identifier` logging to another package.

Again, reduces the need for transitively depending on _logging-entries_.
2021-07-01 16:50:49 +00:00
Samir Talwar
5493e4e2b1
language-support/scala/codegen: Set the logging level to WARN. (#10165)
I don't find the output from Bazel very helpful most of the time. It
does, however, produce a lot of noise which I have to filter through
when something else goes wrong.

CHANGELOG_BEGIN
CHANGELOG_END
2021-07-01 16:49:46 +00:00
mziolekda
40bec44bf9
Simple draw of stopping indexing upon unhealthy DbDispatcher (#10154)
changelog_begin
changelog_end

Co-authored-by: Marton Nagy <marton.nagy@digitalasset.com>
2021-07-01 10:05:18 +02:00
Samir Talwar
eda1245311
daml-lf/data: Add a conversion from Ref.Identifier to LoggingValue. (#10157)
I am getting quite tired of (Bazel + Scala)'s attitude to transitive
dependencies.

CHANGELOG_BEGIN
CHANGELOG_END
2021-06-30 20:05:06 +00:00
Samir Talwar
888f02aa4d
ledger-api-common: There's only one way to order absolute offsets. (#10153)
Let's not mess around with injectable orderings.

This has the pleasant side effect of removing dependencies.

CHANGELOG_BEGIN
CHANGELOG_END
2021-06-30 14:17:59 +00:00
Kamil Bożek
9c7b506922
Increase timeout for TransactionServiceIT:TXFlatTransactionsVisibility (#10150)
CHANGELOG_BEGIN
CHANGELOG_END
2021-06-30 15:00:14 +02:00