When machine disks are full, we can't clean the Bazel cache if it
happens to not be a mount point. I don't quite understand yet why it's
not a mount point, but maybe I'll be able to investigate more if we catch
the issue early, rather than waiting for the disk to be full and the
clean-up to fail.
CHANGELOG_BEGIN
CHANGELOG_END
* Remove Optional[Location] field from transaction nodes.
xoptLocation
working but very messy!
cleanup PartialTransaction code
pass locationInfo in CompleteTransaction
dont pass locationInfo in SumittedTransaction
pass optLocation to CheckAuthorization
temp reinstate xoptLocation in ExerciseContextInfo
cleanup Conversions.scala
remove Optional[Location] field from transaction nodes.
CHANGELOG_BEGIN
CHANGELOG_END
* fix bad merge
* fix build after merge
* rename
* doc comment, and question
* participant-state: Remove the `ParticipantId` alias.
This alias adds nothing. By using `Ref.ParticipantId` directly, many
packages can remove their dependency on the _participant-state_ package.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-state: Remove the `PackageId` and `Party` aliases.
They don't add anything. Let's just use `Ref`.
* kvutils: Restore missing compat imports.
This PR extends the test to test a full matrix of different party &
template id numbers. Summarizing the results, as expected we index by
party but not by template id:
Benchmark (batchSize) (extraParties) (extraTemplates) Mode Cnt Score Error Units
QueryBenchmark.run 10000 1 0 avgt 5 0.255 ± 0.064 s/op
QueryBenchmark.run 10000 10 0 avgt 5 0.304 ± 0.245 s/op
QueryBenchmark.run 10000 100 0 avgt 5 0.296 ± 0.064 s/op
Benchmark (batchSize) (extraParties) (extraTemplates) Mode Cnt Score Error Units
QueryBenchmark.run 10000 0 1 avgt 5 0.277 ± 0.037 s/op
QueryBenchmark.run 10000 0 10 avgt 5 0.479 ± 0.301 s/op
QueryBenchmark.run 10000 0 100 avgt 5 2.131 ± 0.497 s/op
We know how to fix that so I’ll get on that.
changelog_begin
changelog_end
* ledger-configuration: Correct the protobuf package.
CHANGELOG_BEGIN
- [Integration Kit] The *ledger_configuration.proto* Protobuf definition
has been repackaged under ``com.daml.ledger.configuration``, and the
Java and C# packages have been renamed accordingly. If you are using
this Protobuf definition, you will need to update your imports. The
Maven artifact name has not changed.
CHANGELOG_END
* ledger-configuration: Resolve a naming collision in Scala 2.12.
* Add a `--max-connection-attempts` command line option to the Ledger API Test Tool
BEGIN_CHANGELOG
- [Integration Kit] Add a `--max-connection-attempts` command line option to the Ledger API Test Tool
END_CHANGELOG
* Remove `.unbounded()` from the cmdline option
* fmt
* Restore `.unbounded()` in `[endpoints...]`
BEGIN_CHANGELOG
- [Integration Kit] Add a `--max-connection-attempts` command line option to the Ledger API Test Tool
END_CHANGELOG
* Remove stale comment
CHANGELOG_BEGIN
- [Integration Kit] Add a `--max-connection-attempts` command line option to the Ledger API Test Tool
CHANGELOG_END
* Update ledger/ledger-api-test-tool/src/main/scala/com/daml/ledger/api/testtool/infrastructure/LedgerTestCasesRunner.scala
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* fmt
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
The cluster died yesterday. As part of recovery, I connected to the
machines and made manual changes. To ensure that we get back to a known,
documented setup I then proceeded to do a full blue -> green migration,
having not tainted any of the green machines with manual interventions.
CHANGELOG_BEGIN
CHANGELOG_END
CHANGELOG_BEGIN
* [Integration Kit] Removed trace_context field from Ledger API and its bindings as we now have trace context propagation support via gRPC metadata. If you are constructing or consuming Ledger API requests or responses directly, you may need to update your code.
CHANGELOG_END
* ledger-configuration: Add tests for Configuration.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-configuration: Reject a negative maximum deduplication time.
* ledger-configuration: Remove duplication in Configuration.
* ledger-configuration: Make some Configuration methods private.
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* ledger-configuration: Remove some extra braces.
* ledger-configuration: Use tables to simplify rejection tests.
* ledger-configuration: Add a test for an unknown version.
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* ledger-offset: Move `Offset` to a new package.
CHANGELOG_BEGIN
- [Integration Kit] The ``Offset`` type has been moved to a new Maven
package, ``ledger-offset``, from the ``participant-state`` package.
The Java package has been renamed to ``com.daml.ledger.offset``. If
you are using this type, you will need to update your dependencies and
imports.
CHANGELOG_END
* Avoid rewrapping offsets for no reason.
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* participant-integration-api: Sort some imports.
* participant-integration-api: Fix dependencies for the Oracle tests.
I didn't add `ledger-offset`.
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* ledger-configuration: Extract configuration from participant-state.
The configuration is often used without the state, and doesn't need to
be versioned in the same way.
CHANGELOG_BEGIN
- [Integration Kit] The ledger configuration classes, ``Configuration``,
``LedgerInitialConditions``, and ``TimeModel``, have been moved from
*participant-state* to a separate package named
*ledger-configuration*, in the Java package
``com.daml.ledger.configuration``. You will need to update your
dependencies and imports.
CHANGELOG_END
* participant-state: Remove the `LedgerId` aliases.
* ledger-configuration: Rename `TimeModel` to `LedgerTimeModel`.
This avoids confusion with the protobuf-generated `TimeModel` classes.
CHANGELOG_BEGIN
- [Integration Kit] ``TimeModel`` has been renamed to
``LedgerTimeModel``. If you are using the ledger configuration classes
directly, you may need to update your code.
CHANGELOG_END
* ledger-configuration: Remove colons in LedgerInitialConditions' docs.
* kvutils: Restore a missing compat import.
* participant-integration-api: Add ledger-configuration to Oracle tests.
* sandbox-common: Fix `--max-ledger-time-skew` docs.
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
I haven’t found any conclusive information as to why ON COMMIT doesn’t
work incrementally but
https://docs.oracle.com/en/database/oracle/oracle-database/19/adjsn/json-query-rewrite-use-materialized-view-json_table.html#GUID-8B0922ED-C0D1-45BD-9588-B7719BE4ECF0
recommends that for rewriting (which isn’t what we do here but both
involve a materialized view on json_table).
Benchmarks:
before:
InsertBenchmark.run 1000 1 1000 avgt 5 0.327 ± 0.040 s/op
InsertBenchmark.run 1000 3 1000 avgt 5 0.656 ± 0.043 s/op
InsertBenchmark.run 1000 5 1000 avgt 5 1.034 ± 0.051 s/op
InsertBenchmark.run 1000 7 1000 avgt 5 1.416 ± 0.106 s/op
InsertBenchmark.run 1000 9 1000 avgt 5 1.734 ± 0.143 s/op
QueryBenchmark.run 1000 10 N/A avgt 5 0.071 ± 0.016 s/op
After:
Benchmark (batchSize) (batches) (numContracts) Mode Cnt Score Error Units
InsertBenchmark.run 1000 1 1000 avgt 5 0.217 ± 0.034 s/op
InsertBenchmark.run 1000 3 1000 avgt 5 0.232 ± 0.027 s/op
InsertBenchmark.run 1000 5 1000 avgt 5 0.226 ± 0.051 s/op
InsertBenchmark.run 1000 7 1000 avgt 5 0.225 ± 0.048 s/op
InsertBenchmark.run 1000 9 1000 avgt 5 0.232 ± 0.021 s/op
QueryBenchmark.run 1000 10 N/A avgt 5 0.080 ± 0.014 s/op
The difference in query times is just noise and changes across runs.
So we get the expected behavior of inserts being independent of the
total ACS size now. We could still explore if we gain something by
avoiding the materialized view to reduce constant factors but that’s
much less of an issue.
fixes#10243
changelog_begin
changelog_end
@akshayshirahatti-da is taking care of 1.15.0-snapshot.20210713.7343.0.1f35db17 (#10267), so they get pushed back to the end of the line.
Please do not merge this before #10267.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* [In-memory fan-out] Oracle support
CHANGELOG_BEGIN
CHANGELOG_END
* Correct so Oracle backend works with fan out enabled - add implicit conversion and correct view defaulting for BLOB fields
Add missing tests to oracle append only spec as it lagged behind postgres and h2
Regen sql sha for oracle
CHANGELOG_BEGIN
CHANGELOG_END
* avoid global import of implicit that is not needed
* reduce duplication on conformance test targets
* update sha for oracle append only flyway scripts
Co-authored-by: Brian Healey <brian.healey@digitalasset.com>
* LF: change type from Try to Either in archive module
This is the first part of restructuring errors in archive module.
This is part of #9974.
CHANGELOG_BEGIN
CHANGELOG_END
* Apply suggestions from code review
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* remove type alias
* apply stephen suggestion
* fix after rebase
* fix test
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Fix status check in collect_build_data
follow up to #10270 which caused the linux & macos builds to go
through but then screwed us over in collect_build_data. I hate CI.
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
* CHANGELOG_BEGIN
Fix crash in ConfigManagement- and PackageManagement- services on duplicate submissionsIds from different participants.
Extend ledger-api-test-tool to cover the duplicate submissionId cases.
CHANGELOG_END
* Addressing review comments and fixing failing tests
* format
* address more review comments
* adopt after rebase to main
* [JSON-API] Refactor Endpoints.scala to use path directives etc.
changelog_begin
changelog_end
* Don't warn that the ev param in toRoute is not used
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/Endpoints.scala
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/Endpoints.scala
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/Endpoints.scala
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Remove weird stuff to have nice stuff with the toRoute function
* Rename the toRoute function & remove comments as things are now clarified
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* release 1.15.0-snapshot.20210713.7343.0.1f35db17
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@akshayshirahatti-da is in charge of this release.
Commit log:
```
1f35db17c7 ledger-on-sql: Use random log entry ID allocation so we don't depend on `SeedService`. [KVL-1002] (#10255)
b8e2198873 Separate traces from warnings in engine. (#10253)
0a7f2b1a1b [JSON-API] Small refactoring to start using akka http routes internally (#10252)
6e8ec1d618 LF: Drop old depreated code (#10251)
de7a08fa7b [In-memory fan-out] Handle `null` submitters in getTransactionLogUpdates (#10248)
584169a2cc Participant state v1-to-v2 adaptors [KVL-1002] (#10246)
e4d7cd764a LF: Deprecate com.daml.lf.data.TryOps.Bracket (#10249)
b59f36591d [JSON-API] Correctly extract the request source URL/IP (#10244)
edaf541063 Adapt `ledger-api-bench-tool` to work with LR [DPP-424] (#10171)
6fe6ae075d LF: Make DarReader ZipEntries immulatble (#10243)
05e5218d06 Consolidating EventStorageBackend (#10224)
3fd3abfb26 Instrument buffer after contract state events stream (#10209)
643a2de006 v2 participant state API draft [KVL-998] (#10210)
5abcd0482e Adds Unit tests for parallel-ingestion [DPP-455] (#10238)
3cdedcf885 kvutils: Extract validators from TransactionCommitter [KVL-1015] (#10235)
00d622f268 Make @UNTIL-LF exclusive, add @UNTIL-LF-FEATURE. (#10240)
2bcbd4e177 es: switch to persistent nodes (#10236)
7f4bc2a472 update compat versions for 1.14.2 (#10233)
4eba4b00e8 Ledger: decouple BD value serialization library from archive library. (#10218)
729afa8c7b Write a few lines on how to run tests against an Oracle database (#10230)
2b67ebb5d4 tf: refactor appr var (#10232)
202b7f7ae7 add akshay to appr team (#10229)
2c5410c229 Release SDK 1.14.2 (#10226)
89d87ad17c update compat versions for 1.14.2-snapshot.20210708.7135.0.aa297840 (#10223)
48393444df TODO Cleanup: QueryNonPruned [DPP-459] (#10225)
0bea5e3ef7 Allow retriable lookupMaximumLedgerTime on contention (#10211)
ca2a9e9bbb Release 1.14.2 snapshot (#10222)
999577a1a7 tweak ES cluster (#10219)
6680d368a1 Warn on DA.Exception import (#10201)
f19f5b0821 LF: Simplify DarReader (#10217)
8f1a8f2390 Release DAML SDK 1.14.1 (#10204)
ebb76dca4c LF: reorganize errors in com.daml.lf.archive (#10213)
69646451f2 Improvements for validation tests (#10214)
5f124c3b64 Avoid collision in execution log postfix (#10205)
9e27ae0d85 Reshuffle cocreature in daml-lf codeowners (#10215)
41b8448b17 LF: Simplify archive reader. (#10208)
38734f02d7 es-feed: ignore invalid files (#10207)
6586dde11e Move exercise context computation to the client side (#10199)
c92c67832f Mark Daml profiler as stable (#10203)
cb1f4ec773 ci/windows: disable spool (#10200)
2326d425bc Publish execution logs on unix platforms (#10194)
42693452ce Drop cleanup of node_modules (#10195)
c8faed8a17 Deduplicate Java codegen running instructions (#10185)
e4c8c390f2 update compat versions for 1.14.1-snapshot.20210706.7133.0.8ec16e94 (#10196)
0d881f5e2b Improvements to the documentation with regards to offsets (#10180)
1d5ba4fa42 feed elasticsearch cluster (#10193)
e2bdca6be9 Use PartialFunction for more concise code. (#10191)
fe98c48b65 release 1.14.1-snapshot.20210706.7133.0.8ec16e94 (#10186)
4fe626a055 Drop logging on CompletionSource (#10190)
582aa5f08c Fix a typo in an exceptions example. (#10188)
8750c0c47c reduce noise in Scala bindings (#10187)
98b5ffee01 Add divulgence warning and test in script service. (#10179)
8578e56aa7 Tests for transaction validation (#10167)
05a72a3a15 update compat versions for 1.15.0-snapshot.20210705.7286.0.62aabcc4 (#10184)
7b93923c16 [In-memory fan-out] Performance optimizations [DPP-470] (#10127)
6c49619565 Document how to run the Java codegen from the Daml assistant (#10181)
61aa774988 Release SDK 1.15 RC (#10182)
```
Changelog:
```
- [JSON-API] If the service is put behind a proxy filling either of these headers X-Forwarded-For & X-Real-Ip then these will now be respected for logging the request source ip/url
- [Daml Compiler] Importing DA.Exception on LF < 1.14 now produces a
warning that exceptions require Daml-LF >= 1.14.
- [Daml Profiler] The Daml profiler is now a stable feature.
[Docs] Improvements to the documentation with regards to offsets
[Docs] Document how to use the Java codegen using the Daml assistant
```
CHANGELOG_BEGIN
CHANGELOG_END
* 1.16 not 1.15
changelog_begin
changelog_end
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Add a benchmark for contract insertion in the JSON API
Unfortunately the results seem to match up with my initial benchmark
in #10234
Benchmark (batchSize) (batches) (numContracts) Mode Cnt Score Error Units
InsertBenchmark.run 1000 1 1000 avgt 5 336.674 ± 42.058 ms/op
InsertBenchmark.run 1000 3 1000 avgt 5 787.086 ± 223.018 ms/op
InsertBenchmark.run 1000 5 1000 avgt 5 1181.041 ± 317.017 ms/op
InsertBenchmark.run 1000 7 1000 avgt 5 1531.185 ± 341.060 ms/op
InsertBenchmark.run 1000 9 1000 avgt 5 1945.345 ± 436.352 ms/op
Score should ideally be more or less constant but it goes up very
significantly as the total ACS size changes
fixes#10245
changelog_begin
changelog_end
* throughput -> average time
changelog_begin
changelog_end
This has now screwed us over for two releases (1.14 and currently
blocking 1.15) because we didn’t backport the change. While we could
backport this, it is annoying and provides little to no benefit given
that a failure here is harmless so let’s just ignore failures here.
changelog_begin
changelog_end
* Add a ContractDao benchmark
This PR adds a simple benchmark that uses the ContractDao directly and
is therefore a bit more fine-grained and easier to analyze than the
gatling benchmarks. I expect we’ll want to extend this, this really
only tests queries on reasonably large size ACS filtered by party but
let’s start somewhere.
fixes#10247
changelog_begin
changelog_end
* Factorize
changelog_begin
changelog_end
I don't see a reason why it's part of the participant state API, and
it definitely doesn't need to change between v1 and v2.
CHANGELOG_BEGIN
- [Integration Kit] The class ``SeedService`` has been moved from the
*participant-state* Maven package to the *participant-integration-api*
Maven package, under the Java package name
``com.daml.platform.apiserver`` to reflect its usage by the API
server, not the participant state API. If you use this class directly,
you will need to change your imports.
CHANGELOG_END
* ledger-on-sql: Abstract out the log entry ID allocator.
This is so we don't need to depend on `SeedService`.
CHANGELOG_BEGIN
CHANGELOG_END
* kvutils: Remove `allocateRandomLogEntryId` from production code.
* kvutils: Remove `SubmissionValidator.create`, as it's only for tests.
And remove default parameters for
`SubmissionValidator.createForTimeMode`.
* ledger-on-sql: Use the random log entry ID allocator.
In Sandbox, we still use the seed service, but this is not necessary
for ledger-on-sql tests.
* kvutils: Make `LogEntryIdAllocator.random` an `object`.
* ledger-on-sql: Move `RandomLogEntryIdAllocator` here.
It's not used anywhere else.
* ledger-on-sql: Add some documentation about the application code.
Just to make it clear it's not used in production.
* Separate traces from warnings in engine.
I decided to separate the engine warnings from the tracelog after all,
because I think it will make testing and maintenance easier in the
long run.
Part of #9947, follow up from #10179
changelog_begin
changelog_end
* scalafmt
* Apply suggestions from code review
Co-authored-by: Remy <remy.haemmerle@daml.com>
* dont use case class for WarningLog
* revert TraceLog changes from last PR
* Scala 2.12 doesnt have ArrayBuffer.addOne :(
* remove isWarnEnabled check
Co-authored-by: Remy <remy.haemmerle@daml.com>
* [JSON-API] Correctly extract the request source URL/IP
changelog_begin
- [JSON-API] If the service is put behind a proxy filling either of these headers X-Forwarded-For & X-Real-Ip then these will now be respected for logging the request source ip/url
changelog_end
* Return to the simple http server start code
* Remove unused import
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/Endpoints.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
