The flag allows the sandboxes (both classic and next) to use preview
version of the upcoming Daml-LF version (currently LF 1.11).
CHANGELOG_BEGIN
* [Sandbox-classic] add a flag to allow early access to the next
Daml-LF to be released.
* [Sandbox] add a flag to allow early access to the next Daml-LF
to be released.
CHANGELOG_END
This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.
As announced, this will be merged on Saturday to avoid too many conflicts.
changelog_begin
changelog_end
* Replace many occurrences of DAML with Daml
* Update docs logo
* A few more CLI occurrences
CHANGELOG_BEGIN
- Change DAML capitalization and docs logo
CHANGELOG_END
* Fix some over-eager replacements
* A few mor occurrences in md files
* Address comments in *.proto files
* Change case in comments and strings in .ts files
* Revert changes to frozen proto files
* Also revert LF 1.11
* Update get-daml.sh
* Update windows installer
* Include .py files
* Include comments in .daml files
* More instances in the assistant CLI
* some more help texts
This commit fixes a few copyright headers that have been missed in the
automatic update on Jan 1, as well as the generation code in the compat
workspace so it generates the right headers.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-integration-api: Dedicated execution context for requests.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-integration-api: Construct the services executor outside.
* participant-integration-api: Share the services EC with the GRPC stack.
* participant-integration-api: Use the new EC wherever possible.
And stop using DirectExecutionContext.
* sandbox-classic: Fix DevModeIT to use the right configuration.
I have no idea why this breaks now, but it was always technically
broken, so I'm fixing it.
* participant-integration-api: Push the services EC up one more level.
* Update newly added ApiParticipantPruning service accordingly
see https://github.com/digital-asset/daml/pull/7988#discussion_r525319097
* participant-integration-api: Ensure the LedgerConfigProvider is ready.
Somehow this slipped through.
* sandbox-classic: Fix `DefaultConfig` in tests.
* language-support/java: Disable seeding in tests again.
Co-authored-by: Oliver Seeliger <oliver.seeliger@digitalasset.com>
This is necessary to at least attempt an upgrade to 2.13 and
generally, I want to keep our rulesets up2date. rules-scala forces the
version of scalatest so we have to bump that at the same time.
This requires changes to basically all Scala test suites since the
import structure has changed and a bunch of things (primarily
scalacheck support) got split out.
Apologies for the giant PR, I don’t see a way to keep it smaller.
changelog_begin
changelog_end
* Deprecate Sandbox persistence
changelog_begin
changelog_end
* Update docs/source/tools/sandbox.rst
Co-authored-by: Bernhard Elsner <40762178+bame-da@users.noreply.github.com>
* Include in component statuses
changelog_begin
changelog_end
* Fix daml on sql Name param
changelog_begin
changelog_end
* make it compile
changelog_begin
changelog_end
Co-authored-by: Bernhard Elsner <40762178+bame-da@users.noreply.github.com>
Adds TLS certificates revocation checking in the LedgerApiServer using the OCSP.
The feature is implemented by setting global JVM properties.
Integration tests for the new feature include spinning up a local OCSP responder using the openssl command.
CHANGELOG_BEGIN
- new CLI option --cert-revocation-checking for enabling the TLS certificate revocation checking in the LedgerApiServer
- documentation about the new feature
CHANGELOG_END
* Suppress debug and trace logging for Sandbox tests.
* participant-integration-api: Don't submit config after shutting down.
* sandbox: When testing the reset service, wait for the ledger config.
If we don't wait for the ledger configuration, we could potentially ask
for completions before the server is fully ready.
* sandbox-common: Reduce the number of reset attempts in testing.
Now we're also checking for ledger configuration, the time for a reset
to be "accepted" is longer, so 5 tries in 30 seconds is a bit flaky.
Reducing to 4 seems reasonable.
* sandbox: Stop marking the reset service tests as flaky.
I don't have conclusive proof that they're no longer flaky, but I'm
fairly confident. We can revert this commit if they turn out to still
need some attention.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox-common: Check the configuration shows up in reset service tests.
* sandbox-common: Factor out logback-test.xml.
This can cause an issue because we cannot register JMX metrics twice. We
end up with fun exceptions.
The error looks like this:
```
14:37:26.230 [sandbox-worker-grpc-event-loop-e20f0542-e2c0-452d-bf7c-0ba58cda6145-31-1] DEBUG com.codahale.metrics.jmx.JmxReporter - Unable to register timer
javax.management.InstanceAlreadyExistsException: com.daml.platform.sandbox.SandboxServer:name=daml.lapi.reset_service.reset,type=timers
```
Seen in a flaky test on CI.
CHANGELOG_BEGIN
CHANGELOG_END
* resources: Move builders into //ledger/ledger-resources.
Keep the actual constructors in a trait, but instantiate it when working
with ledger code.
This allows us to later introduce an extra "context" type parameter to
ResourceOwner.
* resources-akka: Move the builders in to //ledger/ledger-resources.
* resources: Introduce an abstract `Context` parameter for owners.
This replaces the concrete `ExecutionContext`. While it _can_ be an
execution context, it really doesn't matter as long as we can get at one
somehow.
This is being introduced so we can wrap the context in a container,
either for type tagging or to include extra information.
Because our current context _is_ `ExecutionContext`, and an implicit is
provided to extract it, we can end up with two ways to get the same
value. We use shadowing to prevent this. This problem should go away in
the near future when a new context type is added.
CHANGELOG_BEGIN
- [Integration Kit] The `ResourceOwner` type is now parameterized by a
`Context`, which is filled in by the corresponding `Context` class in
the _ledger-resources_ dependency. This allows us to pass extra
information through resource acquisition.
CHANGELOG_END
* ledger-resources: Move `ResourceOwner` here from `resources`.
* ledger-resources: Remove dependencies from outside //ledger.
* ledger-resource: Wrap the acquisition execution context in `Context`.
So we can add a logging context to it.
* resources: Pass the Context, not the ExecutionContext, to Resource.
* Avoid importing `HasExecutionContext`.
* ledger-resources: Publish to Maven Central.
* resources: Make the small changes suggested by @stefanobaghino-da.
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* ledger-resources: Pull out a trait for test resource contexts.
Saves a few lines of code.
* Restore some imports that were accidentally wildcarded.
* resources: Replace an `implicit def` with a couple of imports.
* participant-integration-api: Simplify the JdbcLedgerDaoBackend tests.
Try and use the right execution context where possible.
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
Make the hardcoded timeout for party allocaction/package upload configurable
This is a short term fix to remediate issues with uploading packages
that take a considerable amount of time to decode and validate and
therefore exhausting the 30 seconds.
Adding a maximum record parameter to the ledger API like we already have
for the config management service is not as straight forward for the
package upload, because one has to account for the time in transit as
well. This topic needs further analysis, but in the meantime making the
timeout configurable and setting the default to 2 minutes should provide
enough headroom to alleviate existing issues with package upload timing.
Contributes to #6880
CHANGELOG_BEGIN
[Integration Kit]: The hardcoded timeout for party
allocation and package uploads in the Ledger API Server can be configured via ParticipantConfig and
the default value is now set to 2 minutes. See
`issue #6880 <https://github.com/digital-asset/daml/issues/6880>`__.
CHANGELOG_END
* participant-integration-api: Inject health checks into the API server.
CHANGELOG_BEGIN
- [Integration Kit] The ``StandaloneApiServer`` now takes a
``healthChecks`` parameter, which should identify the health checks to
be exposed over the gRPC Health Checking Protocol. This will
typically look something like::
healthChecks = new HealthChecks("read" -> readService, "write" -> writeService)
Integrators may also wish to expose the health of more components.
All components wishing to report their health must implement the
``ReportsHealth`` trait.
CHANGELOG_END
* sandbox + kvutils: Add the "read" component back to the health checks.
* sandbox: Use a different H2 database for every test.
Previously tests would share an index database, but not a ledger
database, if not explicitly overridden. This is _very bad_.
Fortunately all tests did specify a backend explicitly, but it shouldn't
be necessary.
I caught this while writing a new test using `SandboxNextFixture`.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox-common: Simplify SandboxBackend a little.
This reverts commit 87a0ee9383.
CHANGELOG_BEGIN
WARNING remove the changelog entry from 87a0ee93 and starting by :
"[Sandbox] By default Sandbox rejects the development versions of ..."
CHANGELOG_END
Originally those two options were set up by calling state mutating
method on a build engine.
In this PR, we move this two options to the recently introduced EngineConfig.
CHANGELOG_BEGIN
CHANGELOG_END
CHANGELOG_BEGIN
* [Sandbox] By default Sandbox rejects the development versions of
DAML-LF and transaction format. One has to explicitly use the
command line option `--dev-mode` to allows those versions.
CHANGELOG_END
* daml-on-sql: Pull out a new `Main` object that wraps sandbox-classic.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-on-sql: Fail if a JDBC URL is not provided or not for PostgreSQL.
* sandbox-classic: Rename the conformance test H2 database.
* daml-on-sql + sandbox-classic: Report configuration errors cleanly.
This means letting `ProgramResource` catch the errors, log, and exit.
* daml-on-sql: Change the name logged on startup.
* daml-on-sql: Change the default participant ID.
* sandbox-common: Give the ledger name its own tagged string type.
* sandbox-classic: Generate random ledger IDs using the ledger name.
* daml-on-sql: Remove the banner, replacing it with a blank line.
* daml-on-sql: Enable strong seeding by default.
And weak seeding in the conformance tests.
* sandbox-classic: Move the ledger name to a separate parameter.
It's not really configurable.
* sandbox-classic: Move LedgerName from sandbox-common.
* daml-on-sql: Remove "-participant" from the participant ID.
* daml-on-sql: Use `Name` where possible.
* daml-on-sql: Make the ledger ID mandatory.
* Revert "sandbox-classic: Move LedgerName from sandbox-common."
This reverts commit 0dad1584a7.
* daml-on-sql: Print "DAML-on-SQL" in the CLI help, not "Sandbox".
* daml-on-sql + sandbox + sandbox-classic: Split out custom CLI parsing. (#6846)
* participant-state: Simplify naming the seeding modes.
* sandbox-common: Move the default Sandbox participant ID here.
* sandbox-common: Move the participant ID into the config.
* sandbox-common: Allow the user to specify the participant ID in the CLI.
CHANGELOG_BEGIN
- [Sandbox] Allow the user to specify the participant ID with the
``--participant-id`` switch.
CHANGELOG_END
* sandbox-common: Move the ledgerIdMode up in the configuration.
* sandbox-classic: Move the configuration from sandbox-common.
CHANGELOG_BEGIN
CHANGELOG_END
* Fix miscellaneous warnings caught by IntelliJ IDEA.
* Move public code into daml-integration-api
CHANGELOG_BEGIN
[DAML Integration Kit]: Removed sandbox specific code from the API intended to be used by ledger integrations. Use the maven coordinates ``com.daml:participant-integration-api:VERSION`` instead of ``com.daml:ledger-api-server`` or ``com.daml:sandbox``.
CHANGELOG_END
Also fixes#5635, removing usage of `ReadService` from the `StandaloneApiServer`.
The configuration stream of the Ledger API LedgerConfigurationService is now properly backed by the configuration entries instead of just serving the initial configuration.
CHANGELOG_BEGIN
[DAML Integration Kit]: ``StandaloneApiServer`` can now be run in a read-only mode.
- The type of the constructor parameter ``writeService`` of ``StandaloneApiServer`` changed to ``Option[WriteService]``. Passing ``None`` will not start any of the admin services, the command service, and the command submission service.
- The constructor parameter ``readService`` of ``StandaloneApiServer`` has been removed.
- A new constructor parameter ``ledgerId`` has been added to ``StandaloneApiServer``. It is used to verify that that ``StandaloneApiServer`` is run against an index storage for the same ledgerId. Initialization is aborted if this is not the case.
[DAML Integration Kit]: The ``LedgerConfigurationService`` now properly streams configuration changes.
CHANGELOG_END
* fixing flaky test, WIP
* introducing `storeSync`
to guarantee that transactions get processed in the specified order
removing unnecessary assertion that hid the fact that the order of offsets
read from DB was invalid
* cleaning up
changelog_begin
changelog_end
* addressing code review comments, thanks @S11001001!
* suspend it
* cleaning up a bit more
SubmittedTransaction and CommittedTransaction are moved from com.daml.lf.transaction.Transaction to
com.daml.lf.transaction
This helps intelliJ type inference.
CHANGELOG_BEGIN
CHANGELOG_END
* adding a test case for the offset order returning expected transactions,
moving the exiting test case under flat transactions behavior group
* Fixing event_offset to event_sequential_id conversion
handling the conversion on an empty ledger,
adding a bit more logging,
skipping DB query on an empty range
CHANGELOG_BEGIN
CHANGELOG_END
* addressing code review comments
simplifying the query
returning SQL that forces the event_offset index usage
* addressing code review comments
simplifying the query
returning SQL that forces the event_offset index usage
* simplifying it a bit more
[Sandbox][Ledger Integration Kit]: Metrics can now be exported to graphite with a prefix using the format ``--metrics-reporter=graphite://host:port/prefix``.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Move helpers in EmptyLedgerIT to a companion object.
* sandbox: Accept a new time model when none is set.
CHANGELOG_BEGIN
- [Ledger API Server + (Sandbox / Ledger Integration Kit)]
Accept a new time model if none is set. Previously, it would
erroneously be rejected because the generation number submitted to
was incorrectly set to `2` rather than `1`.
This would not affect most users of Sandbox or other kvutils-based
ledgers, as if a configuration is set automatically on startup when
creating a new ledger. It only affects users who explicitly override
the initial ledger configuration submit delay to something longer than
a few milliseconds.
CHANGELOG_END
* sandbox: Ensure that completions can be streamed from the beginning.
* sandbox: In EmptyLedgerIT, use the submission service.
* sandbox: Split EmptyLedgerIT into two.
* sandbox: In CompletionServiceWithEmptyLedgerIT, assume CI will be slow.
In other words, don't create a completion stream and expect a submission
to show up in a reasonable amount of time.
* row_id changes
* fixing inserts
* replacing offset with row_id in the flat transaction stream queries
* fixing flat transaction query, updating H2 migration script
* fixing formatting
* ACS query pagination relies on row_id instead of ledger offset
* give a name to the index that we have to drop
* give a name to the index
* Fixing events range query it can return SQL nulls on empty DB.
* remove the debug println
* remove outdated comment
* removing unused orderByColumns constant
* getting rid of new `Source.flatMapConcat` calls that were added as part of this PR.
CHANGELOG_BEGIN
1. ACS, Flat Transaction and Transaction Tree stream pagination based on event_sequential_id instead of event_offset.
2. Events ordering based on the order of insertion: order by event_sequential_id instead of order by (event_offset, transaction_id, node_index).
CHANGELOG_END
* reverting changes to V13 H2 migration script,
figuring out the name of the index that has to be dropped
* Addressing code review comments:
- replacing scalaz Option.cata with stdlib Option.fold
- moving implicit val def into import
* Addressing code review comments:
- extracting re-usable stream query functions
* forcing postgres to use index when looking up lower and upper bound row ids
* fixing the query when it is run on an empty ledger
* resolving rebase conflicts
* Update ledger/sandbox/src/main/scala/com/digitalasset/platform/store/dao/events/EventsRange.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* fetching a single row, fetchSize should NOT matter
* Adding integration test to reproduce invalid order of archived, created events
The test fails, which is expected.
* Fixing the order of archived, created events triggered by exercise
* Addressing code review comments and cleaning up
* Renaming row_id to event_sequential_id
* Investigating flaky tests
* Fixing formatting
* Revert HOTFIX-flaky-client-server changes
`bazel test --runs_per_test=50 //ledger/participant-state/kvutils:reference-ledger-dump` passed on this branch.
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Return the ledger beginning as the ledger end on an empty ledger
Previously, the services would return `Offset.beforeBegin` to the
clients, and the other services deem this offset to be invalid.
CHANGELOG_BEGIN
CHANGELOG_END
* Apply suggestions from code review
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Import DurationInt
* Kill unused import
* Use ApiOffset for the external ledger begin
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* replace traverseU and sequenceU with traverse and sequence
- with -Ypartial-unification on, the extra Unapply typeclass lookup is
unnecessary
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* limit imports; we only need *> and void
small clean up in the testing library TransactionBuilder, bsaically add a Submitted and a Committed version for empty transaction constant and build method.
plus a bit of cleaning in tests using TransactionBuilder
CHANGELOG_BEGIN
CHANGELOG_END
Since we have only absolute contract ids, there is no more distinction
between local and global disclosure/divulgence. Let's please remove the
`global` prefix since it causes confusion (at least for me it did).
CHANGELOG_BEGIN
CHANGELOG_END
* Measure time of interpretation and store it in CommandExecutionResult.
* Added parameters for passing in interpretation time for WriteService and LedgerWriter.
* Code tidying.
* Added CommitMetadata parameter to LedgerWriter.
* Thread through interpretation time. Aggregate interpretation times for a batch.
* Take max of interpretation times.
* Calculate interpretation time in nanos.
* Moved CommitMetadata into separate file.
CHANGELOG_BEGIN
CHANGELOG_END
* Apply suggestions from code review
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Delegate deprecated methods to method with new signature.
* Code tidying.
* Suppress deprecation warnings.
* Made interpretation cost optional in CommitMetadata.
* Test that we populate interpretation time.
* Code tidying.
* Apply suggestions from code review
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Code tidying.
* Some more tests. Do not return interpretation cost for a batch if it only contains non-transaction submissions.
* Reformatted.
* Reformatted.
* Named arbitrary interpretation cost.
* Reverted changes for BatchingLedgerWriter.
* Always drop commit metadata for batches and don't report interpretation cost.
* More specific expectations.
* Include authorization check and blinding in interpretation time.
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Multinode ledgers reflect whether party is non-local (#6382)
* Multinode ledgers reflect whether party is non-local
Closes#2026
The new `PartyManagement` test `PMListKnowPartiesIsLocal` checks whether any known non-local
party is either not known or marked with `PartyDetails.isLocal == false`. This check is not
run for single-participant ledger setups. The test also verifies that `PartyDetails.displayName`
is preserved.
CHANGELOG_BEGIN
- [Ledger Api] The Package Management Service's `ListKnownParties` response's `PartyDetails` now
properly reflects where a party is non-local on distributed, multi-participant ledgers that
expose parties to remote participants.
CHANGELOG_END
* Review feedback from Stefano and sql fix from Ratko
* Simplify #6382 (#6413)
* Simplify https://github.com/digital-asset/daml/pull/6382
changelog_begin
changelog_end
* Test fixes switching alpha/beta and alice/bob
* getParties don't seem to become available
when run against statically created parties until a transaction is run:
when testing
bazel test //ledger/sandbox:next-conformance-test-wall-clock-time-postgresql
Adding transaction to make the test framework happy
Co-authored-by: Oliver Seeliger <oliver.seeliger@digitalasset.com>
* More review feedback
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Accommodate implicitly created, local parties
* Merge fix
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
So far, these two flags only worked in sandbox classic because I wasn't
aware the code path consuming the options is not shared between both
implementation. Now, sandbox next is on par with sandbox classic.
CHANGELOG_BEGIN
CHANGELOG_END
* Bump Flyway version to 6.5
Prevents incurring into https://github.com/flyway/flyway/issues/2759 (which was apparently solved in 6.4.0)
changelog_begin
changelog_end
* Comply with changed method signature
* sandbox: Add a command line flag to disable DAML stack traces
The sandbox now accepts a `--stack-traces no` flag which will turn off
the location tracking in DAML Engine required to produce stack traces
for failing DAML code.
Benchmarks suggest that DAML Engine spends about 10% of its time with
tracking locations. Thus, this flag will give us roughly a 1.1x
speedup when stack traces are not needed.
This flag is still hidden because we would like to validate its
usefulness before we commit to supporting it.
CHANGELOG_BEGIN
CHANGELOG_END
* Make it more obvious where we're overriding methods
CHANGELOG_BEGIN
CHANGELOG_END
* Improve help text
* sandbox: Create proper `ResourceOwner` implementations.
This allows us to use the resource acquisition execution context, rather
than using DirectExecutionContext.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Simplify the construction of SqlLedger.
* sandbox: Inject the dispatcher into the BaseLedger.
* sandbox: Make sure the SqlLedger objects are closed.
Tiny regression. No one noticed. It's OK.
* sandbox: Simplify ReadOnlySqlLedger.
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* sandbox: Pull out functions to make SqlLedger.Owner easier to read.
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* ledger-api-common: Factor out wrapping Dispatcher in a ResourceOwner.
* sandbox: Move the PersistenceQueue into a ResourceOwner.
* ledger-api-common: Add a comma.
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
