This change adds support of append-only schema in sandbox-classic and daml-on-sql ledgers: this is available with feature flag.
The support is PoC grade, it will be stabilized/productionized in the upcoming epic.
Currently enabled CI tests in the respective projects guarding this implementation.
* Introduce SequentialWriteDao for simplified indexing in sandbox-classic
* Use this in appendonly.JdbcLedgerDao to implement necessary methods
* Add support for ledgerEnd query to StorageBackend
* Fix JdbcLedgerDao creation (supporting append-only)
* Add feature flag and wiring for sandbox-classic
* Activate conformance tests with append-only on sandbox-classic
* Add support/ci coverage for daml-on-sql
changelog_begin
changelog_end
* Move Daml Profiler to EE version of sandbox/sandbox-classic
This splits Sandbox targets into EE/CE targets and exposes the option
in the EE version. The option still exists in the CE option for now
until we have released EE artifacts to not break users that might know
about it without an alternative.
There is also a small test that makes sure that this actually works
since classpaths are dumb and it didn’t work at first.
changelog_begin
changelog_end
* Fix publish target
changelog_begin
changelog_end
* Publish transitive dep
changelog_begin
changelog_end
* I hate bash
changelog_begin
changelog_end
* Ledger API test tool: test against legacy and preview version.
+ use dictionary instead of alias to map version keywords to LF version
CHANGELOG_BEGIN
CHANGELOG_END
improve previous generalization from #8695
- use lf version instead keyword (like 'stable', 'latest', 'dev') to
tag actual target. This will allow two keywords to map to the same
versions without doing twice the compilation/test work.
- use alias to map keywords tag target to versioned tag target.
- move package manage dar to test_commong.
CHANGELOG_BEGIN
CHANGELOG_END
* Enable migration only option for startup
CHANGELOG_BEGIN
[Daml Driver for PostgreSQL] Enable migrate only option at startup
CHANGELOG_END
* scalafmt
* Rationalize names of the sql start modes
* no op ledger
* scalafmt
* WIP: choose to run migrationOnly in Main
* revert changes to startup options for sandboxnext
* add sql-start-mode parameter to CLI, validate it
* scalafmt
* Update documentation
* delete some WIP experiments
* removed unused import
* Reorder migrate to properly initialize dao
* use lowercase delimitted options rather than initial caps for sql-start-mode options
* remove redundant parens
* exit after running migration scripts
* return future for flyway migration, only exit on complete, use cached threadpool execution context
* scalafmt
* correct options in docs to lowercase dash delimited
[ledger-api-server] Make database connection pool size configurable
CHANGELOG_BEGIN
[Daml Driver for PostgreSQL]: Added CLI option --database-connection-pool-size
to configure the size of the database connection pool
[Integration Kit]: Added the CLI options api-server-connection-pool-size and
indexer-server-connection-pool-size to configure the database connection pool size
for the Ledger API Server and the indexer respectively.
CHANGELOG_END
* Removed logging JDBC url to prevent leaking passwords to log files.
CHANGELOG_BEGIN
- not logging JDBC url
CHANGELOG_END
* Minor change to fix the build
* Port parts of //ledger/... to Scala 2.13
Fairly random choice of directories, I just went through them in
alphabetical order. The one thing that I had to disable for now are
the conformance tests since the ledger API test tool has a dependency
not compatible with Scala 2.13.
changelog_begin
changelog_end
* Remove accidentally included //ledger/ledger-api-client/...
doesn’t actually work yet
changelog_begin
changelog_end
This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.
As announced, this will be merged on Saturday to avoid too many conflicts.
changelog_begin
changelog_end
* Replace many occurrences of DAML with Daml
* Update docs logo
* A few more CLI occurrences
CHANGELOG_BEGIN
- Change DAML capitalization and docs logo
CHANGELOG_END
* Fix some over-eager replacements
* A few mor occurrences in md files
* Address comments in *.proto files
* Change case in comments and strings in .ts files
* Revert changes to frozen proto files
* Also revert LF 1.11
* Update get-daml.sh
* Update windows installer
* Include .py files
* Include comments in .daml files
* More instances in the assistant CLI
* some more help texts
* Remove single-party check
CHANGELOG_BEGIN
- [Ledger API] The ledger API now supports multi-party submissions.
In order to use multi-party submissions, use the new act_as and
read_as fields in submission requests.
CHANGELOG_END
* Remove usage of temporary SubmitterInfo methods
* Fix validator tests
* Fix auth test
* Add multi-party tests to the ledger API
* Fix compile errors
* Improve tests
* Remove temporary single-party method from SubmitterInfo
* Remove temporary single-party method from SubmitterInfo companion object
* Remove temporary single-party method from TxEntry
* Run multi-party submission ITs for ledger-on-memory and ledger-on-sql
* Minor improvement
Co-authored-by: Kamil Bozek <kamil.bozek@digitalasset.com>
* Deprecate Sandbox persistence
changelog_begin
changelog_end
* Update docs/source/tools/sandbox.rst
Co-authored-by: Bernhard Elsner <40762178+bame-da@users.noreply.github.com>
* Include in component statuses
changelog_begin
changelog_end
* Fix daml on sql Name param
changelog_begin
changelog_end
* make it compile
changelog_begin
changelog_end
Co-authored-by: Bernhard Elsner <40762178+bame-da@users.noreply.github.com>
Adds TLS certificates revocation checking in the LedgerApiServer using the OCSP.
The feature is implemented by setting global JVM properties.
Integration tests for the new feature include spinning up a local OCSP responder using the openssl command.
CHANGELOG_BEGIN
- new CLI option --cert-revocation-checking for enabling the TLS certificate revocation checking in the LedgerApiServer
- documentation about the new feature
CHANGELOG_END
* Participant pruning ledger api server support ported from Canton
CHANGELOG_BEGIN
- [Ledger API]: The preview of `ParticipantPruningService` enables ledger participants to prune the "front" of ledger state at the participant including the ledger api server index.
CHANGELOG_END
* Review feedback from Stefano
* Add pruning tests plus missed command completions change
* Review feedback from Robert
* Improved test readability by having populate helper return offsets
* Review feedback
* Ledger api changes to pruning api and disable canton pruning test
- Change return result to PruneResponse
- Change type of PruneRequest.prune_up_to to string
* Review feedback: Use ApiOffsetSConverter for logged offsets
* [DAML on SQL] Disallow --implicit-party-allocation
CHANGELOG_BEGIN
- [DAML on SQL] Disallow --implicit-party-allocation
CHANGELOG_END
* Don't check for implicit party allocation cmdline as it is unsupported
* Remove --open-world and --exclude=ClosedWorldIT from tests
* Do not override `implicitPartyAllocation` in `SandboxServer`
* Validate party allocation in SqlLedger (PostCommitValidation)
* Validate party allocation only for DAML on SQL
* Fix build
* Name arguments in call to SandboxIndexAndWriteService.postgres
* Make validatePartyAllocation = false explicit in SqlLedgerSpec
* Support --sql-backend-jdbcurl-env
CHANGELOG_BEGIN
- [DAML on SQL] Support passing the PostgreSQL JDBC URL through an environment variable via `--sql-backend-jdbcurl-env`
CHANGELOG_END
* Fix cleanup logic
* Improve cmdline help
* Check that the environment variable is defined
* resources: Move builders into //ledger/ledger-resources.
Keep the actual constructors in a trait, but instantiate it when working
with ledger code.
This allows us to later introduce an extra "context" type parameter to
ResourceOwner.
* resources-akka: Move the builders in to //ledger/ledger-resources.
* resources: Introduce an abstract `Context` parameter for owners.
This replaces the concrete `ExecutionContext`. While it _can_ be an
execution context, it really doesn't matter as long as we can get at one
somehow.
This is being introduced so we can wrap the context in a container,
either for type tagging or to include extra information.
Because our current context _is_ `ExecutionContext`, and an implicit is
provided to extract it, we can end up with two ways to get the same
value. We use shadowing to prevent this. This problem should go away in
the near future when a new context type is added.
CHANGELOG_BEGIN
- [Integration Kit] The `ResourceOwner` type is now parameterized by a
`Context`, which is filled in by the corresponding `Context` class in
the _ledger-resources_ dependency. This allows us to pass extra
information through resource acquisition.
CHANGELOG_END
* ledger-resources: Move `ResourceOwner` here from `resources`.
* ledger-resources: Remove dependencies from outside //ledger.
* ledger-resource: Wrap the acquisition execution context in `Context`.
So we can add a logging context to it.
* resources: Pass the Context, not the ExecutionContext, to Resource.
* Avoid importing `HasExecutionContext`.
* ledger-resources: Publish to Maven Central.
* resources: Make the small changes suggested by @stefanobaghino-da.
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* ledger-resources: Pull out a trait for test resource contexts.
Saves a few lines of code.
* Restore some imports that were accidentally wildcarded.
* resources: Replace an `implicit def` with a couple of imports.
* participant-integration-api: Simplify the JdbcLedgerDaoBackend tests.
Try and use the right execution context where possible.
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* [KVL-519] Instrument command service queues
changelog_begin
changelog_end
* Instrument max-in-flight queue
* Document inputBuffer and maxInFlight metrics
changelog_begin
[Sandbox] New metrics tracking the pending submissions and completions on the
CommandService. Check out the Metrics session in the sandbox documentation
for more details. The new metrics are input_buffer_size, input_buffer_saturation,
max_in_flight_size and max_in_flight_saturation.
changelog_end
* Fix compilations issues (1)
* Fix title underline in docs
* Refactoring of InstrumentedSource
- Rename saturation/size to length/capacity to make it more obvious what they are.
- Move the InstrumentedSource to ledger/metrics. Fits there better, with the utilities
there already for futures. Arguable both should move into libs-scala package at some point though.
- Expand the tests and make the tests less flaky. 200 runs complete fine now.
- Inc/dec the capacity counter within InstrumentedSource.
* Add missing copyright header
* Reformat
* Update ledger/metrics/src/test/scala/com/daml/metrics/InstrumentedSourceSpec.scala
Co-authored-by: hanshoglund-da <67470727+hanshoglund-da@users.noreply.github.com>
* Fix title underline in docs (again)
Co-authored-by: Jussi Maki <jussi.maki@digitalasset.com>
Co-authored-by: hanshoglund-da <67470727+hanshoglund-da@users.noreply.github.com>
* Add metrics for concurrent commands
* Update readme
CHANGELOG_BEGIN
- [DAML on SQL] Add new metrics for measuring the number
of concurrent command executions. The metrics are:
daml.commands.submissions_running, daml.execution.total_running,
daml.execution.engine_running
CHANGELOG_END
* daml-on-sql: Formatting fixes for the README.
* daml-on-sql: Rename to "DAML on PostgreSQL" within the README.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-on-sql: Format README to make Markdown conversion nicer.
Markdown conversion:
pandoc --reference-links --reference-location=section -t markdown+backtick_code_blocks-fenced_code_attributes -o ledger/daml-on-sql/README.md ledger/daml-on-sql/README.rst && sed -i 's/\\//g' ledger/daml-on-sql/README.md
* daml-on-sql: It's "DAML for PostgreSQL", not "on PostgreSQL" now.
* daml-on-sql: Generate a Markdown README with a script.
This is easier to parse by human beings than the reStructuredText.
* daml-on-sql: Reflow the README and manipulate whitespace a little.
* daml-on-sql: Fix references to the JAR file in the README.
* daml-on-sql: Add README info about architecture, OS, and dependencies.
* daml-on-sql: Make README headings consistent in style.
* daml-on-sql: Explain the `--ledgerid` flag in the README.
* daml-on-sql: Everything is important.
* daml-on-sql: Document parameters used to tune performance.
CHANGELOG_BEGIN
- [DAML on SQL] Document the architecture and OS we test against, native
system dependencies on Linux, and parameters for performance tuning.
CHANGELOG_END
* daml-on-sql: Document architecture requirements.
* daml-on-sql + sandbox: Remove unused Bazel `load` declarations.
* daml-on-sql: Build a Docker image.
* daml-on-sql: Move the example Kubernetes file over from Sandbox.
* Add a changelog entry that was dropped.
CHANGELOG_BEGIN
* [DAML on SQL] DAML on SQL is now available with every release,
from the GitHub Releases page.
CHANGELOG_END
* Upgrade the Java Docker base image to the latest version.
* daml-on-sql: Pull out a new `Main` object that wraps sandbox-classic.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-on-sql: Fail if a JDBC URL is not provided or not for PostgreSQL.
* sandbox-classic: Rename the conformance test H2 database.
* daml-on-sql + sandbox-classic: Report configuration errors cleanly.
This means letting `ProgramResource` catch the errors, log, and exit.
* daml-on-sql: Change the name logged on startup.
* daml-on-sql: Change the default participant ID.
* sandbox-common: Give the ledger name its own tagged string type.
* sandbox-classic: Generate random ledger IDs using the ledger name.
* daml-on-sql: Remove the banner, replacing it with a blank line.
* daml-on-sql: Enable strong seeding by default.
And weak seeding in the conformance tests.
* sandbox-classic: Move the ledger name to a separate parameter.
It's not really configurable.
* sandbox-classic: Move LedgerName from sandbox-common.
* daml-on-sql: Remove "-participant" from the participant ID.
* daml-on-sql: Use `Name` where possible.
* daml-on-sql: Make the ledger ID mandatory.
* Revert "sandbox-classic: Move LedgerName from sandbox-common."
This reverts commit 0dad1584a7.
* daml-on-sql: Print "DAML-on-SQL" in the CLI help, not "Sandbox".
* daml-on-sql + sandbox + sandbox-classic: Split out custom CLI parsing. (#6846)
* participant-state: Simplify naming the seeding modes.